Speaking Justice to Power
Speaking Justice to Power
Ethical and Methodological Challenges for Evaluators
Comparative Policy Evaluation, Volume 21
Kim Forss and Mita Marra, editors
First published 2014 by Transaction Publishers
Published 2017 by Routledge
2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN
711 Third Avenue, New York, NY 10017, USA
Routledge is an imprint of the Taylor & Francis Group, an informa business
Copyright 2014 by Taylor & Francis
All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers.
Notice:
Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.
Library of Congress Catalog Number: 2014010759
Library of Congress Cataloging-in-Publication Data
Speaking justice to power : ethical and methodological challenges for evaluators / Kim Forss and Mita Marra, editors.
pages cm. (Comparative policy evaluation ; Vol. XXI)
Includes bibliographical references.
ISBN 978-1-4128-5476-4
1. Equality. 2. Evaluation. I. Forss, Kim, editor. II. Marra, Mita, editor.
HM821.S697 2014
305dc23
2014010759
ISBN 13: 978-1-4128-5476-4 (hbk)
Contents
Kathryn Newcomer
Kim Forss and Mita Marra
Mita Marra and Kim Forss
Barbara Befani
Jens Andersson
Robert Picciotto
Mita Marra
Lydia Ruprecht and Sara Callegari
John Mayne
Tom Ling
Peter Wilkins
Alison Pollard
Steve Jacob
Kim Forss
Ananda S. Millard
Maria Barrados
Maria Barrados, Kim Forss, and Peter Wilkins
Many valuable contributions articulate both evaluation theory and practice in Speaking Justice to Power: Ethical and Methodological Challenges for Evaluators. As the authors astutely note, of the three Es pursued by governments around the worldefficiency, economy, and equityboth policy analysts and evaluators have devoted more effort to measuring and evaluating the first two than to capturing equity. As the authors also recognize, the concept and target of equity entails more value considerations, ambiguity, and cultural grounding than the other two, and thus, it is more challenging for evaluators of public programs to operationalize, contextualize, measure, and judge. Even stakeholder engagement, an ubiquitously valued practice for evaluators, does not necessarily reduce, and may even increase, the challenge of fairly and consistently measuring equity.
Equity is a universally held value, and is literally on the minds of politicians, philanthropists, development practitioners, and evaluators across the world. There has been considerable critical discourse about the effectiveness of international development efforts to enhance equity, as well as the evaluation of such efforts. Two of the first three Millennium Development Goals address equity: to eradicate poverty and to erase gender inequity. However, despite many years of international development experience, there have been no breakthroughs offering program designers and evaluators easily accessible and universally applicable tools to measure and analyze equity on the ground.
The authors in this text bring their extensive evaluation experience to bear to address many key questions about equity that program designers and evaluators must address, such as:
How should evaluators develop criteria for measuring equity as they analyze both program implementation, and impacts of programs, policies, and system-change efforts? How might the principles offered by the great philosophers, such as Aristotle and Rawls, inform operationalization of equity in practice?
What different types of distinctions among people, including children and as yet unborn children, should be taken into account when measuring and valuing distribution of outcomes?
How might equity-focused distributional effects be incorporated along with other valued criteria in the theory of change models evaluators develop to inform their evaluation work?
What sorts of data should be collected and generated to analyze processes and impacts in different settings? How might such data be validated?
The authors employ grounded-theory thinking as they translate key ethical principles into their evaluation work, and then draw important lessons from their experience for others who want to incorporate measurement of equity into their evaluation practice. The wisdom of the authors guidance is enhanced because their efforts to incorporate equity into evaluation work have been conducted in vastly different geographical settingsfrom Canada to Australia to Ethiopiaand have addressed equity in interventions dealing with various social and environmental problemsfrom climate change to sexual and reproductive health challenges.
As with any provocative writing that addresses extremely challenging issues, the text probably raises more questions than it answers about how to treat equity in evaluation work. Clearly, more reflection among evaluators on how effectively they operationalize and measure equity is needed. This is perhaps the first book totally focused on this important topic, but it should not be the last. More syntheses of evaluators efforts to address equity in their work undertaken across the world and across the range of complex problems facing humanity is needed. But this book is an extremely good first effort to inspire and instruct future research.
Kathryn Newcomer
February 2014
Kim Forss and Mita Marra
Equity and Evaluation
Most of us refer to equity when we talk about public policy on tax structure, recruitment to schools, participation in elections, and access to health care, and about the way we are treated under the policy, to name but a few subjects. We often refer to equity when we talk about the private sector as well, and equity is a central concern in many civil society organizations. Equity is everywhere. But does it play an important role in the evaluation discourse? In theory, it should because one of the main criteria in the assessment of worth and merit would be equity.1 Equity is often suggested as one of the key qualifications for a just social order. Equity is concerned with grand themes in society such as the distribution of power and influence, the rights and duties of people, and the qualities of life. These are themes that have attracted the most influential thinkers since ancient times; Plato and Aristotle, Thomas Aquinas, Machiavelli, Rousseau, Smith, Marx, and in recent times, John Rawls (1971), Robert Nozick, Martha Nussbaum (2001), and Amartya Sen (1999), to name but a few. It should attractand challengeevaluators too.
Fundamental changes occur in the pursuit of equality: this was evident in the great liberal remaking of society during the long nineteenth century, the French Revolution, the Russian Revolution, the Chinese Revolutions, the anticolonial movements, the collapse of the Soviet Union and its client states, and the Arab Spring. Throughout, the notion of equity has been a formidable driving force. At the beginning of the twenty-first century, the global community rallied around the Millennium Development Goals. These are supreme international efforts in the pursuit of equity.
