EVALUATION
Seeking
Truth or Power?
Comparative Policy Evaluation
Ray C. Rist, Series Editor
Vol. 17 Evaluation: Seeking Truth or Power?
Pearl Eliadis, Jan-Eric Furubo, and Steve Jacob, editors
Vol. 16 Mind the Gap
Jos Vaessen and Frans L. Leeuw
Vol. 15 The Evidence Book
Olaf Rieper, Frans L. Leeuw, and Tom Ling, editors
Vol. 14 Making Accountability Work
Marie-Louise Bemelmans-Videc, Jeremy Lonsdale, Burt Perrin, editors With a foreword by Amitai Etzioni
Vol. 13 Open to the Public
Richard Boyle, Jonathan D. Breul, and Peter Dahler-Larsen, editors
Vol. 12 From Studies to Streams
Ray C. Rist and Nicoletta Stame, editors
Vol. 11 Quality Matters
Robert Schwartz and John Winston Mayne, editors With a foreword by Christopher Pollitt
Vol. 10 Collaboration in Public Services
Andrew Gray, Bill Jenkins, Frans Leeuw, and John Winston Mayne, editors
Vol. 9 International Atlas of Evaluation
Jan-Eric Furubo, Ray C. Rist, and Rolf Sandahl, editors
Vol. 8 Building Effective Evaluation Capacity
Richard Boyle and Donald Lemaire, editors
Vol. 7 Carrots, Sticks and Sermons
Marie-Louise Bemelmans-Videc, Ray C. Rist, and Evert Oskar Vedung, editors
Vol. 6 Public Policy and Program Evaluation
Evert Oskar Vedung
Vol. 5 Monitoring Performance in the Public Sector
John Mayne and Eduardo Zapico-Go\xF1i, editors With a foreword by Joseph S. Wholey
Vol. 4 Politics and Practices of Intergovernmental Evaluation
Olaf Rieper and Jacques Toulemonde, editors With a foreword by Ray C. Rist
Vol. 3 Can Governments Learn?
Frans L. Leeuw, Ray C. Rist, and Richard C. Sonnichsen, editors
Vol. 2 Budgeting, Auditing, and Evaluation
Andrew Gray, Bill Jenkins, and Bob Segsworth, editors
Vol. 1 Program Evaluation and the Management of Government
Ray C. Rist, editor
First published 2011 by Transaction Publishers
Published 2017 by Routledge
2 Park Square, Milton Park, Abingdon, Oxon, OX 14 4RN
711 Third Avenue, New York, NY 10017, USA
Routledge is an imprint of the Taylor & Francis Group, an informa business
All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers.
Notice:
Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.
Library of Congress Catalog Number: 2010024013
Library of Congress Cataloging-in-Publication Data
Eliadis, F. Pearl, 1959-
Evaluation : seeking truth or power? / Pearl Eliadis, Jan-Eric Furubo,
and Steven Jacob.
p. cm.
ISBN 978-1-4128-1141-5 (alk. paper)
1. Power (Social sciences) 2. Evaluation research (Social action
programs) 3. Public administration--Decision making. 4. Administrative
agencies-Auditing. I. Furubo, Jan-Eric. II. Jacob, Steve. III.
Title.
JC330.E39 2010
352.439-dc22
2010024013
ISBN 13: 978-1-4128-1141-5 (hbk)
This book is dedicated to the memory of Professor William (Bill) Jenkins from University of Kent. Bill was a long-time member and an active contributor to the intellectual life of the International Evaluation Research Group. Among his many academic achievements, he co-edited two books and provided multiple articles to INTEVALs efforts.
Evaluation:
For Public Good or Professional Power?
Jan-Eric Furubo and Ove Karlsson Vestman
The definition of the alternatives is the supreme instrument of power
E. E. Schattschneider, 1960
Evaluation has come of age. Most of us, not only evaluators, would probably have some difficulty imagining a society in which evaluation is not used on a daily basis, everywhere from school to work, and from local authorities to parliamentary committees. Evaluation is also regarded generally as something for the public good. While university researchers or public servants might think that there just might be too many types of evaluation, rankings and reviews, evaluation is nonetheless viewed positively, and evaluators are seen to be among the good guys. One way or another, evaluation is perceived as a tool for improvement, or in the words of one of the most influential theorists: Evaluators are dedicated to using their knowledge for the benefit of society (Hirschon Weiss, 2004: 153).
At the same time, evaluation has an intrinsic relationship to something more ambiguous and even ominous. It is power. Many evaluation practitioners, and certainly scholars in the field, have been very much aware of this relationship. For many, a power relationship actually underpins the rationale for evaluation. Evaluators do often say, alluding to Wildavskys famous book, that evaluation must Speak Truth to Power (Wildavsky, 1979).
It is also said that evaluation can change power relations in society and give power to powerless groups (House & Howe, 1999). Evaluation can bring knowledge to others, and knowledge is important for the exercise of power. It can change how power is allocated in a range of relationships and settings. This is an important but not very new idea. After all, it is exactly what Francis Bacon said 400 years ago when he said that knowledge is power.
While this perspective informs this book, we have added a second one, namely the degree to which evaluation seeks power for its own interests. This perspective is based on a simple assumption: If you are in possession of an asset that can give you power, why not use it for your own interests? In his discussion in this volume, Peter Dahler-Larsen expresses it very bluntly If institutions are interested in protecting and expanding their resources, autonomy, and reputation, so are evaluation institutions. This logic will surprise no political observer of institutions.
Can we really trust evaluation to be a force for the good? To what degree can we talk about self-interest in evaluation, and is this self-interest something that contradicts other interests such as the benefit of society? If so, what can be done to control this?
It is understandable that some readers will question the necessity of such critical questions about evaluation. We think two answers justify this endeavor. But first, we have to address a more fundamental question about what we actually mean by evaluation.
What Do We Mean by Evaluation?
Every reader is probably aware that there is no one authoritative definition of evaluation. It is therefore impossible to state in a precise manner which activities can be called evaluation and which cannot. It is also obvious that the term evaluation becomes more elastic the longer it is used, partly because the word itself is loaded with positive connotations that make it advantageous to use it in many situations. So, we find ourselves in a similar situation as Michael Power in
