Timothy N. Ogden - Experimental Conversations: Perspectives on Randomized Trials in Development Economics

Timothy N. Ogden, editor

The MIT Press

Cambridge, Massachusetts

London, England

2016 Massachusetts Institute of Technology

All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.

This book was set in Stone Sans and Stone Serif by Toppan Best-set Premedia Limited. Printed and bound in the United States of America.

Library of Congress Cataloging-in-Publication Data

Names: Ogden, Timothy N., editor.

Title: Experimental conversations : perspectives on randomized trials in development economics / edited by Timothy N. Ogden.

Description: Cambridge, MA : MIT Press, 2017. | Includes bibliographical references and index.

Identifiers: LCCN 2016020048 | ISBN 9780262035101 (hardcover : alk. paper)

eISBN 9780262336833

Subjects: LCSH: Economic development--Evaluation--Methodology.

Classification: LCC HD75 .E95 2017 | DDC 338.9--dc23 LC record available at https://lccn.loc.gov/2016020048

ePub Version 1.0

Dedicated to Peter Hess, my first economics professor, who pushed me to work harder, ask more questions, and seek better answers

Its difficult to say what this book is without explaining why it is. Certainly it is a book about the use of randomized control trials (RCTs) in development economics, which some have characterized as a revolution in the field. But the book is not a history or explanation of RCTs, though there is plenty of each in these pages. The aim is more than that.

Through the course of the book it should become clear that while I spend a lot of time talking to economists and have read a lot of economics papers, I am not an expert or specialist. The idea and desire to put this book together was born of that nonspecialist statusspecifically my aim to better understand, to think clearer, truer thoughts about the world, and how it might be changed for the better.

In those aims I believe I am far from unique. And thats why this book is. I am grateful that my life arc put me in a place where I could have regular one-on-one conversations with Dean Karlan, Jonathan Morduch, David McKenzie, and other engaged and thoughtful economists whose aims overlapped with mine.

These economists that I found myself in conversation with were part of a significant change in the practice of development economics, a change based in using new methods, particularly randomized control trials or RCTs to answer age-old questions about the impact of antipoverty programs. As is often the case with newcomers, I was fascinated with the novelty of what this group of economists was doing and enthusiastic about what they seemed to be accomplishing.

I found that the private conversations I was able to have with these economists about their work did not match the perceptions I saw and read among policy makers, practitioners, and commenters on development and poverty programs (at the time I was blissfully unaware of most of the debates happening within the economics profession about the value of randomized control trials). It seemed to me that a much larger group of people who cared about making the world a better place were ignorant of the work these economists were doing, or misunderstood it and its motivations. I wanted to give more people a sense of how these economists thought about their own research, that of others, and how to change the worldboth their personal theories of change and in the advice they would give to others based on their research. I wanted to give the economists the opportunity to speak to a broader audience outside of the heavily mediated forms of the academic economics paper or the mass media story, both of which contribute meaningfully to the misunderstandings and distortions I wanted to combat. I wanted to give other nonspecialists a chance to hear from the economists in their own words, to give a sense of what it is like to have a conversation with them and to explore their thinking beyond the narrow confines of a particular study or the difficult-to-penetrate esoterica of econometrics.

So this book is a collection of interviews with economists who conduct randomized control trials, usually field experiments, in development. But it also contains a series of shorter interviews with economists and others who are observers, critics, sponsors, consumers, or supporters of the movement to conduct more field experiments in the search for programs and policies that alleviate poverty. The list of interviewees is idiosyncraticit is made up of the group of people that I thought were doing interesting research or had a useful perspective but also who I believed might respond to my requests.

Being a nonspecialist does not mean I am neutral. When a colleague and I launched our own business, Innovations for Poverty Action (IPA), one of the two original hubs for the promotion of RCTs in development economics, was among our first clients. I consider Dean Karlan a friend and advised him on his book More Than Good Intentions. I have worked closely for years with another friend and interviewee, Jonathan Morduch.

My re-entry (after majoring in International Political Economy but spending a decade in the technology industry) into the world of economic development coincided with the rapid growth in the use of and attention to RCTs. As I mentioned, I was particularly captured by the new tools and new answers RCTs were generating. In other words I was something of a fan of randomized control trials in development. As I began the formal interviews for the book, I wrote a good bit advocating for and defending RCTs in development. Those defenses, and what I heard from some of the interviewees, forced me to engage more with the thoughtful critiques of the RCT movement. That in turn led to the only significant change to the book (other than it taking several years longer than I had hoped) since its conception: the inclusion of the shorter interviews to give voice to some of the critiques as well as to provide a better sense of the context in which these conversations about RCTs in development take place.

My aim was not to set up these interviews as debates or to directly address the specific points made by advocates and critics. Instead, I wanted readers to be able to hear for themselves how the parties involved think and apply research. While I ask many of interviewees about external validity in the abstract, for instance, I think the most interesting and useful parts of the interviews are not the direct answer to those abstract questions but the specific answers to how they apply, or dont apply, specific research findings in the way they think about the world. I learn more from answers that reveal what someone accepts as established fact and where they still have doubts than I do from answers about how someone approaches questions of evidence and proof in general. Thats why many of the interviews include descriptions of experiments and their results. What questions an economist chooses to take on, how they try to answer them in the design of a particular study, and how they interpret the results tells us not just about the preferences of individuals but also, I believe, valuable things about how that economist thinks about the power of theory, gaps in knowledge, what issues matter, and the nature of human beings and political and economic systemseven about their beliefs about free will. And, I believe, understanding those inputs into someones thinking can help us interpret and integrate (or reject) that economists findings into our own thinking.

The result, I hope, is a book that will serve as a valuable primary source document that helps illuminate the ideas of significant participants in the current conversation about not only the use of RCTs in development economics but how we learn about the world, what evidence is and means, and how policy should be and shouldnt be formed. Those are very broad conversations, and I have no illusions of being able to comprehensively address them in this book, though I will return to them in the Conclusion.