Show Me the Evidence is a work of great insight and integrity about Obama administration initiatives to use scientific evidence on what works to guide government social spending. In a thorough and balanced account, the book depicts the vision and commitment of government officials willing to depart from the usual recipe of more spending as the answer to social problems, and to focus instead on whether money is being spent in ways that truly improve people's lives. These pages illuminate a path to effective government that transcends endless partisan debate about more spending versus less.
JON BARON, President, Coalition for Evidence-Based Policy
The quest to insert evidence into policymaking is a long, hard slog. But Haskins and Margolis tell the tale of the fight for rigor and results like investigative reporters revealing Washington's innermost secrets. In this case, the players are working to find what works and convince elected leaders to fund and expand those programs. The stories Haskins and Margolis tell in Show Me the Evidence show that it's possible to convince policymakers to make decisions based on scientific fact. The more we do that, the more American citizens will see the benefits of programs that work. This insightful and engaging account of the effort to insert rigorous evidence into the policy debate shows that progress has been made. Whether the effort can be sustained across congresses and administrations remains to be seen.
ROBERT SHEA, Principal, Grant Thornton LLC
A hallmark of the Obama administration is its evidence-based policy initiatives. Show Me the Evidence is a masterful examination of the challenges and benefits of using an evidence-based strategy to allocate scarce public funds for addressing the myriad social welfare concerns of the nation. The authors conducted case studies of the six evidence-based initiatives of the Obama administration, which ranged from programs to improve prenatal and early childhood outcomes to programs aimed at improving college and career training outcomes. The authors make a compelling case that by creating a space at the table for evidence, the Obama administration has changed the public funding dynamic in important waysit has raised the bar on evidence of promise, created greater attention to and accountability for evidence of effectiveness, and stimulated more routine use of evidence to guide program improvement. The authors also demonstrate the feasibility and power of prioritizing randomized controlled trials for generating evidence to make decisions about efficient allocations of public funds to promote social welfare.
REBECCA A. MAYNARD, University Trustee Chair
Professor of Education and Social Policy, University of Pennsylvania
SHOW ME THE
EVIDENCE
OBAMA'S FIGHT FOR RIGOR and RESULTS IN SOCIAL POLICY
RON HASKINS
GREG MARGOLIS
BROOKINGS INSTITUTION PRESS
Washington, D.C.
Copyright 2015
THE BROOKINGS INSTITUTION
1775 Massachusetts Avenue, N.W., Washington, D.C. 20036
www.brookings.edu
All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means without permission in writing from the Brookings Institution Press.
The Brookings Institution is a private nonprofit organization devoted to research, education, and publication on important issues of domestic and foreign policy. Its principal purpose is to bring the highest quality independent research and analysis to bear on current and emerging policy problems. Interpretations or conclusions in Brookings publications should be understood to be solely those of the authors.
Library of Congress Cataloging-in-Publication data
Haskins, Ron.
Show me the evidence : Obama's fight for rigor and evidence in social policy / Ron Haskins and Greg Margolis.
pages cm
Includes bibliographical references and index.
Summary: Describes development and implementation of six evidence-based social policy initiatives undertaken by the Obama administrationspanning areas such as education, teen pregnancy, employment and training, and healththat led to effective ways in the use of social science evidence to guide federal policymaking and the operation of federal grant programsProvided by publisher.
ISBN 978-0-8157-2571-8 (pbk. : alk. paper) ISBN 978-0-8157-2570-1 (e-book) 1. United StatesSocial policy21st century. 2. Evaluation research (Social action programs)United States. 3. Obama, Barack. I. Margolis, Greg. II. Title.
HN59.2.H38 2014
361.610973dc23 2014022510
9 8 7 6 5 4 3 2 1
Printed on acid-free paper
Typeset in Minion
Composition by Cynthia Stock
Silver Spring, Maryland
JIM MANZI
Foreword
Bloodletting was a common medical practice for thousands of years, used by the ancient Egyptians, Mayans, and Aztecs and by Europeans from the Middle Ages through the mid-nineteenth century. The practice, which appears to have arisen from the intuitive idea that illness is caused by blood and other bodily fluids being out of balance, has been supported by both sophisticated theories and widespread folk wisdom. It has now been all but eliminated from modern medicine, having been shown in well-designed experiments since the mid-nineteenth century to be useless in treating any but a small number of very specialized medical conditions.
It is easy to mock bloodletting now, but very smart people across millennia believed that it worked. Why was it so difficult to learn from experience that this practice was, for the most part, hurting the sick people to whom it was applied?
The idea of designing and executing a test to determine the efficacy of a therapeutic treatmentgiving patient group A one treatment and patient group B a different treatment and then observing whether one group does better than the otheris at least as old as the biblical Book of Daniel. In the tenth and eleventh centuries both Chinese and Islamic scholars recorded therapeutic trials. In the modern West, credit for the first clinical trial is generally given to James Lind, who in 1747 prescribed multiple treatments for scurvy to British sailors aboard HMS Salisbury and discovered that citrus fruit was the best available remedy.
Lind was consciously applying the experimental concept of Baconian science, which had revolutionized physics over the prior century. Major discoveries were made using this method throughout the nineteenth century. However, use of the experimental method progressed much more slowly in therapeutic biology than it had in physics, because the human body is more complex than a prism refracting light or chemicals interacting in a test tube. Therapeutic experiments for many potential treatments raised debates about whether the treatment was the only thing that varied between the test and control groups.
Around 1900 several researchers hit upon the idea of randomly assigning patients to the test and control groups, resulting in the so-called randomized controlled trial (RCT). That superficially minor change turned out to be fundamental. RCTs tell us that everything must be equivalent between the two groups if assignment is purely random, subject only to sampling error. To date, medical science has conducted several hundred thousand therapeutic RCTs, and they are central to the progress of medicine.
It was natural to attempt to apply the RCT method to social interventions like literacy, job training, and prisoner counseling programs. Those programs began in the same era as therapeutic RCTs, but the method did not turn out to be nearly as important to twentieth-century social science as it was to medicine. That was because the effects of social interventions tend to be so context dependent that we usually need to run many experiments to find the conditions under which a given program works. Running multiple experiments, in turn, requires a dramatically lower cost per test. Over the past twenty-five years or so, commercial businesses have been able to reduce costs by applying information technology to routinize and ultimately automate many aspects of testing. The technology needed to apply RCTs to evaluate, improve, and target government social welfare, education, corrections, and other programs is now available. The remaining barriers are political.
