Chambers - The seven deadly sins of psychology: a manifesto for reforming the culture of scientific practice

THE SEVEN DEADLY SINS OF PSYCHOLOGY

THE SEVEN DEADLY SINS OF PSYCHOLOGY

A Manifesto for Reforming the Culture of Scientific Practice

CHRIS CHAMBERS

PRINCETON UNIVERSITY PRESS

Princeton & Oxford

Copyright 2017 by Chris Chambers
Requests for permission to reproduce material from this work should be
sent to Permissions, Princeton University Press
Published by Princeton University Press, 41 William Street,
Princeton, New Jersey 08540
In the United Kingdom: Princeton University Press, 6 Oxford Street,
Woodstock, Oxfordshire OX20 1TR

press.princeton.edu

Artwork for the lead chapter illustrations by Anastasiya Tarasenko

All Rights Reserved

ISBN 978-0-691-15890-7

Library of Congress Control Number: 2016945498

British Library Cataloging-in-Publication Data is available

This book has been composed in Whitman and Helvetica Neue

Printed on acid-free paper.

Printed in the United States of America

1 3 5 7 9 10 8 6 4 2

For J, I and X,

for every junior scientist whose results werent good enough,

and for JD, who inspired so many but will never see what we might become

One should be able to see that things are hopeless
and yet be determined to make them otherwise.
F. Scott Fitzgerald

CONTENTS

PREFACE

This book is borne out of what I can only describe as a deep personal frustration with the working culture of psychological science. I have always thought of our professional culture as a castlea sanctuary of endeavor built long ago by our forebears. Like any home it needs constant care and attention, but instead of repairing it as we go we have allowed it to fall into a state of disrepair. The windows are dirty and opaque. The roof is leaking and wont keep out the rain for much longer. Monsters live in the dungeon.

Despite its many flaws, the castle has served me well. It sheltered me during my formative years as a junior researcher and advanced me to a position where I can now talk openly about the need for renovation. And I stress renovation because I am not suggesting we demolish our stronghold and start over. The foundations of psychology are solid, and the field has a proud legacy of discovery. Our foundersHelmholtz, Wundt, Jamesbuilt it to last.

After spending fifteen years in psychology and its cousin, cognitive neuroscience, I have nevertheless reached an unsettling conclusion. If we continue as we are then psychology will diminish as a reputable science and could very well disappear. If we ignore the warning signs now, then in a hundred years or less, psychology may be regarded as one in a long line of quaint scholarly indulgences, much as we now regard alchemy or phrenology. Our descendants will smile tolerantly at this pocket of academic antiquity, nod sagely to one another about the protoscience that was psychology, and conclude that we were subject to the limitations of the time. Of course, few sciences are likely to withstand the judgment of history, but it is by our research practices rather than our discoveries that psychology will be judged most harshly. And that judgment will be this: like so many other soft sciences, we found ourselves trapped within a culture where the appearance of science was seen as an appropriate replacement for the practice of science.

In this book Im going to show how this distortion penetrates many aspects of our professional lives as scientists. The journey will be grim in places. Using the seven deadly sins as a metaphor, I will explain how unchecked bias fools us into seeing what we want to see; how we have turned our backs on fundamental principles of the scientific method; how we treat the data we acquire as personal property rather than a public resource; how we permit academic fraud to cause untold damage to the most vulnerable members of our community; how we waste public resources on outdated forms of publishing; and how, in assessing the value of science and scientists, we have surrendered expert judgment to superficial bean counting. I will hope to convince you that in the quest for genuine understanding, we must be unflinching in recognizing these failings and relentless in fixing them.

Within each chapter, and in a separate final chapter, I will recommend various reforms that highlight two core aspects of science: transparency and reproducibility. To survive in the twenty-first century and beyond we must transform our secretive and fragile culture into a truly open and rigorous scienceone that celebrates openness as much as it appreciates innovation, that prizes robustness as much as novelty. We must recognize that the old way of doing things is no longer fit for purpose and find a new path.

At its broadest level this book is intended for anyone who is interested in the practice and culture of science. Even those with no specific interest in psychology have reasons to care about the problems we face. Malpractice in any field wastes precious public funding by pursuing lines of enquiry that may turn out to be misleading or bogus. For example, by suppressing certain types of results from the published record, we risk introducing ineffective clinical treatments for mental health conditions such as depression and schizophrenia. In the UK, where the socioeconomic impact of research is measured as part of a regular national exercise called the Research Excellence Framework (REF), psychology has also been shown to influence a wide range of real-world applications. The 2014 REF reported over 450 impact case studies where psychological research has shaped public policy or practice, including (to name just a few) the design and uptake of electric cars, strategies for minimizing exam anxiety, the development of improved police interviewing techniques that account for the limits of human memory, setting of urban speed limits based on discoveries in vision science, human factors that are important for effective space exploration, government strategies for dealing with climate change that take into account public perception of risk, and plain packaging of tobacco products.most basic roots to its most applied branches, psychology is a rich part of public life and a key to understanding many global problems; therefore the deadly sins discussed here are a problem for society as a whole.

Some of the content, particularly sections on statistical methods, will be most relevant to the recently embarked researcherthe undergraduate student, PhD student, or early-career scientistbut there are also important messages throughout the book for more senior academics who manage their own laboratories or institutions, and many issues are also relevant to journalists and science writers. To aid the accessibility of source material for different audiences I have referred as much as possible to open access literature. For articles that are not open access, a Google Scholar search of the article title will often reveal a freely available electronic copy. I have also drawn on more contemporary forms of communication, including freely available blog entries and social media.

I owe a great debt to many friends, academic colleagues, journal editors, science writers, journalists, press officers, and policy experts, for years of inspiration, critical discussions, arguments, and in some cases interviews that fed into this work, including: Rachel Adams, Chris Allen, Micah Allen, Adam Aron, Vaughan Bell, Sven Bestmann, Ananyo Bhattacharya, Dorothy Bishop, Fred Boy, Todd Braver, Bjrn Brembs, Jon Brock, Jon Butterworth, Kate Button, Iain Chalmers, David Colquhoun, Molly Crockett, Stephen Curry, Helen Czerski, Zoltan Dienes, the late Jon Driver, Malte Elson, Alex Etz, John Evans, Eva Feredoes, Matt Field, Agneta Fischer, Birte Forstmann, Fiona Fox, Andrew Gelman, Tom Hardwicke, Chris Hartgerink, Tom Hartley, Mark Haselgrove, Steven Hill, Alex Holcombe, Aidan Horner, Macartan Humphreys, Hans Ijzerman, Helen Jamieson, Alok Jha, Gabi Jiga-Boy, Ben Johnson, Rogier Kievit, James Kilner, Danil Lakens, Natalia Lawrence, Keith Laws, Katie Mack, Leah Maizey, Jason Mattingley, Rob McIntosh, Susan Michie, Candice Morey, Richard Morey, Simon Moss, Ross Mounce, Nils Mulhert, Kevin Murphy, Suresh Muthukumaraswamy, Bas Neggers, Neuroskeptic, Kia Nobre, Dave Nussbaum, Hans Op de Beeck, Ivan Oransky, Damian Pattinson, Andrew Przybylski, James Randerson, Geraint Rees, Ged Ridgway, Robert Rosenthal, Pia Rotshtein, Jeff Rouder, Elena Rusconi, Adam Rutherford, Chris Said, Ayse Saygin, Anne Scheel, Sam Schwarzkopf, Sophie Scott, Dan Simons, Jon Simons, Uri Simonsohn, Sanjay Srivastava, Mark Stokes, Petroc Sumner, Mike Taylor, Jon Tennant, Eric Turner, Carien van Reekum, Simine Vazire, Essi Viding, Solveiga Vivian-Griffiths, Matt Wall, Tony Weidberg, Robert West, Jelte Wicherts, Ed Wilding, Andrew Wilson, Tal Yarkoni, Ed Yong, and Rolf Zwaan. Sincere thanks go to Sergio Della Sala and Toby Charkin for their collaboration and fortitude in championing Registered Reports at