R. Barker Bausell - The Problem with Science: The Reproducibility Crisis and What to Do About It

Oxford University Press is a department of the University of Oxford. It furthers the Universitys objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark of Oxford University Press in the UK and certain other countries.

Published in the United States of America by Oxford University Press

198 Madison Avenue, New York, NY 10016, United States of America.

Oxford University Press 2021

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, by license, or under terms agreed with the appropriate reproduction rights organization. Inquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above.

You must not circulate this work in any other form and you must impose this same condition on any acquirer.

Library of Congress Cataloging-in-Publication Data

Names: Bausell, R. Barker, 1942 author.

Title: The problem with science : the reproducibility crisis and what to do about it /

R. Barker Bausell, Ph.D.

Description: New York, NY : Oxford University Press, [2021] |

Includes bibliographical references and index.

Identifiers: LCCN 2020030312 (print) | LCCN 2020030313 (ebook) |

ISBN 9780197536537 (hardback) | ISBN 9780197536551 (epub) |

ISBN 9780197536568

Subjects: LCSH: ScienceResearchMethodology.

Classification: LCC Q126.9 .B38 2021 (print) | LCC Q126.9 (ebook) |

DDC 507.2/1dc23

LC record available at https://lccn.loc.gov/2020030312

LC ebook record available at https://lccn.loc.gov/2020030313

DOI: 10.1093/ oso/ 9780197536537.001.0001

This book was written and peer reviewed by Oxford University Press before the news concerning the problem in Wuhan broke, hence no mention of COVID-19 appears in the text. Relatedly, since one of my earlier books, Snake Oil Science: The Truth About Complementary and Alternative Medicine, had been published by Oxford more than a decade ago, I had seen no need to pursue this line of inquiry further since the bulk of the evidence indicated that alternative medical therapies were little more than cleverly disguised placebos, with their positive scientific results having been facilitated by substandard experimental design, insufficient scientific training, questionable research practices, or worse. So, for this book, I chose to concentrate almost exclusively on a set of problems bedeviling mainstream science and the initiative based thereupon, one that has come to be called the reproducibility crisis.

However, as everyone is painfully aware, in 2020, all hell broke loose. The internet lit up advocating bogus therapies; the leaders of the two most powerful countries in the world, Xi Jinping and Donald Trump, advocated traditional Chinese herbals and a household cleaner, respectively; and both disparaged or ignored actual scientific results that did not support their agendas. Both world leaders also personally employed (hence served as role models for many of their citizens) unproved, preventive remedies for COVID-19: traditional Chinese herbal compounds by Xi Jinping; hydroxychloroquine (which is accompanied by dangerous side effects) by Donald Trump.

This may actually be more understandable in Xis case, since, of the two countries, China is undoubtedly the more problematic from the perspective of conducting and publishing its science. As only one example, 20 years ago, Andrew Vickerss systematic review team found that 100% of that countrys alternative medical trials (in this case acupuncture) and 99% of its conventional medical counterparts published in China were positive. And unfortunately there is credible evidence that the abysmal methodological quality of Chinese herbal medical research itself (and not coincidentally the almost universally positive results touting their efficacy) has continued to this day.

To be fair, however, science as an institution is far from blameless in democracies such as the United States. Few scientists, including research methodologists such as myself, view attempting to educate our elected officials on scientific issues as part of their civic responsibility.

So while this book was written prior to the COVID-19 pandemic, there is little in it that is not relevant to research addressing future health crises such as this (e.g., the little-appreciated and somewhat counterintuitive [but well documented] fact that early findings in a new area of inquiry often tend to be either incorrect or to report significantly greater effect sizes than follow-up studies). It is therefore my hope that one of the ultimate effects of the reproducibility crisis (which again constitutes the subject matter of this book) will be to increase the societal utility of science as well as the publics trust therein. An aspiration that will not be realized without a substantial reduction in the prevalence of the many questionable research behaviors that permit and facilitate the inane tendency for scientists to manufacture (and publish) false-positive results.

First, I would like to thank the many conscientious and gifted researchers, methodologists, and statisticians whose insightful work informed this book. I have primarily depended upon their written word and sincerely hope that I have effectively captured the essence of their positions and research. I have not listed individual scientists in this acknowledgment since I have cited or been influenced by so many that I am reluctant to single out individuals for fear of omitting anyone (or including anyone who would prefer not to be so listed).

Finally, I would like to acknowledge my excellent Oxford University Press team who were extremely helpful and competent, and without whom the book would have never seen the light of day. Joan Bossert, Vice President/Editorial Director, for her support, who saw the promise in my original manuscript, selected very helpful peer reviewers, and guided me through the revision process. Phil Velinov, Assistant Editor, who seamlessly and competently coordinated the entire process. Suma George, Editorial Manager, who oversaw production. I would also like to extend my appreciation for my former editors at Oxford: Abby Gross and a special shout-out to the retired Marion OsmunEditor Extraordinaire.

This is a story about science. Not one describing great discoveries or the geniuses who make them, but one that describes the labors of scientists who are in the process of reforming the scientific enterprise itself. The impetus for this initiative involves a long-festering problem that potentially affects the usefulness and credibility of science itself.

The problem, which has come to be known as the reproducibility crisis, affects almost all of science, not one or two individual disciplines. Like its name, the problem revolves around the emerging realization that muchperhaps mostof the science being produced cannot be reproduced. And scientific findings that do not replicate are highly suspect if not worthless.

So, three of the most easily accomplished purposes of this book are

To present credible evidence, based on the published scientific record, that there exists (and has existed for some time) a serious reproducibility crisis that threatens many, if not most, sciences;