Robinson James A. - Natural Experiments of History

Natural Experiments of History

Natural Experiments of History

EDITED BY

Jared Diamond

James A. Robinson

THE BELKNAP PRESS OF HARVARD UNIVERSITY PRESS

Cambridge, Massachusetts London, England

Copyright 2010 by the President and Fellows of Harvard College

All rights reserved

Jacket image: Thomas Gardiner, Kororareka Beach, Bay of Islands, New Zealand, c. 1856. Courtesy of Museum of New Zealand Te Papa Tongarewa.

Jacket design: Graciela Galup

First Harvard University Press paperback edition, 2011

Library of Congress Cataloging-in-Publication Data

Natural experiments of history / edited by Jared Diamond and James A. Robinson.

p. cm.

Includes bibliographical references.

ISBN 978-0-674-03557-7 (cloth: alk. paper)

ISBN 978-0-674-06019-7 (pbk.)

1. HistoryComparative methodCase studies. 2. HistoryMethodologyCase studies. I. Diamond, Jared M. II. Robinson, James A., 1960

D16.N335 2010

907.2dc22 2009012678

Contents

JARED DIAMOND AND JAMES A. ROBINSON

PATRICK V. KIRCH

JAMES BELICH

STEPHEN HABER

JARED DIAMOND

NATHAN NUNN

ABHIJIT BANERJEE AND LAKSHMI IYER

DARON ACEMOGLU, DAVIDE CANTONI, SIMON JOHNSON, AND JAMES A. ROBINSON

JARED DIAMOND AND JAMES A. ROBINSON

JARED DIAMOND AND JAMES A. ROBINSON

The controlled and replicated laboratory experiment, in which the experimenter directly manipulates variables, is often considered the hallmark of the scientific method. It is virtually the only method employed in laboratory physical sciences and in molecular biology. Without question, this approach is uniquely powerful in establishing chains of cause and effect. That fact misleads laboratory scientists into looking down on fields of science that cannot employ manipulative experiments.

But the cruel reality is that manipulative experiments are impossible in many fields widely admitted to be sciences. That impossibility holds for any science concerned with the past, such as evolutionary biology, paleontology, epidemiology, historical geology, and astronomy; one cannot manipulate the past. In addition, when one is studying bird communities, dinosaurs, smallpox epidemics, glaciers, or other planets, manipulative experiments that are possible in the present would often be condemned as immoral and illegal; one should not kill birds or melt glaciers. One therefore has to devise other methods of doing science: that is, of observing, describing, and explaining the real world, and of setting the individual explanations within a larger framework.

A technique that frequently proves fruitful in these historical disciplines is the so-called natural experiment or the comparative method. This approach consists of comparingpreferably quantitatively and aided by statistical analysesdifferent systems that are similar in many respects but that differ with respect to the factors whose influence one wishes to study. For instance, to study the ecological effect of woodpeckers known as Red-breasted Sapsuckers on related woodpeckers known as Williamsons Sapsuckers, one can compare mountains, all of which support Williamsons Sapsuckers but some of which support Red-breasted Sapsuckers while others do not. The science of epidemiology is virtually the study of such natural experiments on human populations. As one example, we have learned which human blood groups provide resistance to smallpox, not as a result of manipulative experiments in which we inject people carrying different blood groups either with smallpox virus or with a virus-free control solution, but instead as a result of observations of people carrying different blood groups during one of the last natural smallpox epidemics in India several decades ago. Physicians who were present in a remote village at the time of the outbreak determined villagers blood groups and observed who got sick or died and who did not.

Of course, natural experiments involve many obvious pitfalls. These pitfalls include the risk that the outcome might depend on other factors that the experimenter had not thought to measure; and the risk that the true explanatory factors might be ones merely correlated with the measured factors, rather than being the measured factors themselves. These and other such difficulties are realbut so are the difficulties encountered in executing manipulative laboratory experiments or in writing noncomparative narrative accounts. An extensive literature is now available on how best to overcome these pitfalls.

For example, consider a question that is currently of much practical interest: does smoking cause cancer? It is possible to write a moving, nuanced, in-depth biography of one particular smoker who did die of cancer, but that narrative doesnt prove that smoking causes cancer in general or even that it caused that particular cancer. Some smokers dont get cancer, and some nonsmokers do get it. As we have learned, there are many other risk factors for cancer besides smoking. Hence epidemiologists routinely gather data on thousands or millions of individuals, code them not only for whether they smoke but also for their diet and many other factors, and then carry out a statistical analysis. Such studies yield familiar and now widely accepted conclusions. Yes, smoking is strongly associated with some (though not with other) forms of cancer, but one can also recognize many other causes by means of statistical analyses. Those other causes include dietary fat, dietary fiber, dietary antioxidants, sun exposure, individual air pollutants, specific chemicals in our food and water, numerous hormones, and hundreds of different genes. Hence no epidemiologist would dream of identifying the cause of cancer just by telling the story of a single patient, but one can convincingly identify many causes of cancer by comparing and statistically analyzing many people. Similar conclusions and similar pitfalls that need to be addressed apply to multicausal historical phenomena.

On reflection, one might also expect comparisons and quantitative methods and statistics to play an uncontroversial middle role in the study of history. Historians are constantly making statements of the form This changed (or increased or decreased) with time, or This was more than that, or This person did more (or less) than, or behaved differently from, that person. But merely to make such statements, without providing the underlying numbers and doing the associated statistics, is to frame the comparison without carrying it out. Already in 1979, the historian Lawrence Stone made this same point in his discussion of the role of quantification: Historians can no longer get away with saying more, less, growing, declining, all of which logically imply numerical comparisons, without ever stating explicitly the statistical basis for their assertions. It [quantification] has also made argument exclusively by example seem somewhat disreputable. Critics now demand supporting statistical evidence to show that the examples are typical, and not exceptions to the rule.

In reality, the various social sciences concerned with human societies have made uneven use of natural experiments. Although there is widespread acceptance of natural experiments in archaeology, cultural anthropology, developmental psychology, economics, economic history, political science, and sociology, in the field of human history other than economic history their use has been patchy. Some historians merely call for more use of natural experiments; others claim that other historians already do use them a lot; and still others actually do use them, though sometimes not consciously or without making full use of the methodological advantages potentially associated with this approach. But many historians do not use natural experiments at all and are skeptical or hostile to the approach, especially to systematic comparisons involving quantitative data that are analyzed statistically.