Steven Sloman - The Knowledge Illusion: Why We Never Think Alone

T hree soldiers sat in a bunker surrounded by three-foot-thick concrete walls, chatting about home. The conversation slowed and then stopped. The cement walls shook and the ground wobbled like Jell-O. Thirty thousand feet above them in a B-36, crew members coughed and sputtered as heat and smoke filled their cabin and dozens of lights and alarms blared. Meanwhile, eighty miles due east, the crew of a Japanese fishing trawler, the not-so-lucky Lucky Dragon Number Five (Daigo Fukury Maru), stood on deck, staring with terror and wonder at the horizon.

The date was March 1, 1954, and they were all in a remote part of the Pacific Ocean witnessing the largest explosion in the history of humankind: the detonation of a thermonuclear fusion bomb nicknamed Shrimp, code-named Castle Bravo. But something was terribly wrong. The military men, sitting in a bunker on Bikini Atoll, close to ground zero, had witnessed nuclear detonations before and had expected a shock wave to pass by about 45 seconds after the blast. Instead the earth shook. That was not supposed to happen. The crew of the B-36, flying a scientific mission to sample the fallout cloud and take radiological measurements, were supposed to be at a safe altitude, yet their plane blistered in the heat.

All these people were lucky compared to the crew of the Daigo Fukury Maru. Two hours after the blast, a cloud of fallout blew over the boat and rained radioactive debris on the fishermen for several hours. Almost immediately the crew exhibited symptoms of acute radiation sicknessbleeding gums, nausea, burnsand one of them died a few days later in a Tokyo hospital. Before the blast, the U.S. Navy had escorted several fishing vessels beyond the danger zone. But the Daigo Fukury Maru was already outside the area the Navy considered dangerous. Most distressing of all, a few hours later, the fallout cloud passed over the inhabited atolls Rongelap and Utirik, irradiating the native populations. Those people have never been the same. They were evacuated three days later after suffering acute radiation sickness and temporarily moved to another island. They were returned to the atoll three years later but were evacuated again after rates of cancer spiked. The children got the worst of it. They are still waiting to go home.

The explanation for all this horror is that the blast force was much larger than expected. The power of nuclear weapons is measured in terms of TNT equivalents. The Little Boy fission bomb dropped on Hiroshima in 1945 exploded with a force of sixteen kilotons of TNT, enough to completely obliterate much of the city and kill about 100,000 people. The scientists behind Shrimp expected it to have a blast force of about six megatons, around three hundred times as powerful as Little Boy. But Shrimp exploded with a force of fifteen megatons, nearly a thousand times as powerful as Little Boy. The scientists knew the explosion would be big, but they were off by a factor of about 3.

The error was due to a misunderstanding of the properties of one of the major components of the bomb, an element called lithium-7. Before Castle Bravo, lithium-7 was believed to be relatively inert. In fact, lithium-7 reacts strongly when bombarded with neutrons, often decaying into an unstable isotope of hydrogen, which fuses with other hydrogen atoms, giving off more neutrons and releasing a great deal of energy. Compounding the error, the teams in charge of evaluating the wind patterns failed to predict the easterly direction of winds at higher altitudes that pushed the fallout cloud over the inhabited atolls.

This story illustrates a fundamental paradox of humankind. The human mind is both genius and pathetic, brilliant and idiotic. People are capable of the most remarkable feats, achievements that defy the gods. We went from discovering the atomic nucleus in 1911 to megaton nuclear weapons in just over forty years. We have mastered fire, created democratic institutions, stood on the moon, and developed genetically modified tomatoes. And yet we are equally capable of the most remarkable demonstrations of hubris and foolhardiness. Each of us is error-prone, sometimes irrational, and often ignorant. It is incredible that humans are capable of building thermonuclear bombs. It is equally incredible that humans do in fact build thermonuclear bombs (and blow them up even when they dont fully understand how they work). It is incredible that we have developed governance systems and economies that provide the comforts of modern life even though most of us have only a vague sense of how those systems work. And yet human society works amazingly well, at least when were not irradiating native populations.

How is it that people can simultaneously bowl us over with their ingenuity and disappoint us with their ignorance? How have we mastered so much despite how limited our understanding often is? These are the questions we will try to answer in this book.

The field of cognitive science emerged in the 1950s in a noble effort to understand the workings of the human mind, the most extraordinary phenomenon in the known universe. How is thinking possible? What goes on inside the head that allows sentient beings to do math, understand their mortality, act virtuously and (sometimes) selflessly, and even do simple things, like eat with a knife and fork? No machine, and probably no other animal, is capable of these acts.

We have spent our careers studying the mind. Steven is a professor of cognitive science who has been researching this topic for over twenty-five years. Phil has a doctorate in cognitive science and is a professor of marketing whose work focuses on trying to understand how people make decisions. We have seen directly that the history of cognitive science has not been a steady march toward a conception of how the human mind is capable of amazing feats. Rather, a good chunk of what cognitive science has taught us over the years is what individual humans cant dowhat our limitations are.

The darker side of cognitive science is a series of revelations that human capacity is not all that it seems, that most people are highly constrained in how they work and what they can achieve. There are severe limits on how much information an individual can process (thats why we can forget someones name seconds after being introduced). People often lack skills that seem basic, like evaluating how risky an action is, and its not clear they can ever be learned (hence many of usone of the authors includedare absurdly scared of flying, one of the safest modes of transportation available). Perhaps most important, individual knowledge is remarkably shallow, only scratching the surface of the true complexity of the world, and yet we often dont realize how little we understand. The result is that we are often overconfident, sure we are right about things we know little about.