Stuart Ritchie - Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth

It is the peculiar and perpetual error of the human understanding to be more moved and excited by affirmatives than by negatives.

Francis Bacon, Novum Organum (1620)

January 31, 2011 was the day the world found out that undergraduate students have psychic powers.

A new scientific paper had hit the headlines: a set of laboratory experiments on over 1,000 people had found evidence for psychic precognition the ability to see into the future using extrasensory perception. This wasnt the work of some unknown crackpot: the paper was written by a top psychology professor, Daryl Bem, from the Ivy Leagues Cornell University. And it didnt appear in an obscure outlet it was published in one of the most highly regarded, mainstream, peer-reviewed psychology journals. Science seemed to have given its official approval to a phenomenon that hitherto had been considered completely impossible.

At the time, I was a PhD student, studying psychology at the University of Edinburgh. I dutifully read Bems paper. Heres how one of the experiments worked. Undergraduate students looked at a computer screen, where two images of curtains would appear. They were told that there was another picture behind one of the curtains, and that they had to click whichever they thought it was. Since they had no other information, they could only guess. After theyd chosen, the curtain disappeared and they saw whether theyd been correct. This was repeated thirty-six times, then the experiment was over. The results were quietly stunning. When a picture of some neutral, boring object like a chair was behind one of the curtains, the outcome was almost perfectly random: the students chose correctly 49.8 per cent of the time, essentially fifty-fifty. However and heres where it gets strange when one of the pictures was pornographic, the students tended to choose it slightly more often than chance: 53.1 per cent of the time, to be exact. This met the threshold for statistical significance. In his paper, Bem suggested that some unconscious, evolved, psychic sexual desire had ever-so-slightly nudged the students towards the erotic picture even before it had appeared on screen.

Some of Bems other experiments were less explicit, but no less puzzling. In one of them, a list of forty unrelated words appeared on the screen, one at a time. Afterwards came a surprise memory test, where the students had to type in as many of the words as they could remember. At that point, the computer randomly selected twenty of the words and showed them to the students again. Then the experiment ended. Bem reported that, during the memory test, the students were more likely to remember the twenty words they were about to see again, even though they couldnt have known except by psychic intuition which ones they were going to be shown. This would be a bit like studying for an exam, sitting the exam, then studying again afterwards, and that post-exam study somehow winding its way back in time to improve your grade. Unless the laws of physics had suddenly been repealed, time is supposed to run in only one direction; causes are supposed to come before, not after, their effects. But with the publication of Bems paper, these bizarre results were now a part of scientific literature.

Crucially, Bems experiments were extremely simple, requiring nothing more complicated than a desktop computer. If Bem was right, any researcher could produce evidence for the paranormal just by following his experimental instructions even a PhD student with next to no resources. That is what I was, so that is exactly what I did. I got in touch with two other psychologists who were also sceptical of the results, Richard Wiseman of the University of Hertfordshire and Chris French of Goldsmiths, University of London. We agreed to re-run Bems word-list experiment three times, once at each of our respective universities. After a few weeks of recruiting participants, waiting for them to complete the memory test and then dealing with their looks of bewilderment as we explained afterwards what wed been looking for, we had the results. They showed nothing. Our undergraduates werent psychic: there was no difference in their recall of the words presented after the test. Perhaps the laws of physics were safe after all.

We duly wrote up our results and sent the resulting paper off to the same scientific journal that had published Bems study, the Journal of Personality and Social Psychology. Almost immediately the door was slammed in our faces. The editor rejected the paper within a few days, explaining to us that they had a policy of never publishing studies that repeated a previous experiment, whether or not those studies found the same results as the original.

Were we wrong to feel aggrieved? The journal had published a paper that had made some extremely bold claims claims that, if true, werent just interesting to psychologists, but would completely revolutionise science. The results had made their way into the public domain and received significant publicity in the popular media, including an appearance by Bem on the late-night talk show The Colbert Report where the host coined the memorable phrase time-travelling porn.

Meanwhile, another case was unfolding that also raised alarming questions about the current state of scientific practice. Science, widely considered one of the worlds most prestigious scientific journals (second only to Nature), had published a paper by Diederik Stapel, a social psychologist at Tilburg University in the Netherlands. The paper, entitled Coping with Chaos, described several studies performed in the lab and on the street, finding that people showed more prejudice and endorsed more racial stereotypes when in a messier or dirtier environment.

The problem was that none of it was real. Some of Stapels colleagues became suspicious after they noticed the results of his experiments were a little too perfect. Not only that, but whereas senior academics are normally extremely busy and rely on their students to do such menial tasks as collecting data, Stapel had apparently gone out and collected all the data himself. After the colleagues brought these concerns to the university in September 2011, Stapel was suspended from his professorship. Multiple investigations followed.

In a confessional autobiography he wrote subsequently, Stapel admitted that instead of collecting the data for his studies, he would sit alone in his office or at his kitchen table late into the night, typing the numbers he required for his imaginary results into a spreadsheet, making them all up from scratch. I did some things that were terrible, maybe even disgusting, he wrote. I faked research data and invented studies that had never happened. I worked alone, knowing exactly what I was doing I didnt feel anything: no disgust, no shame, no regrets.

Stapel described printing off the blank worksheets hed ostensibly be giving to his participants, showing them to his colleagues and students, announcing he was heading off to run the study then dumping the sheets into the recycling when nobody was looking. It couldnt last. The findings of the investigations were clear; he was fired not long after his suspension. Since then, no fewer than fifty-eight of his studies have been retracted struck off the scientific record due to their fake data.