Carl T. Bergstrom - Calling Bullshit: The Art of Skepticism in a Data-Driven World

T HE WORLD IS AWASH WITH bullshit, and were drowning in it.

Politicians are unconstrained by facts. Science is conducted by press release. Silicon Valley startups elevate bullshit to high art. Colleges and universities reward bullshit over analytic thought. The majority of administrative activity seems to be little more than a sophisticated exercise in the combinatorial reassembly of bullshit. Advertisers wink conspiratorially and invite us to join them in seeing through all the bullshit. We wink backbut in doing so drop our guard and fall for the second-order bullshit they are shoveling at us. Bullshit pollutes our world by misleading people about specific issues, and it undermines our ability to trust information in general. However modest, this book is our attempt to fight back.

The philosopher Harry Frankfurt recognized that the ubiquity of bullshit is a defining characteristic of our time. His classic treatise On Bullshit begins as follows:

One of the most salient features of our culture is that there is so much bullshit. Everyone knows this. Each of us contributes his share. But we tend to take the situation for granted [yet] we have no clear understanding of what bullshit is, why there is so much of it, or what functions it serves. And we lack a conscientiously developed appreciation of what it means to us. In other words, we have no theory.

To eradicate bullshit, it is helpful to know precisely what it is. And here things get tricky.

First of all, bullshit is both a noun and a verb. Not only can I get tired of listening to your bullshit (n.), I can turn around and bullshit (v.) you myself. This is straightforward enough. To bullshit is, at a first approximation, the act of producing bullshit.

But what does the noun bullshit refer to anyway? As with many attempts to match philosophical concepts to everyday language, it would be a fools errand to attempt a definition that incorporates and excludes everything it should. Instead, we will start with a few examples and then try to describe some things that would qualify as bullshit.

Most people think theyre pretty good at spotting bullshit. That may be true when bullshit comes in the form of rhetoric or fancy language, what we call old-school bullshit. For example:

• Our collective mission is to functionalize bilateral solutions for leveraging underutilized human resource portfolio opportunities. (In other words, we are a temp agency.)
• We exist as transmissions. To embark on the myth is to become one with it. (We might call this new-age old-school bullshit.)
• Just as our forefathers before us, we look to the unending horizons of our great nation with minds fixed and hearts aflame to rekindle the dampened sparks of our collective destiny. (Spare us. How are you going bring jobs back to the district?)

Old-school bullshit doesnt seem to be going away, but it may be overshadowed by the rise of what we call new-school bullshit. New-school bullshit uses the language of math and science and statistics to create the impression of rigor and accuracy. Dubious claims are given a veneer of legitimacy by glossing them with numbers, figures, statistics, and data graphics. New-school bullshit might look something like this:

• Adjusted for currency exchange rates, our top-performing global fund beat the market in seven of the past nine years.

(How exactly were returns adjusted? How many of the companys funds failed to beat the market and by how much? For that matter, was there a single fund that beat the market in seven of nine years, or was it a different fund that beat the market in each of those seven years?)

• While short of statistical significance (p = 0.13), our results underscore the clinically important effect size (relative odds of survival at five years = 1.3) of our targeted oncotherapy and challenge the current therapeutic paradigm.

(What does it mean for a result to be clinically important if it is not statistically significant? Is five-year survival a relevant measure for this particular cancer, or are most patients deceased within three years? Why should we imagine that any of this challenges the current therapeutic paradigm?)

• The teams convolutional neural net algorithm extracts the underlying control logic from a multiplex network composed of the human metabolome, transcriptome, and proteome.

(What is a multiplex network? Why are connections among these various -omes meaningful, and how are they measured? What does the author mean by control logic? How do we know there is an underlying control logic linking these systems, and if there is, how do we know that this approach can actually capture it?)

• Our systematic screening revealed that 34 percent of behaviorally challenged second graders admit to having sniffed Magic Markers at least once in the past year.

(Why does it matter? And if it does, is marker-sniffing a cause or a consequence of being behaviorally challenged? What fraction of nonchallenged second graders admit to sniffing Magic Markers? Perhaps that fraction is even higher!)

New-school bullshit can be particularly effective because many of us dont feel qualified to challenge information that is presented in quantitative form. That is exactly what new-school bullshitters are counting on. To fight back, one must learn when and how to question such statements.

WE HAVE DEVOTED OUR careers to teaching students how to think logically and quantitatively about data. This book emerged from a course we teach at the University of Washington, also titled Calling Bullshit. We hope it will show you that you do not need to be a professional statistician or econometrician or data scientist to think critically about quantitative arguments, nor do you need extensive data sets and weeks of effort to see through bullshit. It is often sufficient to apply basic logical reasoning to a problem and, where needed, augment that with information readily discovered via search engine.

We have civic motives for wanting to help people spot and refute bullshit. It is not a matter of left- or right-wing ideology; people on both sides of the aisle have proven themselves skilled at creating and spreading misinformation. Rather (at the risk of grandiosity), we believe that adequate bullshit detection is essential for the survival of liberal democracy. Democracy has always relied on a critically thinking electorate, but never has this been more important than in the current age of fake news and international interference in the electoral process via propaganda disseminated over social media. In a December 2016