Gary Smith - The 9 Pitfalls of Data Science

A 2012 article in the Harvard Business Review named data scientist the sexiest job of the 21st century. Governments and businesses are scrambling to hire data scientists, and workers are clamoring to become data scientists, or at least label themselves as such.

Many colleges and universities now offer data science degrees, but their curricula differ wildly. Many businesses have data science divisions, but few restrictions on what they do. Many people say they are data scientists, but may have simply taken some online programming courses and dont know what they dont know. The result is that the analyses produced by data scientists are sometimes spectacular and, other times, disastrous. In a rush to learn the technical skills, the crucial principles of data science are often neglected.

Too many would-be data scientists have the misguided belief that we dont need theories, common sense, or wisdom. An all-too-common thought is, We shouldnt waste time thinking about why something may or may not be true. Its enough to let computers find a pattern and assume that the pattern will persist and make useful predictions. This ill-founded belief underlies misguided projects that have attempted to use Facebook status updates to price auto insurance, Google search queries to predict flu outbreaks, and Twitter tweets to predict stock prices.

Data science is surely revolutionizing our lives, allowing decisions to be based on data rather than lazy thinking, whims, hunches, and prejudices. Unfortunately, data scientists themselves can be plagued by lazy thinking, whims, hunches, and prejudicesand end up fooling themselves and others.

One of Jays managers understood the difference between good data science and garbage. He categorized people who knew what they were doing as kids; those who spouted nonsense were clowns. One of our goals is to explain the difference between a data scientist and a data clown.

Our criticism of data clowns should not be misinterpreted as a disdain for science or scientists. The Economist once used a jigsaw puzzle and a house of cards as metaphors:

In any complex scientific picture of the world there will be gaps, misperceptions and mistakes. Whether your impression is dominated by the whole or the holes will depend on your attitude to the project at hand. You might say that some see a jigsaw where others see a house of cards. Jigsaw types have in mind an overall picture and are open to bits being taken out, moved around or abandoned should they not fit. Those who see houses of cards think that if any piece is removed, the whole lot falls down.

We are firmly in the jigsaw puzzle camp. When done right, theres no question that science works and enriches our lives immensely.

We are not going to bombard you with equations or bore you with technical tips. We want to propose enduring principles. We offer these principles as nine pitfalls to be avoided. We hope that these principles will not only help data scientists be more effective, but also help everyone distinguish between good data science and rubbish. Our book is loaded with grand successes and epic failures. It highlights winning approaches and warns of common pitfalls. We are confident that after reading it, you will recognize good data science when you see it, know how to avoid being duped by data, and make better, more informed decisions. Whether you want to be an effective creator, interpreter, user, or consumer of data, it is important to know the nine pitfalls of data science.

The 9 Pitfalls of Data Science. Gary Smith and Jay Cordes. Oxford University Press (2019).

Gary Smith and Jay Cordes 2019. DOI: 10.1093/oso/9780198844396.001.0001