Seth Stephens-Davidowitz - Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are

This book was a team effort.

These ideas were developed while I was a student at Harvard, a data scientist at Google, and a writer for the New York Times .

Hal Varian, with whom I worked at Google, has been a major influence on the ideas of this book. As best I can tell, Hal is perpetually twenty years ahead of his time. His book Information Rules , written with Carl Shapiro, basically predicted the future. And his paper Predicting the Present, with Hyunyoung Choi, largely started the Big Data revolution in the social sciences that is described in this book. He is also an amazing and kind mentor, as so many who have worked under him can attest. A classic Hal move is to do most of the work on a paper you are coauthoring with him and then insist that your name goes before his. Hals combination of genius and generosity is something I have rarely encountered.

My writing and ideas developed under Aaron Retica, who has been my editor for every single New York Times column. Aaron is a polymath. He somehow knows everything about music, history, sports, politics, sociology, economics, and God only knows what else. He is responsible for a huge amount of what is good about the Times columns that have my name on them. Other players on the team for these columns include Bill Marsh, whose graphics continue to blow me away, Kevin McCarthy, and Gita Daneshjoo. This book includes passages from these columns, reprinted with permission.

Steven Pinker, who kindly agreed to write the foreword, has long been a hero of mine. He has set the bar for a modern book on social sciencean engaging exploration of the fundamentals of human nature, making sense of the best research from a range of disciplines. That bar is one I will be struggling to reach my entire life.

My dissertation, from which this book has grown, was written under my brilliant and patient advisers Alberto Alesina, David Cutler, Ed Glaeser, and Lawrence Katz.

Denise Oswald is an amazing editor. If you want to know how good her editing is, compare this final draft to my first draftactually, you cant do that because I am not going to ever show anyone else that embarrassing first draft. I also thank the rest of the team at HarperCollins, including Michael Barrs, Lynn Grady, Lauren Janiec, Shelby Meizlik, and Amber Oliver.

Eric Lupfer, my agent, saw potential in this project from the beginning, was instrumental in forming the proposal, and helped carry it through.

For superb fact-checking, I thank Melvis Acosta.

Other people from whom I learned a lot in my professional and academic life include Susan Athey, Shlomo Benartzi, Jason Bordoff, Danielle Bowers, David Broockman, Bo Cowgill, Steven Delpome, John Donohue, Bill Gale, Claudia Goldin, Suzanne Greenberg, Shane Greenstein, Steve Grove, Mike Hoyt, David Laibson, A.J. Magnuson, Dana Maloney, Jeffrey Oldham, Peter Orszag, David Reiley, Jonathan Rosenberg, Michael Schwarz, Steve Scott, Rich Shavelson, Michael D. Smith, Lawrence Summers, Jon Vaver, Michael Wiggins, and Qing Wu.

I thank Tim Requarth and NeuWrite for helping me develop my writing.

For help in interpreting studies, I thank Christopher Chabris, Raj Chetty, Matt Gentzkow, Solomon Messing, and Jesse Shapiro.

I asked Emma Pierson and Katia Sobolski if they might give advice on a chapter in my book. They decided, for reasons I do not understand, to offer to read the entire bookand give wise counsel on every paragraph.

My mother, Esther Davidowitz, read the entire book on multiple occasions and helped dramatically improve it. She also taught me, by example, that I should follow my curiosity, no matter where it led. When I was interviewing for an academic job, a professor grilled me: What does your mother think of this work you do? The idea was that my mom might be embarrassed that I was researching sex and other taboo topics. But I always knew she was proud of me for following my curiosity, wherever it led.

Many people read sections and offered helpful comments. I thank Eduardo Acevedo, Coren Apicella, Sam Asher, David Cutler, Stephen Dubner, Christopher Glazek, Jessica Goldberg, Lauren Goldman, Amanda Gordon, Jacob Leshno, Alex Peysakhovich, Noah Popp, Ramon Roullard, Greg Sobolski, Evan Soltas, Noah Stephens-Davidowitz, Lauren Stephens-Davidowitz, and Jean Yang. Actually, Jean was basically my best friend while I wrote this, so I thank her for that, too.

For help in collecting data, I thank Brett Goldenberg, James Rogers, and Mike Williams at MindGeek and Rob McQuown and Sam Miller at Baseball Prospectus.

I am grateful for financial support from the Alfred Sloan Foundation.

At one point, while writing this book, I was deeply stuck, lost, and close to abandoning the project. I then went to the country with my dad, Mitchell Stephens. Over the course of a week, Dad put me back together. He took me for walks in which we discussed love, death, success, happiness, and writingand then sat me down so we could go over every sentence of the book. I could not have finished this book without him.

All remaining errors are, of course, my own.

Seth Stephens-Davidowitz is a New York Times op-ed contributor, a visiting lecturer at The Wharton School, and a former Google data scientist. He received a BA in philosophy from Stanford, where he graduated Phi Beta Kappa, and a PhD in economics from Harvard. His researchwhich uses new, big data sources to uncover hidden behaviors and attitudeshas appeared in the Journal of Public Economics and other prestigious publications. He lives in New York City.

Discover Great Authors, Exclusive Offers, and more at hc.com .

Australia

HarperCollins Publishers (Australia) Pty. Ltd.

Level 13, 201 Elizabeth Street

Sydney, NSW 2000, Australia

www.harpercollins.com.au

Canada

HarperCollins Canada

2 Bloor Street East - 20th Floor

Toronto, ON M4W 1A8, Canada

www.harpercollins.ca

New Zealand

HarperCollins Publishers New Zealand

Unit D1, 63 Apollo Drive

Rosedale 0632

Auckland, New Zealand

www.harpercollins.co.nz

United Kingdom

HarperCollins Publishers Ltd.

1 London Bridge Street

London SE1 9GF, UK

www.harpercollins.co.uk

United States

HarperCollins Publishers Inc.

195 Broadway

New York, NY 10007

www.harpercollins.com

I f youre thirty-three years old and have attended a few Thanksgivings in a row without a date, the topic of mate choice is likely to arise. And just about everybody will have an opinion.

Seth needs a crazy girl, like him, my sister says.

Youre crazy! He needs a normal girl, to balance him out, my brother says.

Seths not crazy, my mother says.

Youre crazy! Of course, Seth is crazy, my father says.

All of a sudden, my shy, soft-spoken grandmother, quiet through the dinner, speaks. The loud, aggressive New York voices go silent, and all eyes focus on the small old lady with short yellow hair and still a trace of an Eastern European accent. Seth, you need a nice girl. Not too pretty. Very smart. Good with people. Social, so you will do things. Sense of humor, because you have a good sense of humor.

Why does this old womans advice command such attention and respect in my family? Well, my eighty-eight-year-old grandmother has seen more than everybody else at the table. Shes observed more marriages, many that worked and many that didnt. And over the decades, she has cataloged the qualities that make for successful relationships. At that Thanksgiving table, for that question, my grandmother has access to the largest number of data points. My grandmother is Big Data.

In this book, I want to demystify data science. Like it or not, data is playing an increasingly important role in all of our livesand its role is going to get larger. Newspapers now have full sections devoted to data. Companies have teams with the exclusive task of analyzing their data. Investors give start-ups tens of millions of dollars if they can store more data. Even if you never learn how to run a regression or calculate a confidence interval, you are going to encounter a lot of datain the pages you read, the business meetings you attend, the gossip you hear next to the watercoolers you drink from.