Steve Lohr - Data-ism: The Revolution Transforming Decision Making, Consumer Behavior, and Almost Everything Else

TO TERRY AND NIKKI, FOR TIME
SERVED, AND ALL THE LAUGHS
ALONG THE WAY

A book may be written by a single person, but books are always collaborations. An editor nudges things along, a publisher makes a bet, and the adventure begins. There is an original idea, but it gets refined along the way, modified, honed and crafted by reporting, thinking and guidance.

The germ of this book really came in January 2012, with a call from a senior editor at the New York Times and former editor of the Times Magazine, Gerry Marzorati. At the time, he was helping guide the recently revamped Sunday Review section, and Gerry observed that I had written a number of articles about the field that was being called big data, and that the subject was getting more attention all the time, including being prominently on the agenda of the World Economic Forum annual meeting in Davos that month. We talked for a while, and he asked if there was a long overview piece to be done that explained the phenomenon for a general audience. I replied that I thought there was. The result was a lead article for Sunday Review that ran in early February 2012 under the headline The Age of Big Data.

The Sunday Review piece generated a lot of interest, Web traffic, and reader comments. I hadnt thought of a book, but my agent, Max Brockman of Brockman Inc., did, and encouraged me, and offered sound advice. Hollis Heimbouch, publisher of the HarperBusiness imprint of HarperCollins, was the editor who made the bet that what is a rich subject could be made into a storyand that I could do it. Hollis is everything you want in an editorby turns enthusiastic, patient, and insistent, someone who listens to your ideas and brings plenty of incisive ones of her own. Shes also a very sure hands-on text editor in every way, from shaping the structure of a book to cutting. At HarperCollins, Id also like to thank Eric Meyers, an associate editor and a deft practitioner of the dark arts of manuscript wrangling.

At the Times, several editors haveand still doencourage my reporting in the field of data science, and its implications. They include Larry Ingrassia, Dean Murphy, Damon Darlin, Suzanne Spector, Joseph Plambeck, James Kerstetter, Quentin Hardy, David Gillen, Lon Teter, Thomas Kuntz, and David Corcoran. Conversations with Natasha Singer have sharpened my thinking on the subject of data and privacy. Then there is John Markoff, a science reporter these days; he and I have labored agreeably and often together in the data-laden precincts of business and science coverage for years.

While I was on leave from the Times, Mark Hansen generously offered me office space to work in and travel from and an appointment as a research fellow at the Brown Institute for Media Innovation. Mark is a director of the Brown Institute, a collaboration between the Columbia Graduate School of Journalism and the Stanford School of Engineering. Mark and Bernd Girod, the Stanford director of the institute, are leading an innovative effort to use data tools in storytelling and public policy. At Columbia, my workspace was in the career services office, and my thanks to the professionals there for kindly making the accommodation: Julie Hartenstein, Gina Boubion, Anusha Shrivastava, Izabela Rutkowski, and, earlier, Jacqueline DeLaFuente.

This is a book of narrative nonfiction. So I am especially thankful to the people who agreed to be interviewed, and showed the patience for follow-up and fact-checking inquiries. At the top of the list is Jeffrey Hammerbacher, who gave the most in time, thoughts, and candor. He not only went through many hours of interviews but also offered introductions to his parents, his wife, classmates from high school and college, and friends and work colleagues, past and present. All with no strings attached.

For other parts of the book, there were people I spent whole days with, such as Timothy Buchman at the Emory medical center in Atlanta and Nick Dokoozlian in the grape vineyards of central California and Michael Haydock in suburban Minneapolis.

Many others were interviewed for this book. They include Sam Adams, Brooke Barrett, Richard Berner, Patrick Bosworth, Thomas Botts, Erik Brynjolfsson, John Calkins, Murray Campbell, Dennis Charney, Herbert Chase, Jeffrey Chester, Sharath Cholleti, Adam DAngelo, Arne Duncan, Sue Duncan, Tony Fadell, Edward Felten, David Ferrucci, Rachana Shah Fischer, Brian Gehlich, Jim Goodnight, Nagui Halim, Hendrik Hamann, Glenn Hammerbacher, Lenore Hammerbacher, Danny Hillis, Jeffrey Immelt, Jon Iwata, James Kalina, Kaan Katircioglu, Gary King, Jon Kleinberg, Martin Kohn, Randy Komisar, Patricia Kovatch, Edward Lazowska, and Michael Linderman.

They also include Mark Malhotra, James Manyika, Yoky Matsuoka, Andrew McAfee, David McQueeney, Douglas Merrill, Tom Mitchell, Craig Mundie, Arvind Narayanan, Tim ODonnell, Sharoda Paul, Alex Pentland, Claudia Perlich, Stephanie Pieroni, William Pulleyblank, Tara Richardson, Virginia Rometty, Itamar Rosenn, Jeff Rothschild, Marc Rotenberg, Alex Rubinsteyn, William Ruh, Eric Schadt, Benjamin Scheuer, Neal Sidhwaney, Larry Smarr, Andrew Smeall, Jim Spohrer, Halle Tecco, Menka Uttamchandani, Veronica Vargas, Anil Varma, David Vivero, David Vladeck, Donald Walker, Danny Weitzner, and Michelle Zhou.

At IBM, I owe particular thanks to Jon Iwata and Ed Barbini for clearing the way for a broad reporting swath with IBM scientists and executives, and to Vineeta Durani and Angela Lee Sullivan for tirelessly making the interviews happen.

Finally, thanks to Terry and Nikki for understanding about work and the missed vacations, and to Nickie Trucco for my missing the Christmas holidays in 2013. But then, as I told them at the time, this book was a good problem to have.

J ust outside Memphis, an industrial symphony of machines and humans shuttles goods to and fro, their carefully orchestrated movements and identifying marks tracked by bar-code scanners and chips emitting radio waves. Mechanical arms snatch the plastic shrink-wrapped bundles off a conveyor belt, as forklifts ferry the packages onto trucks for long-distance travel. Flesh-and-blood humans guide and monitor the flow of goods and drive the forklifts and trucks.

McKesson, which distributes about a third of all of the pharmaceutical products in America, runs this sprawling showcase of efficiency. Its buildings span the equivalent of more than eight football fields, forming the hub of McKessons national distribution networka feat of logistics that sends goods to 26,000 customer locations, from neighborhood pharmacies to Walmart. The main cargo is drugs, roughly 240 million pills a day. The pharmaceutical distribution business is one of high volumes and razor-thin profit margins. So, understandably, efficiency has been all but a religion for McKesson for decades.

Yet in the last few years, McKesson has taken a striking step further by cutting the inventory flowing through its network at any given time by $1 billion. The payoff came from insights gleaned from harvesting all the product, location, and transport data, from scanners and sensors, and then mining that data with clever software to identify potential time-saving and cost-cutting opportunities. The technology-enhanced view of the business was a breakthrough that Donald Walker, a senior McKesson executive, calls making the invisible visible.

In Atlanta, I stand outside one of the glassed-in rooms in the fifth-floor intensive care unit at the Emory University Hospital. Inside, a dense thicket of electronic devices, a veritable forest of medical computing, crowds the room: a respirator, a kidney machine, infusion machines pumping antibiotics and painkilling opiates, and gadgets monitoring heart rate, breathing, blood pressure, oxygen saturation, and other vital signs. Nearly every machine has its own computer monitor, each emitting an electronic cacophony of beeps and alerts. I count a dozen screens, larger flat panels and smaller ones, smartphone-sized.