Davenport Thomas H.Kim JinhoPatil D. J.Harris Jeanne - Analytics and Big Data

Harvard Business Review Press
Boston, Massachusetts

ISBN-13: 978-1-62527-774-9

Analytics 3.0

In the new era, big data will power
consumer products and services.
by Thomas H. Davenport

T hose of us who have spent years studying data smart companies believe weve already lived through two eras in the use of analytics. We might call them BBD and ABDbefore big data and after big data. Or, to use a naming convention matched to the topic, we might say that Analytics 1.0 was followed by Analytics 2.0. Generally speaking, 2.0 releases dont just add some bells and whistles or make minor performance tweaks. In contrast to, say, a 1.1 version, a 2.0 product is a more substantial overhaul based on new priorities and technical possibilities. When large numbers of companies began capitalizing on vast new sources of unstructured, fast-moving informationbig datathat was surely the case.

Some of us now perceive another shift, fundamental and far-reaching enough that we can fairly call it Analytics 3.0. Briefly, it is a new resolve to apply powerful data-gathering and analysis methods not just to a companys operations but also to its offeringsto embed data smartness into the products and services customers buy.

Ill develop this argument in what follows, making the case that just as the early applications of big data marked a major break from the 1.0 past, the current innovations of a few industry leaders are evidence that a new era is dawning. When a new way of thinking about and applying a strength begins to take hold, managers are challenged to respond in many ways. Change comes fast to every part of a businesss world. New players emerge, competitive positions shift, novel technologies must be mastered, and talent gravitates toward the most exciting new work.

Managers will see all these things in the coming months and years. The ones who respond most effectively will be those who have connected the dots and recognized that competing on analytics is being rethought on a large scale. Indeed, the first companies to perceive the general direction of changethose with a sneak peek at Analytics 3.0will be best positioned to drive that change.

My purpose here is not to make abstract observations about the unfolding history of analytics. Still, it is useful to look back at the last big shift and the context in which it occurred. The use of data to make decisions is, of course, not a new idea; it is as old as decision making itself. But the field of business analytics was born in the mid-1950s, with the advent of tools that could produce and capture a larger quantity of information and discern patterns in it far more quickly than the unassisted human mind ever could.

Analytics 1.0the era of business intelligence. What we are here calling Analytics 1.0 was a time of real progress in gaining an objective, deep understanding of important business phenomena and giving managers the fact-based comprehension to go beyond intuition when making decisions. For the first time, data about production processes, sales, customer interactions, and more were recorded, aggregated, and analyzed.

New computing technologies were key. Information systems were at first custom-built by companies whose large scale justified the investment; later, they were commercialized by outside vendors in more-generic forms. This was the era of the enterprise data warehouse, used to capture information, and of business intelligence software, used to query and report it.

New competencies were required as well, beginning with the ability to manage data. Data sets were small enough in volume and static enough in velocity to be segregated in warehouses for analysis. However, readying a data set for inclusion in a warehouse was difficult. Analysts spent much of their time preparing data for analysis and relatively little time on the analysis itself.

More than anything else, it was vital to figure out the right few questions on which to focus, because analysis was painstaking and slow, often taking weeks or months to perform. And reporting processesthe great majority of business intelligence activityaddressed only what had happened in the past; they offered no explanations or predictions.

Did people see analytics as a source of competitive advantage? In broad terms, yesbut no one spoke in todays terms of competing on analytics. The edge came in the form of greater operational efficiencymaking better decisions on certain key points to improve performance.

Analytics 2.0the era of big data. The basic conditions of the Analytics 1.0 period predominated for half a century, until the mid-2000s, when internet-based and social network firms primarily in Silicon ValleyGoogle, eBay, and so onbegan to amass and analyze new kinds of information. Although the term big data wasnt coined immediately, the new reality it signified very quickly changed the role of data and analytics in those firms. Big data also came to be distinguished from small data because it was not generated purely by a firms internal transaction systems. It was externally sourced as well, coming from the internet, sensors of various types, public data initiatives such as the human genome project, and captures of audio and video recordings.

As analytics entered the 2.0 phase, the need for powerful new toolsand the opportunity to profit by providing themquickly became apparent. Companies rushed to build new capabilities and acquire customers. The broad recognition of the advantage a first mover could gain led to an impressive level of hype but also prompted an unprecedented acceleration of new offerings. LinkedIn, for example, has created numerous data products, including People You May Know, Jobs You May Be Interested In, Groups You May Like, Companies You May Want to Follow, Network Updates, and Skills and Expertise. To do so, it built a strong infrastructure and hired smart, productive data scientists. Its highly successful Year in Review, which summarizes the job changes of people in a members network, was developed in just a month. And LinkedIn is not the only company focused on speed. One CEO of a big data start-up told me, We tried agile [development methodology], but it was too slow.

Innovative technologies of many kinds had to be created, acquired, and mastered. Big data couldnt fit or be analyzed fast enough on a single server, so it was processed with Hadoop, an open source software framework for fast batch data processing across parallel servers. To deal with relatively unstructured data, companies turned to a new class of databases known as NoSQL. Much information was stored and analyzed in public or private cloud-computing environments. Other technologies introduced during this period include in memory and in database analytics for fast number crunching. Machine-learning methods (semiautomated model development and testing) were used to rapidly generate models from the fast-moving data. Black-and-white reports gave way to colorful, complex visuals.