Sandeep Uttamchandani - The Self-Service Data Roadmap

Please address comments and questions concerning this book to the publisher:

• OReilly Media, Inc.
• 1005 Gravenstein Highway North
• Sebastopol, CA 95472
• 800-998-9938 (in the United States or Canada)
• 707-829-0515 (international or local)
• 707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/ssdr.

Email to comment or ask technical questions about this book.

For news and information about our books and courses, visit http://oreilly.com.

Find us on Facebook: http://facebook.com/oreilly

Follow us on Twitter: http://twitter.com/oreillymedia

Watch us on YouTube: http://youtube.com/oreillymedia

Data is the new oil. There has been exponential growth in the amount of structured, semi-structured, and unstructured data collected within the enterprise. Insights extracted from the data are becoming a valuable differentiator for enterprises in every industry vertical, and machine learning (ML) models are used in product features as well as improved business processes.

Enterprises today are data-rich, but insights-poor. ). The remaining 95% of the effort is spent on data engineering related to discovering, collecting, and preparing data, as well as building and deploying the models in production.

While an enormous amount of data is being collected within data lakes, it may not be consistent, interpretable, accurate, timely, standardized, or sufficient. Data scientists spend a significant amount of time on engineering activities related to aligning systems for data collection, defining metadata, wrangling data to feed ML algorithms, deploying pipelines and models at scale, and so on. These activities are outside of their core insight-extracting skills and bottlenecked by dependency on data engineers and platform IT engineers who typically lack the necessary business context. The engineering complexity also limits data accessibility to data analysts and scientists rather than democratizing it to a growing number of data citizens in product management, marketing, finance, engineering, and so on. While there is a plethora of books on advancement in ML programming as well as deep-dive books on specific data technologies, there is little written about operational patterns for data engineering required to develop a self-service platform to support a wide spectrum of data users.