Henrik Brink - Real-World Machine Learning

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact

Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com

2017 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps.

Recognizing the importance of preserving what has been written, it is Mannings policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine.

Manning Publications Co.20 Baldwin RoadPO Box 761Shelter Island, NY 11964

Development editor: Susanna KlineTechnical development editor: Al SchererReview editors: Olivia Booth, Ozren HarlovicProject editor: Kevin SullivanCopyeditor: Sharon WilkeyProofreader: Katie TennantTechnical proofreader: Valentin CrettazTypesetter: Dennis DalinnikCover designer: Marija Tudor

ISBN: 9781617291920

Printed in the United States of America

1 2 3 4 5 6 7 8 9 10 EBM 21 20 19 18 17 16

Machine learning (ML) has become big business in the last few years: companies are using it to make money, applied research has exploded in both industrial and academic settings, and curious developers everywhere are looking to level up their ML skills. But this newfound demand has largely outrun the supply of good methods for learning how these techniques are used in the wild. This book fills a pressing need.

Applied machine learning comprises equal parts mathematical principles and tricks pulled from a bagit is, in other words, a true craft. Concentrating too much on either aspect at the expense of the other is a failure mode. Balance is essential.

For a long time, the bestand the onlyway to learn machine learning was to pursue an advanced degree in one of the fields that (largely separately) developed statistical learning and optimization techniques. The focus in these programs was on the core algorithms, including their theoretical properties and bounds, as well as the characteristic domain problems of the field. In parallel, though, an equally valuable lore was accumulated and passed down through unofficial channels: conference hallways, the tribal wisdom of research labs, and the data processing scripts passed between colleagues. This lore was what actually allowed the work to get done, establishing which algorithms were most appropriate in each situation, how the data needed to be massaged at each step, and how to wire up the different parts of the pipeline.

Cut to today. We now live in an era of open source riches, with high-quality implementations of most ML algorithms readily available on GitHub, as well as comprehensive and well-architected frameworks to tie all the pieces together. But in the midst of this abundance, the unofficial lore has remained stubbornly inaccessible. The authors of this book provide a great service by finally bringing this dark knowledge together in one place; this is a key missing piece as machine learning moves from esoteric academic discipline to core software engineering skillset.

Another point worth emphasizing: most of the machine-learning methods in broad use today are far from perfect, meeting few of the desiderata we might list, were we in a position to design the perfect solution. The current methods are picky about the data they will accept. They are, by and large, happy to provide overly confident predictions if not carefully tended. Small changes in their input can lead to large and mysterious changes in the models they learn. Their results can be difficult to interpret and further interrogate. Modern ML engineering can be viewed as an exercise in managing and mitigating these (and other) rough edges of the underlying optimization and statistical learning methods.

This book is organized exactly as it should be to prepare the reader for these realities. It first covers the typical workflow of machine-learning projects before diving into extended examples that show how this basic framework can be applied in realistic (read: messy) situations. Skimming through these pages, youll find few equations (theyre all available elsewhere, including the many classic texts in the field) but instead much of the hidden wisdom on how to go about implementing products and solutions based on machine learning.

This is, far and away, the best of times to be learning about this subject, and this book is an essential complement to the cornucopia of mathematical and formal knowledge available elsewhere. It is that crucial other book that many old hands wish they had back in the day.

B EAU C RONIN

H EAD OF D ATA , 21 I NC .

B ERKELEY , CA

As a student of physics and astronomy, I spent a significant proportion of my time dealing with data from measurements and simulations, with the goal of deriving scientific value by analyzing, visualizing, and modeling the data. With a background as a programmer, I quickly learned to use my programming skills as an important aspect of working with data. When I was first introduced to the world of machine learning, it showed not only great potential as a new tool in the data toolbox, but also a beautiful combination of the two fields that interested me the most: data science and programming.

Machine learning became an important part of my research in the physical sciences and led me to the UC Berkeley astronomy department, where statisticians, physicists, and computer scientists were working together to understand the universe, with machine learning as an increasingly important tool.

At the Center for Time Domain Informatics, I met Joseph Richards, a statistician and coauthor of this book. We learned not only that we could use our data science and machine-learning techniques to do scientific research, but also that there was increasing interest from companies and industries from outside academia. We co-founded Wise.io with Damian Eads, Dan Starr, and Joshua Bloom to make machine learning accessible to businesses.

For the past four years, Wise.io has been working with countless companies to optimize, augment, and automate their processes via machine learning. We built a large-scale machine-learning application platform that makes hundreds of millions of predictions every month for our clients, and we learned that data in the real world is messy in ways that continue to surprise us. We hope to pass on to you some of our knowledge of how to work with real-world data and build the next generation of intelligent software with machine learning.