Hands-On Machine Learning with Scikit-Learn and TensorFlow
Concepts, Tools, and Techniques to Build Intelligent Systems
Aurlien Gron
Hands-On Machine Learning with Scikit-Learn and TensorFlow
by Aurlien Gron
Copyright 2017 Aurlien Gron. All rights reserved.
Printed in the United States of America.
Published by OReilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472.
OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com/safari). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .
- Editor: Nicole Tache
- Production Editor: Nicholas Adams
- Copyeditor: Rachel Monaghan
- Proofreader: Charles Roumeliotis
- Indexer: Wendy Catalano
- Interior Designer: David Futato
- Cover Designer: Randy Comer
- Illustrator: Rebecca Demarest
- March 2017: First Edition
Revision History for the First Edition
- 2017-03-10: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781491962299 for release details.
The OReilly logo is a registered trademark of OReilly Media, Inc. Hands-On Machine Learning with Scikit-Learn and TensorFlow, the cover image, and related trade dress are trademarks of OReilly Media, Inc.
While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.
978-1-491-96229-9
[LSI]
Preface
The Machine Learning Tsunami
In 2006, Geoffrey Hinton et al. published a paper and most researchers had abandoned the idea since the 1990s. This paper revived the interest of the scientific community and before long many new papers demonstrated that Deep Learning was not only possible, but capable of mind-blowing achievements that no other Machine Learning (ML) technique could hope to match (with the help of tremendous computing power and great amounts of data). This enthusiasm soon extended to many other areas of Machine Learning.
Fast-forward 10 years and Machine Learning has conquered the industry: it is now at the heart of much of the magic in todays high-tech products, ranking your web search results, powering your smartphones speech recognition, and recommending videos, beating the world champion at the game of Go. Before you know it, it will be driving your car.
Machine Learning in Your Projects
So naturally you are excited about Machine Learning and you would love to join the party!
Perhaps you would like to give your homemade robot a brain of its own? Make it recognize faces? Or learn to walk around?
Or maybe your company has tons of data (user logs, financial data, production data, machine sensor data, hotline stats, HR reports, etc.), and more than likely you could unearth some hidden gems if you just knew where to look; for example:
- Segment customers and find the best marketing strategy for each group
- Recommend products for each client based on what similar clients bought
- Detect which transactions are likely to be fraudulent
- Predict next years revenue
- And more
Whatever the reason, you have decided to learn Machine Learning and implement it in your projects. Great idea!
Objective and Approach
This book assumes that you know close to nothing about Machine Learning. Its goal is to give you the concepts, the intuitions, and the tools you need to actually implement programs capable of learning from data.
We will cover a large number of techniques, from the simplest and most commonly used (such as linear regression) to some of the Deep Learning techniques that regularly win competitions.
Rather than implementing our own toy versions of each algorithm, we will be using actual production-ready Python frameworks:
- Scikit-Learn is very easy to use, yet it implements many Machine Learning algorithms efficiently, so it makes for a great entry point to learn Machine Learning.
- TensorFlow is a more complex library for distributed numerical computation using data flow graphs. It makes it possible to train and run very large neural networks efficiently by distributing the computations across potentially thousands of multi-GPU servers. TensorFlow was created at Google and supports many of their large-scale Machine Learning applications. It was open-sourced in November 2015.
The book favors a hands-on approach, growing an intuitive understanding of Machine Learning through concrete working examples and just a little bit of theory. While you can read this book without picking up your laptop, we highly recommend you experiment with the code examples available online as Jupyter notebooks at https://github.com/ageron/handson-ml.
Prerequisites
This book assumes that you have some Python programming experience and that you are familiar with Pythons main scientific libraries, in particular NumPy, Pandas, and Matplotlib.
Also, if you care about whats under the hood you should have a reasonable understanding of college-level math as well (calculus, linear algebra, probabilities, and statistics).
If you dont know Python yet, http://learnpython.org/ is a great place to start. The official tutorial on python.org is also quite good.
If you have never used Jupyter, will guide you through installation and the basics: it is a great tool to have in your toolbox.
If you are not familiar with Pythons scientific libraries, the provided Jupyter notebooks include a few tutorials. There is also a quick math tutorial for linear algebra.
Roadmap
This book is organized in two parts. , covers the following topics:
- What is Machine Learning? What problems does it try to solve? What are the main categories and fundamental concepts of Machine Learning systems?
- The main steps in a typical Machine Learning project.
- Learning by fitting a model to data.
- Optimizing a cost function.
- Handling, cleaning, and preparing data.
- Selecting and engineering features.
- Selecting a model and tuning hyperparameters using cross-validation.
- The main challenges of Machine Learning, in particular underfitting and overfitting (the bias/variance tradeoff).
- Reducing the dimensionality of the training data to fight the curse of dimensionality.
- The most common learning algorithms: Linear and Polynomial Regression, Logistic Regression, k-Nearest Neighbors, Support Vector Machines, Decision Trees, Random Forests, and Ensemble methods.
, covers the following topics:
- What are neural nets? What are they good for?
- Building and training neural nets using TensorFlow.
- The most important neural net architectures: feedforward neural nets, convolutional nets, recurrent nets, long short-term memory (LSTM) nets, and autoencoders.
- Techniques for training deep neural nets.
- Scaling neural networks for huge datasets.