Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow
by Aurlien Gron
Copyright 2019 Aurlien Gron. All rights reserved.
Printed in Canada.
Published by OReilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472.
OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .
- Editors: Rachel Roumeliotis and Nicole Tache
- Production Editor: Kristen Brown
- Copyeditor: Amanda Kersey
- Proofreader: Rachel Head
- Indexer: Judith McConville
- Interior Designer: David Futato
- Cover Designer: Karen Montgomery
- Illustrator: Rebecca Demarest
- September 2019: Second Edition
Revision History for the Second Edition
- 2019-09-05: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781492032649 for release details.
The OReilly logo is a registered trademark of OReilly Media, Inc. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, the cover image, and related trade dress are trademarks of OReilly Media, Inc.
The views expressed in this work are those of the author, and do not represent the publishers views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.
978-1-492-03264-9
[TI]
Preface
The Machine Learning Tsunami
In 2006, Geoffrey Hinton technique could hope to match (with the help of tremendous computing power and great amounts of data). This enthusiasm soon extended to many other areas of Machine Learning.
A decade or so later, Machine Learning has conquered the industry: it is at the heart of much of the magic in todays high-tech products, ranking your web search results, powering your smartphones speech recognition, recommending videos, and beating the world champion at the game of Go. Before you know it, it will be driving your car.
Machine Learning in Your Projects
So, naturally you are excited about Machine Learning and would love to join the party!
Perhaps you would like to give your homemade robot a brain of its own? Make it recognize faces? Or learn to walk around?
Or maybe your company has tons of data (user logs, financial data, production data, machine sensor data, hotline stats, HR reports, etc.), and more than likely you could unearth some hidden gems if you just knew where to look. With Machine Learning, you could accomplish the following and more:
Segment customers and find the best marketing strategy for each group.
Recommend products for each client based on what similar clients bought.
Detect which transactions are likely to be fraudulent.
Forecast next years revenue.
Whatever the reason, you have decided to learn Machine Learning and implement it in your projects. Great idea!
Objective and Approach
This book assumes that you know close to nothing about Machine Learning. Its goal is to give you the concepts, tools, and intuition you need to implement programs capable of learning from data.
We will cover a large number of techniques, from the simplest and most commonly used (such as Linear Regression) to some of the Deep Learning techniques that regularly win competitions.
Rather than implementing our own toy versions of each algorithm, we will be using production-ready Python frameworks:
Scikit-Learn is very easy to use, yet it implements many Machine Learning algorithms efficiently, so it makes for a great entry point to learning Machine Learning.
TensorFlow is a more complex library for distributed numerical computation. It makes it possible to train and run very large neural networks efficiently by distributing the computations across potentially hundreds of multi-GPU (graphics processing unit) servers. TensorFlow (TF) was created at Google and supports many of its large-scale Machine Learning applications. It was open sourced in November 2015.
Keras is a high-level Deep Learning API that makes it very simple to train and run neural networks. It can run on top of either TensorFlow, Theano, or Microsoft Cognitive Toolkit (formerly known as CNTK). TensorFlow comes with its own implementation of this API, called tf.keras, which provides support for some advanced TensorFlow features (e.g., the ability to efficiently load data).
The book favors a hands-on approach, growing an intuitive understanding of Machine Learning through concrete working examples and just a little bit of theory. While you can read this book without picking up your laptop, I highly recommend you experiment with the code examples available online as Jupyter notebooks at https://github.com/ageron/handson-ml2.
Prerequisites
This book assumes that you have some Python programming experience and that you are familiar with Pythons main scientific librariesin particular, NumPy, pandas, and Matplotlib.
Also, if you care about whats under the hood, you should have a reasonable understanding of college-level math as well (calculus, linear algebra, probabilities, and statistics).
If you dont know Python yet, http://learnpython.org/ is a great place to start. The official tutorial on Python.org is also quite good.
If you have never used Jupyter, will guide you through installation and the basics: it is a powerful tool to have in your toolbox.
If you are not familiar with Pythons scientific libraries, the provided Jupyter notebooks include a few tutorials. There is also a quick math tutorial for linear algebra.
Roadmap
This, covers the following topics:
What Machine Learning is, what problems it tries to solve, and the main categories and fundamental concepts of its systems
The steps in a typical Machine Learning project
Learning by fitting a model to data
Optimizing a cost function
Handling, cleaning, and preparing data
Selecting and engineering features
Selecting a model and tuning hyperparameters using cross-validation
The challenges of Machine Learning, in particular underfitting and overfitting (the bias/variance trade-off)
The most common learning algorithms: Linear and Polynomial Regression, Logistic Regression, k-Nearest Neighbors, Support Vector Machines, Decision Trees, Random Forests, and Ensemble methods
Reducing the dimensionality of the training data to fight the curse of dimensionality
Other unsupervised learning techniques, including clustering, density estimation, and anomaly detection
, covers the following topics:
What neural nets are and what theyre good for
Building and training neural nets using TensorFlow and Keras
The most important neural net architectures: feedforward neural nets for tabular data, convolutional nets for computer vision, recurrent nets and long short-term memory (LSTM) nets for sequence processing, encoder/decoders and Transformers for natural language processing, autoencoders and generative adversarial networks (GANs) for generative learning