Seth Weidman - Deep Learning from Scratch: Building with Python from First Principles

by Seth Weidman

Copyright 2019 Seth Weidman. All rights reserved.

Printed in the United States of America.

Published by OReilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .

• Development Editor: Melissa Potter
• Acquisitions Editors: Jon Hassell and Mike Loukides
• Production Editor: Katherine Tozer
• Copyeditor: Arthur Johnson
• Proofreader: Rachel Monaghan
• Indexer: Judith McConville
• Interior Designer: David Futato
• Cover Designer: Karen Montgomery
• Illustrator: Rebecca Demarest

• September 2019: First Edition

See http://oreilly.com/catalog/errata.csp?isbn=9781492041412 for release details.

The OReilly logo is a registered trademark of OReilly Media, Inc. Deep Learning from Scratch, the cover image, and related trade dress are trademarks of OReilly Media, Inc.

The views expressed in this work are those of the author, and do not represent the publishers views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-492-04141-2

[LSI]

If youve tried to learn about neural networks and deep learning, youve probably encountered an abundance of resources, from blog posts to MOOCs (massive open online courses, such as those offered on Coursera and Udacity) of varying quality and even some booksI know I did when I started exploring the subject a few years ago. However, if youre reading this preface, its likely that each explanation of neural networks that youve come across is lacking in some way. I found the same thing when I started learning: the various explanations were like blind men describing different parts of an elephant, but none describing the whole thing. That is what led me to write this book.

These existing resources on neural networks mostly fall into two categories. Some are conceptual and mathematical, containing both the drawings one typically finds in explanations of neural networks, of circles connected by lines with arrows on the ends, as well as extensive mathematical explanations of what is going on so you can understand the theory. A prototypical example of this is the very good book Deep Learning by Ian Goodfellow et al. (MIT Press).

Other resources have dense blocks of code that, if run, appear to show a loss value decreasing over time and thus a neural network learning. For instance, the following example from the PyTorch documentation does indeed define and train a simple neural network on randomly generated data:

Explanations like this, of course, dont give much insight into what is really going on: the underlying mathematical principles, the individual neural network components contained here and how they work together, and so on.

What would a good explanation of neural networks contain? For an answer, it is instructive to look at how other computer science concepts are explained: if you want to learn about sorting algorithms, for example, there are textbooks that will contain: