Michael Taylor - Neural Networks Math; A Visual Introduction for Beginners

A few points to help you make the most of this book:

1. This book is designed as a visual introduction to the math of neural networks. It is for BEGINNERS and those who have minimal knowledge of the topic. If you already have a general understanding, you might not get much out of this book.

1. You dont need to read front to back. Skip around to what you find the most helpful or is perking your interest. Weve included lots of links throughout the book to make this easy. Just click on them.

1. We *slowly* layer in new terminology and concepts each chapter. This means that if you jump to chapter 4 without having read chapter 3, you might come across terminology that you do not understand. Be aware of this, and remember: you can always jump back to clarify a topic or concept.

On a high level, a network learns just like we do, through trial and error. This is true regardless if the network is supervised, unsupervised, or semi-supervised. Once we dig a bit deeper though, we discover that a handful of mathematical functions play a major role in the trial and error process. It also becomes clear that a grasp of the underlying mathematics helps clarify how a network learns.

In the following chapters we will unpack the mathematics that drive a neural network. To do this, we will use a feedforward network as our model and follow input as it moves through the network.

This "process" of moving through the network is complex though, so to make it easier and bite-size, we will divide it into 5 stages and take it slow.

Each of the stages includes at least one mathematical function, and we will devote an entire chapter to every stage. Our goal is that by the end, you will find neural networks less mind-bending and simply more fascinating. What's more, we hope that you'll be able to confidently explain these concepts to others and make use of them for yourself.

Here are the stages we will be exploring:

Required: Summation operator

Required: Activation function

Required: Cost function

Required: Partial derivative

Required: Chain Rule

Required: Gradient checking formula

Required: Weight update formula

However, before diving into the stages, we are going to clarify the terminology and notation that we will be using. We will also take a moment to build on the topics and ideas introduced in the previous chapters. Accomplishing these things will build up a solid framework and provide us with the tools required to move forward.

A extra few words before moving on:

You might be scratching your head and wondering if all networks have the same mathematical functions . The answer is a loud no , and for a few good reasons, including the network architecture, its purpose, and even the personal preferences of its developer. However, there is a general framework for feedforward models that includes a handful of mathematical functions, which is what we will be investigating.

On another note, the following chapters require a college-level grasp of algebra, statistics and calculus. A major concept you'll come across is calculating partial derivatives, which itself makes use of the chain rule. Please be aware of this! If you are rusty in these areas you might find yourself repeatedly frustrated. The Khan Academy offers top-notch free courses in algebra , calculus and statistics .