Unknown - Convolutional Neural Networks in Python: Master Data Science and Machine Learning with Modern Deep Learning in Python, Theano, and TensorFlow (Machine Learning in Python)

Train and Predict You should know that the basic API that we can use for all supervised learning problems is fit(X,Y) or train(X,Y) function, which takes in some data X and labels Y, and a predict(X) function which just takes in some data X and makes a prediction that we will try to make close to the corresponding Y. Predict We know that for neural networks the predict function is also called the feedforward action, and this is simply the dot product and a nonlinear function on each layer of the neural network. e.g. z1 = s(w0x), z2 = s(w1z1), z3 = s(w2z2), y = s(w3z3) We know that the nonlinearities we usually use in the hidden layers is usually a relu, sigmoid, or tanh. We know that the output is a sigmoid for binary classification and softmax for classification with >= 2 classes. Train We know that training a neural network simply is the application of gradient descent, which is the same thing we use for logistic regression and linear regression when we dont have a closed-form solution.

We know that linear regression has a closed form solution but we dont necessarily have to use it, and that gradient descent is a more general numerical optimization method. W W - learning_rate * dJ/dW We know that libraries like Theano and TensorFlow will calculate the gradient for us, which can get very complicated the more layers there are. Youll be thankful for this feature of neural networks when you see that the output function becomes even more complex when we incorporate convolution (although the derivation is still do-able and I would recommend trying for practice). At this point you should be familiar with how the cost function J is derived from the likelihood and how we might not calculate J over the entire training data set but rather in batches to improve training time. If you want to learn more about backpropagation and gradient descent youll want to check out my first course on deep learning, Deep Learning in Python part 1, which you can find at https://udemy.com/data-science-deep-learning-in-python Data Preprocessing When we work with images you know that an image is really a 2-D array of data, and that if we have a color image we have a 3-D array of data where one extra dimension is for the red, green, and blue channels. In the past, weve flattened this array into a vector, which is the usual input into a neural network, so for example a 28 x 28 image becomes a 784 vector, and a 3 x 32 x 32 image becomes a 3072 dimensional vector.

In this book, we are going to keep the dimensions of the original image for a portion of the processing. Where to get the data used in this book This book will use the MNIST dataset (handwritten digits) and the streetview house number (SVHN) dataset. The streetview house number dataset is a much harder problem than MNIST since the images are in color, the digits can be at an angle and in different styles or fonts, and the dimensionality is much larger. To get the code we use in this book youll want to go to: https://github.com/lazyprogrammer/machine_learning_examples And look in the folder: cnn_class If youve already checked out this repo then simply do a git pull since this code will be on the master branch. I would highly recommend NOT just running this code but using it as a backup if yours doesnt work, and try to follow along with the code examples by typing them out yourself to build muscle memory. Once you have the machine_learning_examples repo youll want to create a folder adjacent to the cnn_class folder called large_files if you havent already done that for a previous class.

That is where we will expect all the data to reside. To get the MNIST data, youll want to go to https://www.kaggle.com/c/digit-recognizer I think its pretty straightforward to download at that point. Were only going to use the train.csv file since thats the one with labels. You are more than welcome to attempt the challenge and submit a solution using the techniques you learn in this class. You can get the streetview house number data from http://ufldl.stanford.edu/housenumbers/ Youll want to get the files under format 2, which are the cropped digits.

If you really want to dig deep on this topic youll want to take a course on signal processing or linear systems. So what is convolution? Think of your favorite audio effect (suppose thats the echo). An echo is simply the same sound bouncing back at you in the future, but with less volume. Well see how we can do that mathematically later. All effects can be thought of as filters, like the one Ive shown here, and they are often drawn in block diagrams. ------- x(t)--->| h(t) |--->y(t) ------- Im representing our audio signal by this triangle. ------- x(t)--->| h(t) |--->y(t) ------- Im representing our audio signal by this triangle.