LazyProgrammer - Deep Learning in Python Prerequisites

Computer programs typically follow very deterministic processes.

IF THIS

THEN THAT

This is desired behavior for most programs. You wouldnt want a human doing arithmetic for you, or your operating system to make human errors when youre trying to get your work done.

One very useful application of computer programs is modeling or simulation. You can write physics simulations and models and use them in video games to produce realistic graphics and motion. We can do simulations using the equations of fluid mechanics to determine how an airplane with a new design would move through the air, without actually building it.

This leads us to an interesting question: Can we model the brain?

The brain is a complex object but we have decades of research that tells us how it works. The brain is made up of neurons that send electrical and chemical signals to each other.

We can certainly do electrical circuit simulations. We can do simulations of chemical reactions. We have circuit models of the neuron that simulate its behavior pretty accurately. So why cant we just hook these up and make a brain?

Realize that there is a heavy assumption here - that there is no soul, and that your consciousness is merely the product of very specifically organized biological circuits.

Whether or not that is true remains to be seen.

This is a very high-level view that kind of disappears when you study machine learning. The study of machine learning involves lots of math and optimization (finding the minimum or maximum of a function).

Another way to think of machine learning is its pattern recognition.

In human terms you would think of this as learning by example.

You learn that 1 + 1 = 2. 2 + 2 = 4. 1 + 2 = 3. And so on. You begin to figure out the pattern and then you learn to generalize that pattern to new problems.

You dont need to re-learn how to add 1000 + 1000, you just know how to add.

This is what machine learning tries to achieve.

In less abstract terms, machine learning very often works as follows:

You have a set of training samples:

X = { x , x , , x N }

Y = { y , y , , y N }

We call X the inputs and Y the outputs. Sometimes we call Y the target or labels and name it T instead of Y.

These come in pairs, so y is the output when the input is x , and so on.

We hope that, given enough examples of xs and ys, our machine learning algorithm will learn the pattern.

Then, when we later input a new x NEW into our model, we hope that the y NEW that it outputs is accurate.

Note that there are other types of learning, but what I described above, where we are given X and try to make it so that we can predict Y accurately, is called supervised learning.

There is a type of machine learning called unsupervised learning where we try to learn the distribution of the data (we are just given X). Clustering algorithms are an example of unsupervised learning. Principal components analysis is another example. While deep learning and neural networks can be used to do unsupervised learning and they are indeed very useful in that context, unsupervised learning doesnt really come into play with linear regression or logistic regression.

Another way to view machine learning is that we are trying to accurately model a system.

As an example, think of your brain driving a car. The inputs are the environment. The outputs are how you steer the car.

X [real world system] Y

An automated system to drive the car would be a program that outputs the best Ys.

X [machine learning model] Y prediction

We hope that after training or learning or fitting, Y prediction is approximately equal to Y.

To look at this from an API perspective, all supervised machine learning algorithms have 2 functions:

train(X, Y) where the model is adjusted to be able to predict Y accurately.

predict(X) where the model makes a prediction for each input it is given.