II - Introduction to Deep Learning Using R A Step-by-Step Guide to Learning and Implementing Deep Learning Models Using R

Taweh Beysolow II 2017

Taweh Beysolow II 1

(1)

With advances in hardware and the emergence of big data, more advanced computing methods have become increasingly popular. Increasing consumer demand for better products and companies seeking to leverage their resources more efficiently have also been leading this push. In response to these market forces, we have recently seen a renewed and widely spoken about interest in the field of machine learning. At the cross-section of statistics, mathematics, and computer science, machine learning refers to the science of creating and studying algorithms that improve their own behavior in an iterative manner by design. Originally, the field was devoted to developing artificial intelligence, but due to the limitations of the theory and technology that were present at the time, it became more logical to focus these algorithms on specific tasks. Most machine learning algorithms as they exist now focus on function optimization, and the solutions yielded dont always explain the underlying trends within the data nor give the inferential power that artificial intelligence was trying to get close to. As such, using machine learning algorithms often becomes a repetitive trial and error process, in which the choice of algorithm across problems yields different performance results. This is fine in some contexts, but in the case of language modeling and computer vision, it becomes problematic.

In response to some of the shortcomings of machine learning, and the significant advance in the theoretical and technological capabilities at our disposal today, deep learning has emerged and is rapidly expanding as one of the most exciting fields of science. It is being used in technologies such as self-driving cars, image recognition on social media platforms, and translation of text from one language to others. Deep learning is the subfield of machine learning that is devoted to building algorithms that explain and learn a high and low level of abstractions of data that traditional machine learning algorithms often cannot. The models in deep learning are often inspired by many sources of knowledge, such as game theory and neuroscience, and many of the models often mimic the basic structure of a human nervous system. As the field advances, many researchers envision a world where software isnt nearly as hard coded as it often needs to be today, allowing for a more robust, generalized solution to solving problems.

Although it originally started in a space similar to machine learning, where the primary focus was constraint satisfaction to varying degrees of complexity, deep learning has now evolved to encompass a broader definition of algorithms that are able to understand multiple levels of representation of data that correspond to different hierarchies of complexity. In other words, the algorithms not only have predictive and classification ability, but they are able to learn different levels of complexity. An example of this is found in image recognition, where a neural network builds upon recognizing eyelashes, to faces, to people, and so on. The power in this is obvious: we can reach a level of complexity necessary to create intelligent software. We see this currently in features such as autocorrect, which models the suggested corrections to patterns of speech observed, specific to each persons vocabulary.

The structure of deep learning models often is such that they have layers of non-linear units that process data, or neurons, and the multiple layers in these models process different levels of abstraction of the data. Figure shows a visualization of the layers of neural networks.

Deep neural networks are distinguished by having many hidden layers, which are called hidden because we dont necessarily see what the inputs and outputs of these neurons are explicitly beyond knowing they are the output of the preceding layer. The addition of layers, and the functions inside the neurons of these layers, are what distinguish an individual architecture from another and establish the different use cases of a given model.

More specifically, lower levels of these models explain the how, and the higher-levels of neural networks process the why. The functions used in these layers are dependent on the use case, but often are customizable by the user, making them significantly more robust than the average machine learning models that are often used for classification and regression, for example. The assumption in deep learning models on a fundamental level is that the data being interpreted is generated by the interactions of different factors organized in layers. As such, having multiple layers allows the model to process the data such that it builds an understanding from simple aspects to larger constructs. The objective of these models is to perform tasks without the same degree of explicit instruction that many machine learning algorithms need. With respect to how these models are used, one of the main benefits is the promise they show when applied to unsupervised learning problems, or problems where we dont know prior to performing the experiment that the response variable y should be given a set of explanatory variables x . An example would be image recognition, particularly after a model has been trained against a given set of data. Lets say we input an image of a dog in the testing phase, implying that we dont tell the model what the picture is of. The neural network will start by recognizing eyelashes prior to a snout, prior to the shape of the dogs head, and so on until it classifies the image as that of a dog.