Ghatak - Deep Learning with R

The term Artificial Intelligence (AI) was coined by John McCarthy in 1956, but the journey to understand if machines can truly think began much before that. Vannevar Bush [1] in his seminal work As We May Think , proposed a system which amplifies peoples own knowledge and understanding.

Alan Turing was a pioneer in bringing AI from the realm of philosophical prediction to reality. He wrote a paper on the notion of machines being able to simulate human beings and the ability to do intelligent things. He also realized in the 1950s that it would need a greater understanding of human intelligence before we could hope to build machines which would think like humans. His paper titled Computing Machinery and Intelligence in 1950 (published in a philosophical journal called Mind ) opened the doors to the field that would be called AI , much before the term was actually adopted. The paper defined what would be known as the Turing test, which is a model for measuring intelligence.

Significant AI breakthroughs have been promised in the next 10 years, for the past 60 years. One of the proponents of AI, Marvin Minsky, claimed in 1967Within a generation , the problem of creating artificial intelligence will substantially be solved, and in 1970, he quantified his earlier prediction by statingIn from three to eight years we will have a machine with the general intelligence of a human being.

In the 1960s and early 1970s, several other experts believed it to be right around the corner. When it did not happen, it resulted in drying up of funds and a decline in research activities, resulting in what we term as the first AI winter .

During the 1980s, interest in an approach to AI known as expert systems started gathering momentum and a significant amount of money was being spent on research and development. By the beginning of the 1990s, due to the limited scope of expert systems , interest waned and this resulted in the second AI winter . Somehow, it appeared that expectations in AI always outpaced the results.

An expert system (ES) is a program that is designed to solve problems in a specific domain, which can replace a human expert. By mimicking the thinking of human experts, the expert system was envisaged to analyze and make decisions.

The knowledge base of an ES contains both factual knowledge and heuristic knowledge. The ES inference engine was supposed to provide a methodology for reasoning the information present in the knowledge base. Its goal was to come up with a recommendation, and to do so, it combined the facts of a specific case (input data), with the knowledge contained in the knowledge base (rules), resulting in a particular recommendation (answers).

Though ES was suitable to solve some well-defined logical problems, it proved otherwise in solving other types of complex problems like image classification and natural language processing (NLP). As a result, ES did not live up to its expectations and gave rise to a shift from the rule-based approach to a data-driven approach. This paved the way to a new era in AImachine learning.

Research over the past 60 years has resulted in significant advances in search algorithms, machine learning algorithms, and integrating statistical analysis to understand the world at large.

In machine learning, the system is trained rather than explicitly programmed (unlike that in ES). By exposing large quantities of known facts (input data and answers) to a learning mechanism and performing tuning sessions, we get a system that can make predictions or classifications of unseen input data. It does this by finding out the statistical structure of the input data (and the answers) and comes up with rules for automating the task.

Starting in the 1990s, machine learning has quickly become the most popular subfield of AI. This trend has also been driven by the availability of faster computing and availability of diverse data sets.

A machine learning algorithm transforms its input data into meaningful outputs by a process known as representations . Representations are transformations of the input data, to represent it closer to the expected output. Learning, in the context of machine learning, is an automatic search process for better representations of data. Machine learning algorithms find these representations by searching through a predefined set of operations.

To summarize, machine learning is searching for useful representations of the input data within a predefined space, using the loss function (difference between the actual output and the estimated output) as a feedback to modify the parameters of the model.

It turns out that machine learning focuses on learning only one or two layers of representations of the input data. This proved intractable for solving human perception problems like image classification, text-to-speech translation, handwriting transcription, etc. Therefore, it gave way to a new take on learning representations, which put an emphasis on learning multiple successive layers of representations, resulting in deep learning. The word deep in deep learning only implies the number of layers used in a deep learning model.

In deep learning, we deal with layers. A layer is a data transformation function which carries out the transformation of the data which goes through that layer. These transformations are parametrized by a set of weights and biases, which determine the transformation behavior at that layer.