Sandro Skansi - Guide to Deep Learning Basics: Logical, Historical and Philosophical Perspectives

This Springer imprint is published by the registered company Springer Nature Switzerland AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Artificial neural networks came to existence in 1943 with the seminal paper by Walter Pitts and Warren McCulloch. It is commonly said that the rest is history. But this history, which is seldom explored, holds many interesting details. From the purely historical unknowns to the conceptual connections often spanning back to medieval and even classical times which we often take for granted. By doing so, we often simplify things to a degree when it is no longer evident how rich and intricate the history (and prehistory) of deep learning was. The present volume brings new light on some foundational issues, and we hope that it will shed a new light on this amazing field of research.

My personal pivotal point for editing this volume was the discovery of a lost Croatian machine translation project from 1959. It was interesting to see how I was rediscovering the ideas that were so geographically close, and yet so remote and lost. But one question arose: If there was a whole project in machine translation no one knows about, what else is there to dig out? Can we find new old ideas that contribute to the rich history deep learning? Or more generally, how did this amazing field survive against the tide and finally flourish? Such complex history is bound to have nooks and crannies just waiting to be rediscovered and explored like a lost city of a bygone civilization.

But we could ask this general historical and archeological question in a different way, making it sound more philosophical and analytical: Was the success of deep learning wholly due to its technological superiority? Or was deep learning conceptually a better theory, but it did not prosper due to high computational needs not available back in the day? Following McCarthy, GOFAI separated itself from its philosophical backgrounds, and by doing so it shed layer after layer of what was seen as computational inefficiencies. But was this (past) methodological necessity also a conceptual necessity? Deep learning more often than not embraced its humanistic call and tried hard not to dismiss problems that were too vague or imprecise to tackle, even at the cost of not producing working systems. It can be argued, as some authors of this monograph do, that this focus on concepts rather than working production systems has paved the way for the dramatic rise of deep learning, and in turn enabled it to surpass GOFAI and develop better working AI systems.

One could ask what could such a book do for a technical field such as deep learning. It is my deep belief that as technology progresses, it needs philosophy and even art to make room for it in the common culture and in the everyday lives of people. And this is a challenge for the professional, not for the common person. People need to feel the need for science and technology, and welcome it in their lives without fear or reservation. And this can only be done if the scientist and coder are able to take a step back from its lab, GPUs and code, and explain the why. It is our hope that this book will help in doing so.

This book is not technical, and definitely not made to be inaccessible, but since it explores very specific and sometimes demanding ideas from a very abstract perspective it might come across daunting. Some people are geared in such a way that they like this perspective and need to see the big picture first before digging in into the technical details. Other people will not take kindly on philosophizing without first tackling and idea heads-on and coding. I can guarantee that both of these types of people will find interesting topics explored here, but the ideal reader of this volume is a wholly different person. She is someone who knows deep learning, and has spent countless hours studying it and coding. By knowing how deep learning systems are made and exactly how they work, she has become somewhat disenchanted by it. She is thinking that deep learning and AI in general is actually simple computation. It is our hope, by going sometimes far beyond actual applications, that we will rekindle her passion for this unique discipline and show that even when everything technical is mastered, and the magic goes away, that there is still something special, unique, and mystical on the very edges of deep learning. The ideas presented here are lights and echoes over the horizon.

As a final note, I would like to note that the authors of the chapters who are affiliated with the University of Zagreb but are not current employees of the university have their affiliation written as University of Zagreb with no faculty or department. I would also like to thank the reviewers and editors at Springer for all their help. A volume such as this is necessarily incomplete (since there are many interesting facets not covered), and I hold only myself accountable for any such incompleteness.