Ian Pointer - Programming PyTorch for Deep Learning: Creating and Deploying Deep Learning Applications

• 2019-09-20: First Release

Hello and welcome! This book will introduce you to deep learning via PyTorch, an open source library released by Facebook in 2017. Unless youve had your head stuck in the ground in a very good impression of an ostrich the past few years, you cant have helped but notice that neural networks are everywhere these days. Theyve gone from being the really cool bit of computer science that people learn about and then do nothing with to being carried around with us in our phones every day to improve our pictures or listen to our voice commands. Our email software reads our email and produces context-sensitive replies, our speakers listen out for us, cars drive by themselves, and the computer has finally bested humans at Go. Were also seeing the technology being used for more nefarious ends in authoritarian countries, where neural networkbacked sentinels can pick faces out of crowds and make a decision on whether they should be apprehended.

And yet, despite the feeling that this has all happened so fast, the concepts of convolutional neural networks were being used to recognize digits on check in the late 90s. Theres been a solid foundation building up all this time, so why does it feel like an explosion occurred in the last 10 years?

There are many reasons, but prime among them has to be the surge in tensor processing units (TPUs), which are devices custom-built for performing deep learning as fast as possible, and are even available to the general public as part of their Google Cloud ecosystem.

Another way to chart deep learnings progress over the past decade is through the ImageNet competition. A massive database of over 14 million pictures, manually labeled into 20,000 categories, ImageNet is a treasure trove of labeled data for machine learning purposes. Since 2010, the yearly ImageNet Large Scale Visual Recognition Challenge has sought to test all comers against a 1,000-category subset of the database, and until 2012, error rates for tackling the challenge rested around 25%. That year, however, a deep convolutional neural network won the competition with an error of 16%, massively outperforming all other entrants. In the years that followed, that error rate got pushed down further and further, to the point that in 2015, the ResNet architecture obtained a result of 3.6%, which beat the average human performance on ImageNet (5%). We had been outclassed.