Stephan Raaijmakers - Deep Learning for Natural Language Processing MEAP V04

Thank you for purchasing the MEAP for Deep Learning for Natural Language Processing. This book will teach you to apply deep learning to one of most vibrant applications of current AI: the analysis of natural language. The book is addressed to anyone who needs a practical guide to current deep learning techniques, from developers to students and researchers of computational linguistics.

While rooted in linguistics, the field of computational linguistics, or Natural Language Processing (NLP, for short), borrows many tools and techniques from neighboring fields such as computer science, logic, statistics and machine learning. Much of the current research in NLP now uses deep learning. It is quite hard to keep track of the many papers and software repositories that are being produced in this fast-paced field. You may even feel youre missing out on the cutting edge stuff. This is exactly what this book attempts to remedy: it will open up the toolbox of deep learning for NLP practitioners, outlining the pros and cons of various actual techniques and architectures. It speaks the language of NLP and is actually written by a computational linguist.

The book is written to be self-contained, but cannot cover all details of deep learning and NLP. Two excellent companion books, which can either be used as reference or primer, would be Mannings Deep Learning with Python, and Natural Language Processing in Action. All examples in this book are written in Python and the elegant Keras library. This is a code-intensive book, with lots of examples of deep learning solutions for well-known NLP problems. The book contains references to recent literature and some of our code draws inspiration from great public domain software repositories, including the source code examples in the Keras distribution.

Deep learning is under continuous development and may seem like a volatile moving target. I hope this practical and hands-on book will help you become proficient in the current types of deep learning, and will provide you with a solid basis for keeping up with new developments.

Writing an ambitious book like this one is a daunting task, and is prone to imperfection. Your feedback is more than welcome. If you have any questions, comments, or suggestions, please share them in Mannings Author Online forum for my book.

Stephan Raaijmakers

Brief Table of Contents

Part 1Introduction

Part 2Deep NLP

Part 3Advanced topics

Attention

Multitask learning

Appendixes:

Matrix algebra

Hyperparameter estimation and classifier performance evaluation

Current natural language processing heavily relies on machine learning. Machine learning has its roots in statistics, building among others on the seminal work by Thomas Bayes and LaPlace in the 18-th and 19-th century ( Bayes , LaPlace ) and the least squares methods for curve approximation by Legendre in 1812 ( legendre ). The field of neural computing started with the work of McCulloch and Pitts in 1943, who put forward a formal theory (and logical calculus) of neural networks ( mccullochpits43 ). It would take until 1950 before learning machines were proposed by Alan Turing ( Turing .

All machine learning algorithms that perform classification (labeling) share a single goal: to arrive at linear separability of data that is labeled with classes