Masato Hagiwara - Real-World Natural Language Processing

Masato Hagiwara

To comment go to liveBook

Manning

Shelter Island

For more information on this and other Manning titles go to

www.manning.com

For online information and ordering of these and other Manning books, please visit www.manning.com. The publisher offers discounts on these books when ordered in quantity.

For more information, please contact

Special Sales Department

Manning Publications Co.

20 Baldwin Road

PO Box 761

Shelter Island, NY 11964

Email: orders@manning.com

2021 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps.

Recognizing the importance of preserving what has been written, it is Mannings policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine.

ISBN: 9781617296420

To Daphne, Laurel, and Lynn

Having worked at the intersection of machine learning (ML), natural language processing (NLP), and education for the last two decades, I have always been passionate about education and helping people learn new technologies. Thats why I didnt think twice when I heard about the opportunity of publishing a book on NLP.

The field of artificial intelligence (AI) went through a lot of changes over the past several years, including the explosive popularization of neural network-based methods and the advent of large, pretrained language models. This change made advanced language technologies possible, many of which you interact with dailyvoice-based virtual assistants, speech recognition, and machine translation, to name a few. However, the technology stack of NLP, characterized by the use of pretrained models and transfer learning, has finally stabilized in the last few years and is expected to remain so, at least for the next couple of years. This is why I think now is a good time to start learning about NLP.

Developing a book on AI is never easy. It feels like you are chasing a moving target that doesnt slow down and wait for you. When I started writing this book, the Transformer had just been published, and BERT did not yet exist. Over the course of writing, AllenNLP, the main NLP framework we use in this book, went through two major updates. Few people were using Hugging Face Transformer, a widely popular deep NLP library currently used by many practitioners all over the world. Within two years, the landscape of the NLP field changed completely, due to the advent of the Transformer and pretrained language models such as BERT. The good news is that the basics of modern machine learning, including word and sentence embeddings, RNNs, and CNNs, have not become obsolete and remain important. This book intends to capture this core of ideas and concepts that help you build real-world NLP applications.

Many great books about ML and deep learning in general are on the market, but some of them focus heavily on math and theories. Theres a gap between whats taught in books and what the industry needs. I hope this book will serve to bridge this gap.

This book would not be possible without the help of many people. I must start by thanking Karen Miller, the development editor at Manning Publications. Thank you for your support and patience during the development of this book. Im also grateful for the rest of the Manning team: technical development editor Mike Shepard, review editor Adriana Sabo, production editor Deirdre Hiam, copy editor Pamela Hunt, proofreader Keri Hales, and technical proofreader Mayur Patil. Denny ( http://www.designsonline.id/ ) also created some of the high-quality illustrations you see in this book.

Id also like to thank the reviewers who gave valuable feedback after reading the manuscript of this book: Al Krinker, Alain Lompo, Anutosh Ghosh, Brian S. Cole, Cass Petrus, Charles Soetan, Dan Sheikh, Emmanuel Medina Lopez, Frdric Flayol, George L. Gaines, James Black, Justin Coulston, Lin Chen, Linda Ristevski, Luis Moux, Marc-Anthony Taylor, Mike Rosencrantz, Nikos Kanakaris, Ninoslav erkez, Richard Vaughan, Robert Diana, Roger Meli, Salvatore Campagna, Shanker Janakiraman, Stuart Perks, Taylor Delehanty, and Tom Heiman.

Id like to acknowledge the AllenNLP team at the Allen Institute for Artificial Intelligence. Ive had great discussions with the team, namely, Matt Gardner, Mark Neumann, and Michael Schmitz. I always look up to their great work that makes deep NLP technologies easy and accessible to the world.

Last but not least, Id like to thank my awesome wife, Lynn. She not only helped me choose the right cover image for this book but has also been understanding and supportive of my work throughout the development of this book.

Real-World Natural Language Processing is not a typical NLP textbook. We focus on building real-world NLP applications. Real-worlds meaning here is twofold: first, we pay attention to what it takes to build real-world NLP applications. As a reader, you will learn not just how to train NLP models but also how to design, develop, deploy, and monitor them. Along the way, you will also learn the basic building blocks of modern NLP models, as well as recent developments in the NLP field that are useful for building NLP applications. Second, unlike most introductory books, we take a top-down approach to teaching. Instead of a bottom-up approach, spending page after page showing neural network theories and mathematical formulae, we focus on quickly building NLP applications that just work. We then dive deeper into individual concepts and models that make up NLP applications. Youll also learn how to build end-to-end custom NLP applications tailored to your needs using these basic building blocks.