Thushan Ganegedara - Natural Language Processing with TensorFlow: The definitive NLP book to implement the most sought-after machine learning models and tasks, 2nd Edition

Natural Language Processing with TensorFlow

Second Edition

The definitive NLP book to implement the most sought-after machine learning models and tasks

Thushan Ganegedara

BIRMINGHAMMUMBAI

Natural Language Processing with TensorFlow

Second Edition

Copyright 2022 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Senior Publishing Product Manager: Tushar Gupta

Acquisition Editor Peer Reviews: Saby Dsilva

Project Editor: Parvathy Nair

Content Development Editor: Georgia Daisy van der Post

Copy Editor: Safis Editing

Technical Editor: Tejas Mhasvekar

Proofreader: Safis Editing

Indexer: Subalakshmi Govindhan

Presentation Designer: Rajesh Shirsath

First published: May 2018

Second edition: July 2022

Production reference: 1260722

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-83864-135-1

www.packt.com

This book addresses the important need for describing how Natural Language Processing (NLP) problems can be solved using TensorFlow-based NLP stacks.

Deep Learning revolutionized NLP recently. Many industrial and academic NLP problems that required a large amount of work in terms of designing new features, tuning models, and finding the best modeling approach (CRF, SVM, Bayesian methods, etc.) can now be solved by NLP scientists in a significantly smaller amount of time. Also, the new deep-learning-based methods typically produce much more accurate models than traditional NLP methods. The big problem is how to make these models work in a modern production setting with operational parameters such as latency and throughput, cloud costs, and operational quality (uptime, etc.). The TensorFlow environment is designed to solve these problems when running NLP models.

In this book, the author teaches the fundamentals of TensorFlow and Keras, a Python-based interface for TensorFlow. Then, the bulk of the book, from Chapter 3, Word2vec Learning Word Embeddings, onward, is focused on NLP problems and solving them using TensorFlow.

This book provides:

• A knowledge of NLP methods in good detail, from their definition to various evaluation methods
• Information about TensorFlow, Keras, and Hugging Face libraries, which are powerful tools to build NLP solutions
• An understanding of neural architectures, which is important to build better models, by building architectures for specific tasks that the reader will encounter in their practice.

The author describes the process of building embeddings and other vector representations that are the basis of most modern deep learning NLP methods. The author also describes popular Neural Network architectures, such as Recurrent Neural Networks, Convolutional Neural Networks, Long Short-Term Memory networks, and Transformer-based architectures, in detail and shows their application in solving various NLP tasks, such as sentence classification, named entity recognition, text generation, machine translation, image caption generation, and more.

In each chapter, the author provides a deep dive into the neural network architecture, with an explanation of why this architecture works; the nature of the NLP problem and why it is an important NLP task; and how the solution to the problem is evaluated. Such deep dives will help readers to address industrial tasks that are reducible to these NLP problems, and to solve other NLP problems through understanding how typical NLP problems are evaluated. These deep dives will also help provide the reader with the knowledge to modify and improve necessary network architectures for particular practical tasks. The author also provides a detailed, step-by-step description of how such models are trained in a TensorFlow/Keras environment.

At the end, the author writes about Transformers, the modern state-of-the-art method to solve NLP problems, with a focus on BERT (a popular transformer method developed by Google). The author provides exercises on how BERT can be used for practical tasks such as answering questions, but the explanations of BERT will also help to solve other tasks with BERT-based networks. The author also dives into Hugging Face, a popular software library for transformer-based NLP solutions.

All of this content makes this book invaluable for practitioners who want to learn how to build TensorFlow-based solutions for NLP problems.

Andrei Lopatenko

VP Engineering and Head of Search & NLP at Zillow

Thushan Ganegedara is a Senior Machine Learning engineer at Canva, an Australian technology unicorn thats democratizing graphic designing and visualizations.. Thushan works with large-scale visual and text data, in order to build and deploy Machine Learning models to make products smarter. Before this, Thushan worked as a Senior Data Scientist at QBE Insurance, helping to solve business problems and make claim processing more efficient using machine learning. Thushan has a PhD from the University of Sydney specializing in Deep Learning.

I would like to acknowledge my parents and my wife, Thushani, for all the support and encouragement provided during the development of this book.

Arman Cohan is a Research Scientist at the Allen Institute for AI (AI2). His broad research interest is in developing Natural Language Processing methods for addressing information overload. This includes language models for complex document and multi-document tasks, natural language generation and summarization, and information discovery and filtering. His research has been recognized with multiple awards from leading conferences in the field, including a best paper award at EMNLP 2017, an honorable mention at COLING 2018, and the 2019 Harold N. Glassman Distinguished Doctoral Dissertation award.

Pratik Kotian is a Senior Conversation AI engineer with six years of experience in building conversational AI agents and designing products related to conversational design. He is working as a Senior Conversation Bot Engineer (specializing in conversational AI) at Quantiphi, which is an AI company and recognized Google Partner. He has also worked with Packt on reviewing The TensorFlow Workshop and Conversational AI with Rasa.

I would like to thank my family and friends, who are always supportive and have always believed in me and my talents. Its because of them that I am doing well in my career. And lastly, I would like to thank all the readers of this book: you are definitely going to learn a lot about recent developments in NLP and TensorFlow from this book.