Balakrishnan - TensorFlow reinforcement learning quick start guide: get up and running with training and deploying intelligent, self-learning agents using Python

Copyright 2019 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Commissioning Editor: Pravin Dandre
Acquisition Editor: Joshua Nadar
Content Development Editor: Roshan Kumar
Technical Editor: Snehal Dalmet
Copy Editor: Safis Editing
Project Coordinator: Namrata Swetta
Proofreader: Safis Editing
Indexer: Pratik Shirodkar
Graphics: Alishon Mendonsa
Production Coordinator: Aparna Bhagat

First published: March 2019

Production reference: 1290319

Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.

ISBN 978-1-78953-358-3

www.packtpub.com

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

• Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals

• Improve your learning with Skill Plans built especially for you

• Get a free eBook or video every month

• Mapt is fully searchable

• Copy and paste, print, and bookmark content

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at customercare@packtpub.com for more details.

At www.packt.com , you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Kaushik Balakrishnan works for BMW in Silicon Valley, and applies reinforcement learning, machine learning, and computer vision to solve problems in autonomous driving. Previously, he also worked at Ford Motor Company and NASA Jet Propulsion Laboratory. His primary expertise is in machine learning, computer vision, and high-performance computing, and he has worked on several projects involving both research and industrial applications. He has also worked on numerical simulations of rocket landings on planetary surfaces, and for this he developed several high-fidelity models that run efficiently on supercomputers. He holds a PhD in aerospace engineering from the Georgia Institute of Technology in Atlanta, Georgia.

Narotam Singh recently took voluntary retirement from his post of meteorologist with the Indian Meteorological Department, Ministry of Earth Sciences, to pursue his dream of learning and helping society. He has been actively involved with various technical programs and the training of GOI officers in the field of IT and communication. He did his masters in the field of electronics, having graduated with a degree in physics. He also holds a diploma and a postgraduate diploma in the field of computer engineering. Presently, he works as a freelancer. He has many research publications to his name and has also served as a technical reviewer for numerous books. His present research interests involve AI, ML, DL, robotics, and spirituality.

If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

This book provides a summary of several different reinforcement learning (RL) algorithms, including the theory involved in the algorithms as well as coding them using Python and TensorFlow. Specifically, the algorithms covered in this book are Q-learning, SARSA, DQN, DDPG, A3C, TRPO, and PPO. The applications of these RL algorithms include computer games from OpenAI Gym and autonomous driving using the TORCS racing car simulator.

This book is designed for machine learning (ML) practitioners interested in learning RL. It will help ML engineers, data scientists, and graduate students. A basic knowledge of ML, and experience of coding in Python and TensorFlow, is expected of the reader in order to be able to complete this book successfully .

, Up and Running with Reinforcement Learning, provides an overview of the basic concepts of RL, such as an agent, an environment, and the relationship between them. It also covers topics such as reward functions, discounted rewards, and value and advantage functions. The reader will also get familiar with the Bellman equation, on-policy and off-policy algorithms, as well as model-free and model-based RL algorithms.

, Temporal Difference, SARSA, and Q-learning , introduces the reader to temporal difference learning, SARSA, and Q-learning. It also summarizes how to code these algorithms in Python, and to train and test them on two classical RL problems GridWorld and Cliff Walking.

, Deep Q-Network, introduces the reader to the first deep RL algorithm of the book, DQN. It will also discuss how to code this in Python and TensorFlow. The code will then be used to train an RL agent to play Atari Breakout.

, Double DQN, Dueling Architectures, and Rainbow