coll - A Greater Foundation for Machine Learning Engineering.

A GREATER

FOUNDATION FOR

MACHINE

LEARNING

ENGINEERING

The Hallmarks of the Great Beyond in

PyTorch, R, TensorFlow, and Python

DR. GANAPATHI PULIPAKA

Copyright 2021 by Dr. Ganapathi Pulipaka.

Pulipaka.

Pulipaka.

Pulipaka.

Pulipaka.

All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the copyright owner.

Any people depicted in stock imagery provided by Getty Images are models, and such images are being used for illustrative purposes only.

Certain stock imagery Getty Images.

Rev. date: 08/23/2021

Xlibris

844-714-8691

www.Xlibris.com

823838

CONTENTS

Foreword by the Author

About the Author

Preface

Chapter Introduction: A Greater Foundation for Machine Learning Engineering

Chapter Supervised Learning

Chapter Unsupervised Learning

Chapter Origins of Deep Learning

Chapter Linear Algebra

Chapter Calculus

Chapter Swarm Intelligence, Machine Learning Algorithms and In-Memory Computing

Chapter Deep Learning Frameworks

Chapter HPC with Deep Learning Frameworks

Chapter History of Supercomputing

Chapter Healthcare

Chapter Real-World Research Projects with Supercomputers

Chapter HPC with Parallel Computing

Chapter Installation of PyTorch

Chapter Introduction to reinforcement learning algorithms

Chapter Reinforcement Learning TRPO

Chapter Reinforcement Learning Cross-entropy Algorithm

Chapter Reinforcement Learning REINFORCE Algorithm

Chapter The Gridworld: Dynamic Programming with PyTorch and Reinforcement Learning - Frozen Lake

References

GANAPATHI

PULIPAKA

There have been significant developments from reinforcement learning in 2020 with accelerated commercialization of reinforcement learning algorithms from various industries. There has been a greater industry-academia collaboration with the implementation of papers, including autonomous vehicles for making complex decisions in dynamic environments in both discrete and continuous state spaces. A number of corporations have explored implementing reinforcement learning applications for edge computing applications for Industrial IoT and IoT. A number of research papers were released on both model-free reinforcement and model-based reinforcement learning algorithms. The significant advancements include MARL (Multi-agent reinforcement learning) framework COGMENT with internal interactions among the agents for leveraging the effective results from the observations and rewards in highly dynamic environments introducing a new technique Human-MARL learning, where a human individually cannot achieve the goal without an agent and agent cannot achieve the goal without the human. A new hybrid MARL reinforcement algorithm has been introduced dubbed D3-MADDPG (Dueling Double Deep Q Learning) for parallel training of decentralized policy rollouts with a joint centralized policy. Graph convolutional reinforcement learning algorithm has been introduced to learn to cooperate among humans and multiple agents in human-MARL environments. Deepmind has introduced behavior suite for reinforcement learning in 2020 with Python implementation. The Github code repository also shows some examples from OpenAI Baselines and Dopamine as reference implementations. Another reinforcement learning algorithm has been introduced in a randomized environment by Deepmind with randomized convolutional neural networks in 2D CoinRun, 3D DeepMind Lab, and 3D Robotics control tasks. Berkeleys research in 2020 on deep reinforcement learning algorithms have found new adversarial policies can be reapplied from a particular adversary with reinforcement learning. An encoder-decoder neural network has been developed to search for the direct acyclic graph with reinforcement learning for best scoring. The model-free based reinforcement learning algorithms in the Atari gaming environment have been applied to learn effective policies. However, a new reinforcement learning algorithm has been introduced with a mode-free reinforcement learning environment dubbed SimPLe.

PyTorch vs. TensorFlow

PyTorch from Facebook was released in 2017, and TensorFlow was released in 2015 by Google. In 2020, the line blurred as both frameworks have seen a convergence in terms of popularity and functionality. The hardships for the machine learning engineers start to fade away with TensorFlow 2.0 as there was a major revamp on the programming API of TensorFlow with the inclusion of Keras into the main API. TensorFlows static computational graphs were great for wrapping the modules and the ability to run on a number of devices such as CPUs, GPUs, or TPUs. However, its always hard to debug a static computational graph. PyTorch always had a dynamic computational graph that allowed the data scientists to perform the computation line by line as the code gets interpreted, making it easy to debug and identify the problem. In 2020, TensorFlow introduced a dynamic computational graph similar to PyTorch with Eager mode. Now, PyTorch also allows static computational graph. In 2020, TensorFlow works similar to PyTorch in many ways, including distributed computing with the ability to run on single or multiple distributed GPUs or CPUs. OpenAI in 2020 has announced PyTorch as their official machine learning framework for 2020 and 2021 reinforcement learning and all the other deep learning projects. PyTorch has shown rapid uptake in data science and deep learning engineering community, as the fastest-growing open-source projects according to Github report. According to the analysis conducted by Gradient, in 2019, the platform grew 50% year-over-year with every major AI conference presented papers implemented in PyTorch. OReilly mentioned that PyTorch citations grew by more than 194% in the first quarter of 2019 alone. While Israel has shown a 54% increase in the interest towards PyTorch, people from Columbia have shown more interest in TensorFlow at 84%. Overall, PyTorch has shown a giant leap of growth on PapersWithCode with PyTorch implementations when compared with TensorFlow.

Natural language processing