Preface
Distributed computing is a fascinating topic.Looking back at the early days of computing, one cant help but be impressed by the fact that so many companies today distribute their workloads across clusters of computers.Its impressive that we have figured out efficient ways to do so, but scaling out is also becoming a necessity.Individual computers keep getting faster and more powerful, and yet our need for large scale computing keeps exceeding what single machines can do.
Recognizing that scaling is both a necessity and a challenge, Ray aims to make distributed computing easy and simple for developers.It makes distributed computing accessible to non-experts and makes it possible to scale your Python scripts across multiple nodes fairly easily.Ray is good at scaling both data and compute heavy workloads, such as data preprocessing and model trainingand it explicitly targets machine learning (ML) workloads with the need to scale.While it is possible today to scale these two types of workloads without Ray, you would likely have to use different APIs and distributed systems for each.But managing and deploying several (distributed) systems can be messy and inefficient in many ways.
The addition of the Ray AI Runtime (AIR) with the release of Ray 2.0 in August 2022 increased the support for complex ML workloads in Ray even further.AIR is a collection of libraries and tools that make it easy to build and deploy end-to-end ML applications in a single distributed system.With AIR, even the most complex workflows can usually be expressed as a single Python script.That means you can run your programs locally first, which can make a big difference in terms of debugging and development speed.
Data scientists benefit from Ray, as they can rely on a growing ecosystem of Ray ML libraries and third party integrations.Ray AIR helps you to quickly prototype ideas and go more easily from development to production.Unlike many other distributed systems, Ray has native support for GPUs as well, which can be particularly important to roles like ML engineers.To support data engineers, Ray also has tight integrations with tools like Kubernetes and can be deployed in multi-cloud setups.You can use it as unified compute layer to provide scaling, fault tolerance, scheduling and orchestration of your workloads.By auto-scaling your Ray clusters you can improve utilization and reduce cost.
Who Should Read This Book
Its likely that you picked up this book because youre interested in some aspects of Ray.Maybe youre a distributed systems engineer who wants to know how Rays engine works.You might also be a software developer interested in picking up a new technology.Or you could be a data engineer who wants to evaluate how Ray compares to similar tools.You could also be a machine learning practitioner or data scientist who needs to find ways to scale his experiments.
No matter what your concrete role is, the common denominator to get the most out of this book is to feel comfortable programming in Python.This is a book written in Python and an intermediate knowledge of the language is a requirement.Explicit is better than implicit, as you know full well as Pythonista.So, let us be explicit by saying that knowing Python implies to me that you know how to use the command line on your system, how to get help when stuck, and how to set up a programming environment on your own.
If youve never worked with distributed systems before, thats ok.We cover all the basics you need to get started with this topic in the book.On top of that, you can run most code examples presented here on your laptop alone.At the same time, covering the basics means that we cant go into too much detail about distributed systems in this book.This book is ultimately focused on application developers using Ray, specifically for data science and machine learning (ML).
If you want to follow the later chapters of this book, you need some familiarity with ML, but we dont expect you to have worked in the field.In particular, you should have a basic understanding of the machine learning paradigm and how it differs from traditional programming.You should also know the basics of using NumPy and Pandas.On top of that, you should at least feel comfortable reading examples using the popular TensorFlow and PyTorch libraries.Its enough to follow the flow of the code, on the API level, but you dont need to know how to write your own models.
We cover a lot of ground in advanced machine learning topics, but the main focus is on Ray as a technology and how to use it.The ML examples we discuss might be new to you and could require a second reading, but you can still focus on Rays API and how to use it in practice.
Knowing the requirements, lets discuss what you might get out of this book.
As a data scientist, Ray will open up new ways for you to think about and build distributed ML applications.You will know how to do hyperparameter selection for your experiments at scale, gain practical knowledge on large-scale model training and get to know a state-of-the-art reinforcement learning library.
As a data engineer you will learn to use Ray Datasets for large-scale data ingest, how to improve your pipelines by leveraging tools such as Dask on Ray, and how to effectively deploy models at scale.