Matthew Fowler - Python Concurrency with asyncio

Matthew Fowler

To comment go to liveBook

Manning

Shelter Island

For more information on this and other Manning titles go to

www.manning.com

For online information and ordering of these and other Manning books, please visit www.manning.com. The publisher offers discounts on these books when ordered in quantity.

For more information, please contact

Special Sales Department

Manning Publications Co.

20 Baldwin Road

PO Box 761

Shelter Island, NY 11964

Email: orders@manning.com

2022 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps.

Recognizing the importance of preserving what has been written, it is Mannings policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine.

ISBN: 9781617298660

To my beautiful wife Kathy, thank you for always being there.

Nearly 20 years ago, I got my start in professional software engineering writing a mashup of Matlab, C++, and VB.net code to control and analyze data from mass spectrometers and other laboratory devices. The thrill of seeing a line of code trigger a machine to move how I wanted always stuck with me, and ever since then, I knew software engineering was the career for me. Over the years, I gradually moved toward API development and distributed systems, mainly focusing on Java and Scala, learning a lot of Python along the way.

I got my start in Python around 2015, primarily by working on a machine learning pipeline that took sensor data and used it to make predictionssuch as sleep tracking, step count, sit-to-stand transitions, and similar activitiesabout the sensors wearer. At the time, this machine learning pipeline was slow to the point that it was becoming a customer issue. One of the ways I worked on alleviating the issue was utilizing concurrency. As I dug into the knowledge available for learning concurrent programming in Python, I found things hard to navigate and learn compared to what I was used to in the Java world. Why doesnt multithreading work the same way that it would in Java? Does it make more sense to use multiprocessing? What about the newly introduced asyncio? What is the global interpreter lock, and why does it exist? There werent a lot of books on the topic of concurrency in Python, and most knowledge was scattered throughout documentation and a smattering of blogs with varying consistency of quality. Fast-forward to today, and things havent changed much. While there are more resources, the landscape is still sparse, disjointed, and not as friendly for newcomers to concurrency as it should be.

Of course, a lot has changed in the past several years. Back then, asyncio was in its infancy and has since become an important module in Python. Now, single-threaded concurrency models and coroutines are a core component of concurrency in Python, in addition to multithreading and multiprocessing. This means the concurrency landscape in Python has gotten larger and more complex, while still not having comprehensive resources for those wanting to learn it.

My motivation for writing this book was to fill this gap that exists in the Python landscape on the topic of concurrency, specifically with asyncio and single-threaded concurrency. I wanted to make the complex and under-documented topic of single-threaded concurrency more accessible to developers of all skill levels. I also wanted to write a book that would enhance generic understanding of concurrency topics outside of Python. Frameworks such as Node.js and languages such as Kotlin have single-threaded concurrency models and coroutines, so knowledge gained here is helpful in those domains as well. My hope is that all who read it find this book useful in their day-to-day lives as developersnot only within the Python landscape but also within the domain of concurrent programming.

First, I want to thank my wife, Kathy, who was always there for me to proofread when I wasnt sure if something made sense, and who was extremely supportive through the entire process. A close second goes to my dog, Dug, who was always around to drop his ball near me to remind me to take a break from writing to play.

Next, Id like to thank my editor, Doug Rudder, and my technical reviewer, Robert Wenner. Your feedback was invaluable in helping keep this book on schedule and high quality, ensuring that my code and explanations made sense and were easy to understand.

To all the reviewers: Alexey Vyskubov, Andy Miles, Charles M. Shelton, Chris Viner, Christopher Kottmyer, Clifford Thurber, Dan Sheikh, David Cabrero, Didier Garcia, Dimitrios Kouzis-Loukas, Eli Mayost, Gary Bake, Gonzalo Gabriel Jimnez Fuentes, Gregory A. Lussier, James Liu, Jeremy Chen, Kent R. Spillner, Lakshmi Narayanan Narasimhan, Leonardo Taccari, Matthias Busch, Pavel Filatov, Phillip Sorensen, Richard Vaughan, Sanjeev Kilarapu, Simeon Leyzerzon, Simon Tschke, Simone Sguazza, Sumit K. Singh, Viron Dadala, William Jamir Silva, and Zoheb Ainapore, your suggestions helped make this a better book.

Finally, I want to thank the countless number of teachers, coworkers, and mentors Ive had over the past years. Ive learned and grown so much from all of you. The sum of the experiences weve had together has given me the tools needed to produce this work as well as succeed in my career. Without all of you, I wouldnt be where I am today. Thank you!

Python Concurrency with asyncio was written to teach you how to utilize concurrency in Python to improve application performance, throughput, and responsiveness. We start by focusing on core concurrency topics, explaining how asyncios model of single-threaded concurrency works as well as how coroutines and async/await syntax works. We then transition into practical applications of concurrency, such as making multiple web requests or database queries concurrently, managing threads and processes, building web applications, and handling synchronization issues.