Caleb Hattingh - Using Asyncio in Python 3

by Caleb Hattingh

Copyright 2018 OReilly Media, Inc. All rights reserved.

Printed in the United States of America.

Published by OReilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472.

OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com/safari). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .

• Editors: Jeff Bleiel and Susan Conant
• Production Editor: Nicholas Adams
• Copyeditor: Dwight Ramsey
• Interior Designer: David Futato
• Cover Designer: Karen Montgomery
• Illustrator: Rebecca Demarest
• Tech Reviewer: Yury Selivanov

• March 2018: First Edition

The OReilly logo is a registered trademark of OReilly Media, Inc. Using Asyncio in Python 3, the cover image, and related trade dress are trademarks of OReilly Media, Inc.

While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-491-99968-4

[LSI]

Python 3.4 introduced the asyncio library, and Python 3.5 producedthe async and await keywords to use it palatably. These newadditions allow so-called asynchronous programming.

All of these new features, which well refer to under the single nameAsyncio, have been received by the Python community somewhat warily;there seems to be a segment of the community that views the new Asynciofeatures as very complex and difficult to understand. This view isnot limited to beginners: several high-profile contributors to thePython community have expressed doubts about the complexity of the AsyncioAPI in Python, and educators in the community have expressed concernabout how best to teach Asyncio to students.

Most people with a few years experience in Python have used threadsbefore, and even if you havent, you are still likely to haveexperienced blocking before. For example, if youve written programsusing the wonderful requests library, you will surely have noticedhow your program pauses for a bit while it does requests.get(url); thisis blocking behavior.

For one-off tasks this is fine, but if you want to fetch tenthousand URLs simultaneously, its going to be difficult to userequests; so large-scale concurrency is a big reason to learn anduse Asyncio. But the other big attraction of Asyncio over preemptivethreading is safety: it will be much easier for you to avoidrace-condition bugs with Asyncio.

My goal with this book is to help as much as I can to give you a basicunderstanding of why these new features have been introduced, how tounderstand them, and how to approach making use of these new featuresin your own projects. More specifically, Im aiming to provide:

1. a critical comparison of asyncio and threading for concurrentnetwork programming

2. understanding of the new async/awaitlanguage syntax

3. a general overview of the new asynciostandard library featuresin Python

4. detailed, extended case studies with code, showing how to use afew of the more popular Asyncio-compatible third-party libraries

Well begin with a story that illustrates the shift in thinking thatmust accompany a transition from threaded to async programming. Then,in the next part we take a look at the changes in the Python languageitself that were made to accommodate async programming. Finally, wellexplore some of the ways in which these new features can be used mosteffectively.

The new Asyncio features are not going to radically change the way youwrite programs. They provide specific tools that make sense only forspecific situations; but in the right situations, asynciois exceptionally useful. In this book were going to describe thosesituations, and how you can best approach them using the new Asynciofeatures.