Baker - Serious Python: Black-Belt Advice on Deployment, Scalability, Testing, and More

Black-Belt Advice on Deployment, Scalability, Testing, and More

by Julien Danjou

San Francisco

SERIOUS PYTHON. Copyright 2019 by Julien Danjou.

All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher.

ISBN-10: 1-59327-878-0
ISBN-13: 978-1-59327-878-6

Publisher: William Pollock
Production Editor: Laurel Chun
Cover Illustration: Josh Ellingson
Interior Design: Octopod Studios
Developmental Editors: Liz Chadwick with Ellie Bru
Technical Reviewer: Mike Driscoll
Copyeditor: Paula L. Fleming
Compositor: Laurel Chun
Proofreader: James Fraleigh

For information on distribution, translations, or bulk sales, please contact No Starch Press, Inc. directly:

No Starch Press, Inc.
245 8th Street, San Francisco, CA 94103
phone: 1.415.863.9900;
www.nostarch.com

Library of Congress Cataloging-in-Publication Data

Names: Danjou, Julien, author.
Title: Serious Python : black-belt advice on deployment, scalability,
testing, and more / Julien Danjou.
Description: San Francisco, CA : No Starch Press, Inc., [2019].
Identifiers: LCCN 2018042631 (print) | LCCN 2018050473 (ebook) | ISBN
9781593278793 (epub) | ISBN 1593278799 (epub) | ISBN 9781593278786 (print)
| ISBN 1593278780 (print) | ISBN 9781593278793 (ebook) | ISBN 1593278799
(ebook)
Subjects: LCSH: Python (Computer program language)
Classification: LCC QA76.73.P98 (ebook) | LCC QA76.73.P98 D36 2019 (print) |
DDC 005.13/3--dc23
LC record available at https://lccn.loc.gov/2018042631

No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product and company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark.

The information in this book is distributed on an As Is basis, without warranty. While every precaution has been taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in it.

Julien Danjou has been a free software hacker for close to twenty years and has been developing software with Python for twelve years. He currently works as Project Team Leader for the distributed cloud platform OpenStack, which has the largest existing open-source Python codebase at 2.5 million lines of Python. Before building clouds, Julien created the awesome window manager and contributed to various software such as Debian and GNU Emacs.

Mike Driscoll has been programming with Python for more than a decade. He has been writing about Python on his blog, The Mouse vs. The Python, for many years. Mike is the author of several Python books including Python 101, Python Interviews, and ReportLab: PDF Processing with Python. You can find Mike on Twitter or GitHub via his handle: @driscollis.

STARTING YOUR PROJECT

MODULES, LIBRARIES, AND FRAMEWORKS

DOCUMENTATION AND GOOD API PRACTICE

HANDLING TIMESTAMPS AND TIME ZONES

DISTRIBUTING YOUR SOFTWARE

UNIT TESTING

METHODS AND DECORATORS

FUNCTIONAL PROGRAMMING

THE ABSTRACT SYNTAX TREE, HY, AND LISP-LIKE ATTRIBUTES

PERFORMANCES AND OPTIMIZATIONS

SCALING AND ARCHITECTURE

MANAGING RELATIONAL DATABASES

WRITE LESS, CODE MORE

Writing this first book has been a tremendous effort. Looking back, I had no clue how crazy this journey would be but also no idea how fulfilling it would turn out to be.

They say that if you want to go fast you should go alone, but that if you want to go far you should go together. This is the fourth edition of the original book I wrote, and I would not have made it here without the people who helped along the way. This is a team effort and I would like to thank everyone who participated.

Most of the interviewees gave me their time and trust without a second thought, and I owe a lot of what we teach in this book to them: Doug Hellmann for his great advice about building libraries, Joshua Harlow for his good humor and knowledge about distributed systems, Christophe de Vienne for his experience in building frameworks, Victor Stinner for his incredible CPython knowledge, Dimitri Fontaine for his database wisdom, Robert Collins for messing up with testing, Nick Coghlan for his work in getting Python into better shape, and Paul Tagliamonte for his amazing hacker spirit.

Thanks to the No Starch crew for working with me on bringing this book to a brand new level especially to Liz Chadwick for her editing skills, Laurel Chun for keeping me on track, and Mike Driscoll for his technical insight.

My gratitude also goes to the free software communities who shared their knowledge and helped me grow, especially to the Python community which always has been welcoming and enthusiastic.

If youre reading this, the odds are good youve been working with Python for some time already. Maybe you learned it using some tutorials, delved into some existing programs, or started from scratch. Whatever the case, youve hacked your way into learning it. Thats exactly how I got familiar with Python up until I started working on big open source projects 10 years ago.

It is easy to think that you know and understand Python once youve written your first program. The language is that simple to grasp. However, it takes years to master it and to develop a deep comprehension of its advantages and shortcomings.

When I started Python, I built my own Python libraries and applications on a garage project scale. Things changed once I started working with hundreds of developers on software that thousands of users rely on. For example, the OpenStack platforma project I contribute torepresents over 9 million lines of Python code, which collectively needs to be concise, efficient, and scalable to the needs of whatever cloud computing application its users require. When you have a project of this size, things like testing and documentation absolutely require automation, or else they wont get done at all.

I thought I knew a lot about Python before working on projects of this scalea scale I could hardly imagine when I started outbut Ive learned a lot more. Ive also had the opportunity to meet some of the best Python hackers in the industry and learn from them. Theyve taught me everything from general architecture and design principles to various helpful tips and tricks. Through this book, I hope to share the most important things Ive learned so that you can build better Python programsand build them more efficiently, too!