Moritz Lenz - Python Continuous Integration and Delivery: A Concise Guide with Examples

Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub via the books product page, located at www.apress.com/9781484242803 . For more detailed information, please visit www.apress.com/source-code .

One of the keys to successful software development is getting fast feedback. This helps developers avoid going down blind alleys, and in the case of a bug that is revealed quickly, it can be fixed while the code is still fresh in the developers mind.

On the business side, fast feedback also helps the stakeholders and the product manager not to build features that turn out not to be useful, thus avoiding wasted effort. Achieving a mutual understanding about the desired product is a very difficult problem in any software project. Showing a working (if partial) product early on often helps to eliminate misunderstandings between stakeholders and developers.

There are many ways to add feedback loops on different levels, from adding linting and other code checks in the IDE to agile processes that emphasize incremental value delivery. The first part of this book focuses on software tests and their automatic execution, a practice known as continuous integration (CI).

When implementing CI, you set up a server that automatically tests every change to the source code, potentially in multiple environments, such as on combinations of operating system and programming language versions.

The next logical step, and the topic of the second part of this book, is continuous delivery (CD). After building and testing code, you add more steps to the automated process: automated deployment to one or more test environments, more tests in the installed state, and, finally, deployment to a production environment. The last step is typically guarded by a manual approval gate.

CD extends the automation and, thus, the possibility for quick iteration cycles, all the way into the production environment, where the software can deliver value for you. With such a mechanism in place, you can quickly obtain feedback or usage data from real customers and assess whether expanding on a feature is useful or discover bugs while the developers still remember the code that they wrote.

The code examples in this book use Python. Owing to its dynamic nature, Python is well-suited to small experiments and fast feedback. The well-stocked standard library and vast ecosystem of available libraries and frameworks, as well as Pythons clean syntax, make it a good choice, even for larger applications. Python is commonly used in many domains, for example, web development, data science and machine learning, Internet of things (IoT), and system automation. It is becoming the lingua franca of professional programmers and those who just touch a subject to automate some part of their job or hobby.

Python comes in two major language versions, 2 and 3. Because Python 2 support is scheduled to end in 2020, and nearly all major libraries now support Python 3, new projects should be started in Python 3, and legacy applications should be ported to that language version as well, if possible. Hence, this book assumes that Python refers to Python 3, unless explicitly stated otherwise. If you only know Python 2, rest assured that you will easily understand the source code contained in this book, and transferring the knowledge to Python 3 is very easy.

This book is targeted at technical people involved in the software-delivery process: software developers, architects, release engineers, and DevOps engineers.

The chapters that use source code examples assume basic familiarity with the Python programming language. If you are familiar with other programming languages, spending a few hours reading introductory material on Python will likely bring you to a level at which you can easily follow the code examples in this book.

The sample infrastructure uses Debian GNU/Linux, so familiarity with that operating system is helpful, though not required.

Code examples used in this book are available on GitHub under the python-ci-cd organization at https://github.com/python-ci-cd or via the Download Source Code link located at www.apress.com/9781484242803 .

Because some examples rely on automatically fetching code from certain Git repositories, a split into several repositories is necessary. Several chapters reference individual repositories under this namespace.

Writing a book is not a solitary endeavor and is only possible with help from many individuals and organizations. I would like to thank all of my beta readers who provided feedback. These include, in no particular order, Karl Vogel, Mikhail Itkin, Carl Msak, Martin Thurn, Shlomi Fish, Richard Lippmann, Richard Foley, and Roman Filippov. Paul Cochrane deserves special thanks for reviewing and providing feedback on the blog posts and manuscript and for being available to discuss content, ideas, and organization matters.

I also want to thank my publishing team at Apress: Steve Anglin, Mark Powers, and Matthew Moodie, as well as everybody doing awesome work in the background, such as cover design, typesetting, and marketing.