This is a Leanpub book. Leanpub empowers authors and publishers with the Lean Publishing process. Lean Publishing is the act of publishing an in-progress ebook using lightweight tools and many iterations to get reader feedback, pivot until you have the right book and build traction once you do.
2020 - 2021 Continuous Delivery Ltd.
This book is dedicated to my wife Kate, who helped to create it in both small and big ways.
Preface
This book is intended as a practical description and guide to the idea of the Continuous Delivery Deployment Pipeline: specifically how to create Deployment Pipelines, what to consider while creating them, and how to use them to their best advantage to support software development.
While writing my book Continuous Delivery I coined the term Deployment Pipeline to describe the automated practice that supports CD. The CD Deployment Pipeline encompasses all of the steps necessary to create something releasable, and then, finally, to deploy that change into production. The automation of these steps makes this process repeatable and reliable and dramatically more efficient than more conventional approaches.
This book is a guide to help you to get started on your first Deployment Pipeline, or to help you to improve and further develop your existing Pipelines. This book is a close companion to my training course Anatomy of a Deployment Pipeline in which we explore the information in this book in some depth.
The first section, Deployment Pipeline Foundations, describes the principles of Continuous Delivery and Deployment Pipelines in general and how to think about them to best effect. It describes several ideas that underpin the Continuous Delivery approach, and ideas for the basis for the rest of this book. It moves on to describe some practical steps that I believe represent the best approach to getting your Deployment Pipeline started, what to focus on and in what order to create things.
Section two, Deployment Pipeline Anatomy, describes each of the principal, essential, stages of a Deployment Pipeline in some detail, and offers guidance and advice on the construction of effective implementations of each stage. In this section we also cover several common, though optional, stages that may be worth considering, depending on your business and the nature of your software: looking at ideas like Testing Performance, Security and other Non-Functional requirements.
The last section, Whole Pipeline Considerations examines some broader, cross-cutting ideas that inform the use and creation of effective Pipeline: looking at ideas like Infrastructure as Code and Continuous Compliance.
This is not intended as a replacement for my other book Continuous Delivery, which is a much more authoritative exploration of Continuous Delivery in general, and of Deployment Pipelines specifically. Rather, this is a condensed, practical guide to getting an effective Deployment Pipeline up and running.
Our intent in writing this book is for it to act as a useful practical guide to making changes, it is intended to be short, and to the point.
This book was written in collaboration with my wife, Kate Farley. Kate and I worked on this together, which helped to keep both of us a bit more sane, during COVID19 lockdown.
Deployment Pipeline Foundations
Chapter 1 - Introduction to Continuous Delivery
Continuous Delivery emerged in the early 2000s, building on the techniques of Continuous Integration, and was popularised by mine and Jezs award-winning 2010 book Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation.
The term Continuous Delivery is taken from the first principle of the Agile Manifesto (2001), which states:
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software
What is Continuous Delivery?
Continuous Delivery is the best way of developing software that we currently know of. It is the state of the art for software development, enabling its practitioners to deliver Better Software Faster. Teams that practise Continuous Delivery develop higher quality software, more efficiently, and have more fun while doing so. Businesses that employ Continuous Delivery have better staff retention, are more successful, and make more money.
We have research to back up these claims. Data from The State of DevOps Reports quantify the profound impact of Continuous Delivery on the effectiveness and efficiency of software development. For example:
44% more time spent on new features
50% higher market cap growth over 3 years
8000x faster deployment lead time
50% less time spent fixing security defects
50% lower change-failure rate
21% less time spent on unplanned work and rework
Its little wonder then, that some of the biggest, most successful software businesses on the planet practise Continuous Delivery. Companies like Amazon, Google, Netflix, Tesla, Paypal, and many more.
Continuous Delivery is achieved by working so that our software is always in a releasable state. But why is this fundamental idea so valuable to the development of quality software?
This way of working is the antithesis of traditional software development, where whole products, or systems, are developed over weeks and months, often with different elements being written by different teams, and brought together for a major release exercise: only then are defects and integration issues discovered, involving lots of rework, stress, delays, the abandonment of features, and disappointed customers.
Instead, through Continuous Delivery, we work in small steps, testing each tiny piece of new code as we proceed: incrementally building up more comprehenisve changes from these many small steps. Our aim is to commit each of these small steps as we make them, committing changes multiple times per day: all the time maintaining our software in a releasable state with each tiny change. This fast, dynamic, creative process means that we can make progress every day, avoid the problems of major releases, and have software that is always in a releasable state.
By working in small steps and making tiny changes frequently, we reduce the risk of complex interactions, compound errors and lengthy delays caused by rework.
There is only one definition of done - not partial or proxy measures suggested by other disciplines. The change is complete when it is delivered into the hands of its users.
Figure 1.1 - Idea to Working Software
Continuous Delivery is an holistic approach
Continuous Delivery encompasses all aspects of software development and is an holistic approach, which I define as:
going from idea to valuable, working software in the hands of users.
Continuous Delivery focuses on optimising the whole of that span and is therefore not only a technical discipline: it also requires optimal organisational structure and performance; and, an appropriate culture and ways of working. It includes the ideas of DevOps, collaboration and teamwork - where teams are empowered to make decisions and share responsibility for their code.