Chapter 1. Continuous Delivery: Why and What
In this chapter, you will explore the core concepts of continuous delivery and learn more about the benefits for developers, QA, operations, and business teams. An important question to ask before embarking on any change in the way you work is Why? Here you will learn how enabling rapid feedback reduces context switching; how automatic, repeatable, and reliable releases reduce much of the stress and challenges with delivering software to customers; and how codifying the definition of done allows rapid verification and facilitates any auditing required. Finally, you will examine what a typical Java continuous delivery build pipeline looks like and learn the fundamentals of each stage in the pipeline.
Setting the Scene
Continuous delivery (CD) is fundamentally a set of practices and disciplines in which software delivery teams produce valuable and robust software in short cycles. Care is taken to ensure that functionality is added in small increments and that the software can be reliably released at any time. This maximizes the opportunity for rapid feedback and learning, both from a business and technical perspective. In 2010, Jez Humble and Dave Farley published their seminal book Continuous Delivery (Addison-Wesley), which collated their experiences of deploying software delivery projects around the world, and this publication is still the go-to reference for CD. The book contains a valuable collection of techniques, methodologies, and advice from the perspective of both technology and organizations.
Much has changed in the world of software development and delivery over the past 20 years. Business requirements and expectations have changed dramatically, with a focus on innovation, speed, and time to market. Architects and developers have reacted accordingly, and new architectures have been designed to support these requirements. New deployment fabrics and platforms have been created and have co-evolved alongside new methodologies like DevOps, Release Engineering, and Site Reliability Engineering (SRE). Alongside these changes, a series of best practices for creating a continuous delivery build pipeline has co-evolved. The core concept is that any candidate change to the software being delivered is built, integrated, tested, and validated before determining that it is ready for deployment to a production environment.
In this book, you will focus on accomplishing the task of creating an effective build pipeline for modern Java-based applications, whether you are creating a monolith, microservices, or serverless style function as a service (FaaS) application.
Enabling Developers: The Why
An important questions to ask before undertaking any major task within software development and your approach to this is Why? Why, as a Java developer, should you invest your valuable time in embracing continuous delivery and creating a build pipeline?
Rapid Feedback Reduces Context Switching
Feedback is vital when working with complex systems, and nearly all software applications are complex adaptive systems. This is especially true of modern component-based software systems that are deployed to the web, which are essentially distributed systems. A quick review of the IT press publications over the past 20 years reveals that software development issues are often discovered only when large (and costly) failures occur. Continual, rapid, and high-quality feedback provides early opportunities to detect and correct errors. This allows the detection and remediation of problems while they are smaller, cheaper, and easier to fix.
Rapid Feedback Can Provide a Business Competitive Advantage
In their book Accelerate (IT Revolution Press), Nicole Forsgren, Jez Humble, and Gene Kim argue that organizations in all industries are moving away from using big projects with long lead times to small teams that work in short cycles and measure feedback from users in order to build products that delight customers and rapidly deliver value. In addition to using metrics for technical concerns, you can also facilitate feedback for the organization by working closely with the business team to identify key performance indicators and metrics that can be implemented within the application.
From a developers point of view, one of the clear advantages of rapid feedback is the reduced cost in context switching and attempting to remember what you were doing with a piece of code that contains a bug. We dont need to remind you that it is much easier to fix an issue that you were working on five minutes ago, rather than one you were working on five months ago.
Automatic, Repeatable, and Reliable Releases
The build pipeline must provide rapid feedback for the development team in order to be useful within their daily work cycles, and the operation of the pipeline must be highly repeatable and reliable. Accordingly, automation is used extensively, with the goal of 100% automation or as close as you can realistically get to this. The following items should be automated:
Software compilation and code-quality static analysis
Functional testing, including unit, component, integration, and end-to-end
Provisioning of all environments, including the integration of logging, monitoring, and alerting hooks
Deployment of software artifacts to all environments, including production
Data store migrations
System testing, including nonfunctional requirements like fault tolerance, performance, and security
Tracking and auditing of change history
Shifting Left Thinking and Testing
You will often hear people talk about shifting left processes like security verification or practices like acceptance testing with continuous delivery. The core idea behind the shift left is that you move things that are typically done in later stages earlier, which results in a high-quality implementation or less cost to fix issues that would be discovered only far into the delivery process. In the examples mentioned here, this would mean consulting the InfoSec team or performing some threat modeling when starting to design a new feature, or encouraging developers to implement automated acceptance testing that runs as part of the pipeline test suite. Continuous delivery can be a catalyst for this shift, as not only does the pipeline provide a visualization of all the verification stages involved from code to deployment, but it also provides a framework to implement automated verification and validation.
With the automation of the release process complete (and repeatable and reliable), you, as a developer or operator, have confidence in continually releasing new functionality without causing breakages or regressions. Nothing destroys morale as quickly as having to rely on an unreliable and flaky deployment process. This leads to fear in deploying code, which, in turn, encourages teams to batch large amounts of functionality in a big bang release, which ultimately leads to even more problematic releases. This negative feedback loop must be broken, and the adoption of the continuous delivery of functionality in small batch sizes (ideally, with single-piece flow) is a great approach to help encourage this.
Codifying the Definition of Done
The fast feedback and automation of the release process is useful for developers in and of itself. However, another clear advantage of creating a build pipeline is that you can codify the definition of done. When a software component successfully traverses a build pipeline, this should unequivocally indicate that it is ready to go into production, provide the value planned, and function within acceptable operational parameters that include availability, security, and cost. Historically, it has been difficult for teams to ensure a consistent definition of done, and this can be a friction point between development and business teams within an organization.