Steve Smith - Measuring Continuous Delivery

I believe that Continuous Delivery is important to the advancement of software development. Organisations that practice Continuous Delivery dont see five or ten percent improvements in quality and productivity, but usually closer to orders of magnitude improvements.

These are somewhat grandiose assertions, but I do not make them lightly. I believe that software development has seen in recent years a step-change in our ability to create more complex software faster and more reliably than ever before. This is largely due to the practices of Continuous Delivery.

For those of us that have been practising these techniques for a long time, there has been a noticeable change in industry assumptions over the past few years. To a significant degree sequential, waterfall-style processes have been rejected. Even large, complex enterprises that used to strongly defend the need for high-ceremony development processes are attempting to move away from them. This is for a good reason - those approaches dont work very well!

The big web-shops, who were amongst the early pioneers for these more effective approaches to software development, made the change out of necessity. The old approaches, both in terms of technology and process simply did not work well enough at the scale and pace that these new companies needed to move.

Most of us, that I suppose are now seen as pioneers in this field, evolved these techniques empirically. We tried lots of things, often found things that didnt work, but over time refined techniques, processes and approaches that have stood the test of time and are now at the heart of many of the worlds most successful software companies.

I believe that there are good reasons for all of this.

First, the old sequential, planned approaches to software development often fail because they are not based on effective feedback mechanisms. They actively work against the need to reflect, learn, refine, and adapt as we grow our understanding of a problem and our solutions to it.

Second, the new approach to software development is in every sense a change in paradigm. It is challenging because it is so different to what has gone before in most organisations. Organisations will struggle with the transition to these more effective practices as long as they try to hold on to the old ways of thinking.

Third, I believe that Continuous Delivery works because it is based on the application of the Scientific Method to software development. The Scientific Method is humanitys strongest problem solving technique, by far! Science is based on a process of practical, rational, and empirical discovery. We work in ways that build on the work that has come before us, and allow us to learn from our mistakes. The magical thing about an empirical approach to problem solving is that we learn the most at the moment of failure. We organise our process so that these failures are safe, contained, and ideally low-cost. We learn, we adapt, and we move on.

All of this probably sounds a bit philosophical for the foreword about a book on software delivery. However, I believe Continuous Delivery as an engineering discipline is generally applicable to all kinds of software. It is engineering in the sense of the application of scientific methods and knowledge to practical problem solving. We create hypotheses about outcomes, we carry out small changes, we test our hypotheses with low-cost experiments as a falsification mechanism (we refer to many of these as automated tests), we reject changes that do not meet expectations, and we change the direction of our thinking if need be.

This is the Scientific Method in practice, and what does science rely on? Data, facts, and measurements. If we want to be rational in our decision making, if we want to make the highest quality decisions that we can, we need to work in ways that give us insights. This allows us to design new experiments that prove the functional correctness and performance of our software, understand the effectiveness of our teams, and help us learn what our users make of our changes.