You might lately have heard a lot about the term DevOps. Gartner even stated that 2016 is the year of DevOps ( http://www.gartner.com/newsroom/id/2999017 ). Is DevOps more than a marketing term? Or is it just some hype that someone created?
Why Are We Doing DevOps?
Software development is still a young industry. As an industry, its rapidly evolving and trying to become better. And growth is really necessary. The software development industry doesnt have a very good reputation when it comes to producing software on time and on budget. In whats now called a waterfall process, software was developed in a couple of discrete steps. In Figure you see the steps a typical project used to have to develop software. All these phases follow each other and the total timespan could be several months or even years.
Figure 1-1.
Stages in a waterfall project
Other industries use the same kind of distinct steps. For example, consider a city block where all houses are being built to look alike. It makes sense to have a clear specification of all the steps that go into building a house and then repeat those steps for all houses until the block is finished. Software is different. Customers dont ask developers to build them multiple copies of the same product. Instead, each project is unique. Development teams are always building something new. The problem with wanting something new is that you dont know exactly what you want since you first have to come up with an idea. Software development organizations thought the solution was to create better specifications. The documents that were created became bigger and bigger. And of course, creating a detailed upfront specification takes a lot of time. So customers were asked to sign those documents and then treat them as the absolute truth.
After the first analysis phase ends with a detailed specification, design, code, and test phases follow. After the first pass through the steps, a working version is demoed to the customer. The first time a customer sees a working version of his idea, he gets a better idea of what he actually needs. Maybe the customer wants some small changes or maybe he decides he wants something completely different. This has plagued the software industry for a long time and led to the reputation that the software industry builds the wrong thing, while missing deadlines and costing more than was promised. This is when a big step in the software industry was made: the beginning of Agile.
In 2001, the Agile Manifesto was released. The Agile Manifesto can be seen as a response to the state that software development was in. The Agile Manifesto stated a couple of simple values:
We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
The Agile Manifesto
The Agile Manifesto caused a lot of change in the software industry. Instead of writing thick documents, development teams started actually working with the customer. The fact that a customer changes his mind was no longer viewed as something that a team needed to avoid. Instead, teams accepted that customers change their minds and that its their right to do so. Software development teams started working together with their customers. So instead of working for months or even years on a product and then finally showing it to the customer, teams started to involve customers in the process by delivering small iterations and responding to changes.
The most popular method based on the Agile Manifesto is Scrum . Scrum has a couple of key elements. A team is self-organizing with all required roles in it. The team works in short iterations called Sprints . The team is led by a product owner, preferably a customer. The product owner helps the team constantly adjust to changes by constantly reprioritizing and making sure that the team delivers the most business value.
Scrum took off. It took off huge. A lot of teams liked the idea and when I ask at a developer conference who is doing Scrum, I normally see all hands going up! (This doesnt mean that everyone is doing it right. Some teams only think they are doing Scrum.)
The story doesnt end with the invention of Agile and Scrum. If a team practices Agile well, the team will pick up speed. The developers, helped by the product owner, will create a steady release of new features. Agile is part of the solution but we needed more. Successful Agile teams still have barriers.
Testers often struggle in an Agile organization. They are always trying to keep up with the developers. The discipline of testers is changing and I see the distinction between testers and developers blurring. Testers and developers often pair up or blend into a multi-skilled developer role. While testers struggle to keep up, the situation is even worse for the operations team. These teams are often in another department. They are responsible for running the datacenter and are tasked with deploying applications that the development team creates while making sure that everything keeps running. This leads to silos, which are different teams in different departments reporting to different managers with different responsibilities. The operations team is responsible for making sure the software is stable. The development team is responsible for getting out new features as fast as possible. The testers are caught in between and the business only sees that the things they need take too long to become available.
In the years since the introduction of Agile, the software world has changed even more. Mobile systems and the cloud were big game changers. Customers have changed the way they interact with companies and startups take advantage of this. A company like Uber suddenly competes with the regular taxi world. PayPal, which is a software company, competes with banks. Netflix has revolutionized the way people watch TV. How have these companies succeeded?
This is where DevOps come in. DevOps helps Agile to fully realize its potential. The opinions on what DevOps is differ, but I like the following definition :
DevOps is the union of people, process, and products to enable continuous delivery of value to our end users.
Donovan Brown, DevOps Senior Program Manager at Microsoft
The key phrase is continuous delivery of value. Where Agile helped the software industry respond to customer needs and keep up with changes, DevOps is the key to actually delivering the value to the hands of the end users. DevOps is more than tooling. DevOps is about the whole organization working together to deliver value to its customers. This makes DevOps as much about people and processes as it is about choosing the right tools. DevOps helps you in optimizing whats called the software delivery pipeline , which involves taking software from an idea to the hands of the end user. By putting DevOps patterns into practice, organizations like Netflix, Facebook, Amazon, Twitter, Google and Microsoft are achieving levels of performance that were unthinkable while using Agile. These organizations dont deploy once a year or even once a week. Instead they deploy multiple times a day while delivering a stable and reliable user experience.