Software, as an artefact, has been tremendously successful. It has pervaded every aspect of our professional and social life, due mainly to the outstanding advances in hardware, but also to undeniable progress in software engineering practices that allow the timely production of high-quality computing products.
Software is however a victim of its own success. The software systems of today have to constantly face new and demanding requirements in terms of their availability, robustness, dynamism and pervasiveness. This is challenging, the way software systems are produced and managed. In particular, great pressure is put on the maintenance of software and systems; maintenance tasks are becoming increasingly difficult and correspondingly more time-consuming to carry out. Today, many believe that we have reached a barrier in terms of complexity and that innovative practices are needed to ensure the continuous delivery of software-based services.
In this introductory chapter, we present how software systems are currently being developed and managed. We show how the use of software has evolved and how this has impacted on the software development and maintenance processes. In particular, we show that much of the complexity involved with the software life cycle has moved from the development stage to the maintenance stage, which raises formidable challenges for practitioners.
Finally, we briefly introduce the field of autonomic computing, a relatively new spin on the ways we build and maintain software systems and whose purpose is to overcome some of these aforementioned problems we highlight. This chapter motivates the need for autonomic computing systems.
1.1 Software Complexity
Software systems can be amazingly complex. They can be difficult to conceive, to implement, to understand and to maintain. This raises significant challenges that gave birth to the software engineering approach to creating computing systems a few decades ago and has motivated the autonomic computing movement today. But what is a software system, and why is it so complex?
A software system is a collection of programmes and data deployed on one or several computers for execution. It is complex for a number of reasons. First, programmes are heterogeneous constructions. They can be made of a number of interacting computing entities, very diverse in the sense that they have their own structure, their own state at runtime and, sometimes, their own language. These computing entities are typically project specific. That is, they are created for the purpose of a single project, and this makes it difficult to reuse the experience obtained from one project to another, in terms of the development and maintenance of these entities across projects.
As observed by Frederic Brooks in his famous essay about software issues [], as the size of systems increases, the type and number of entities to be assembled increase exponentially, meaning structural complexity can be amplified.
Brooks also pointed out that software is intangible. Accurately representing the computing entities that compose a system and their behaviour is a non-trivial task. It requires defining a number of views at different levels of abstraction and many relationships between these different views. This separates the software building process from other more traditional engineering disciplines, where entities are more concrete and can be represented and understood more easily.
A software project is not limited to programmes and data though. It also comes with various software artefacts built throughout the software development process, including requirements specifications, architecture diagrams, documents, code, test suites and configurations specifications. There are tight relationships between these artefacts that are hard to express and to maintain. Most of the time, they are not entirely completed to perfection and some artefacts can be lost over the course of a project (such as design rationales). For instance, it is not unheard of that a piece of code can no longer be directly related to the requirement that motivated it.
Software artefacts are many, and the sheer number of these artefacts (and their relationships) adds to the complexity associated with modern software systems.
Brooks also stated that an essential feature of software systems is their ability to change. The source of the pressure to change comes from the clients and users, a phenomenon not normally associated with objects that are traditionally manufactured. For those objects, evolutions are carefully planned and incorporated in subsequent releases. In contrast, most software systems have to be regularly updated to stay relevant. There are of course many good reasons for that: satisfied users want more functions, bugs need to be fixed, market conditions change, incoming load has increased, hardware or software resources have evolved, etc. But the bottom line is that most people do not understand enough about the software to comprehend the extent of the difficulty and risks involved with updating existing systems.
This pressure to regularly adapt software to varying conditions has deeply impacted software engineering in the past, and this remains true today. In recent years, for instance, software development practices have been made more agile. Indeed, in many cases, a development project has to be able to start even when some business and technical aspects have yet to be nailed down. Similarly, market pressures sometimes push companies to release unfinished products, leaving bugs and missing functionalities for subsequent releases. There are of course advantages in releasing products early. An example of this is where a product is conceived for some purpose, but on release its usage changes and then its subsequent development follows that usage. Flickr is one such example, it was released as part of a multiplayer online game (from Ludicorp), but users availed of the photo storage capabilities and this popularity drove the focus to photo storage and exchange.
