Raymond Roestenburg - Akka in Action

In this chapter:

Up until not too long ago (say up to the middle of the 90s, just before the internet revolution) it was completely normal to build applications that would only ever run on a single computer, single CPU. If the application wasnt fast enough, the standard response would be to wait for a while for CPUs to get faster; no need to change any code. Problem solved. Programmers around the world were having a free lunch and life was good.

Around 2005, Herb Sutter wrote in Dr. Dobbs Journal about the need for a fundamental change (link: http://www.gotw.ca/publications/concurrency-ddj.htm). In short, a limit to increasing CPU clock speeds has been reached and the free lunch is over.

If applications need to perform faster, or if they need to support more users, they will have to be concurrent . (Well get to a strict definition later; for now lets simply define this as not single-threaded . Thats not really correct but good enough for the moment.)

Scalability is the measure to which a system can adapt to a change in demand for resources, without negatively impacting performance. Concurrency is a means to achieve scalability: the premise is that, if needed, more CPUs can be added to servers, which the application then automatically starts making use of. Its the next best thing to a free lunch.

Around the year 2005 when Herb Sutter wrote his excellent article, youd find companies running applications on clustered multi-processor servers (often no more than two to three, just in case one of them crashes). Support for concurrency in programming languages was available but limited and considered black magic by many mere mortal programmers. Herb Sutter predicted in his article that programming languages will increasingly be forced to deal well with concurrency.

Lets see what changes in about a decade since! Fast forward to today and you find applications run on large numbers of servers in the cloud, integrating many systems across many data centers. The ever-increasing demands of end users push the requirements of performance and stability of the systems that we build.

So where are those new concurrency features? Support for concurrency in most programming languages, especially on the JVM, has hardly changed. Although the implementation details of concurrency APIs have definitely improved, we still have to work with low-level constructs like threads and locks, which are notoriously difficult to work with.

Next to scaling up (increasing resources, for instance CPUs on existing servers), scaling out refers to dynamically adding more servers to a cluster. Nothing much has changed to how programming languages support networking either since the 90s. Many technologies still essentially use RPC (remote procedure calls) to communicate over the network.

In the mean time, advances in cloud computing services and multi-core CPU architecture have made computing resources ever more abundant.

PaaS (Platform As a Service) offerings have simplified provisioning and deployment of very large distributed applications, once the domain for only the largest players in the IT industry. Cloud services like AWS EC2 (Amazon Web Services Elastic Compute Cloud) and Google Compute Engine give you the ability to literally spin up thousands of servers in minutes, while tools like Docker, Puppet, Ansible, and many others make it easier to manage and package applications on virtual servers.

The number of CPU cores in devices is also ever-increasing: even mobile phones and tablets have many CPU cores today.

But that doesnt mean that we can afford to throw any number of resources at any problem. In the end, everything is about cost and efficiency. So its all about effectively scaling applications, or in other words, getting bang for your buck. Just as youd never use a sorting algorithm with exponential time complexity, it makes sense to think about the cost of scaling.

Ideally, youd expect the following two statements to be true when scaling your application:

Figure 1.1. Demand against resources