Tomasz Nurkiewicz - Reactive Programming with RxJava: Creating Asynchronous, Event-Based Applications

• 2016-10-04: First Release

On October 28, 2005, the then newly appointed chief architect of Microsoft, Ray Ozzie, emailed a now infamous memo to his staff with the subject The Internet Services Disruption. In this memo, Ray Ozzie outlines basically how the world looks today where enterprises like Microsoft, Google, Facebook, Amazon, and Netflix use the Web as the main delivery channel for their services.

From a developer perspective, Ozzie made a rather remarkable statement for an executive of a large corporation:

Complexity kills. It sucks the life out of developers, it makes products difficult to plan, build and test, it introduces security challenges, and it causes end-user and administrator frustration.

First of all, we have to take into account that in 2005, the big IT enterprises were deeply in love with mind-blowingly complicated technologies like SOAP, WS-*, and XML. This was a time where the word microservice was not yet invented, and there was no simple technology on the horizon to help developers manage the complexity of asynchronously composing complex services from smaller ones, and dealing with concerns such as failure, latency, security, and efficiency.

For my Cloud Programmability Team at Microsoft, Ozzies memo was a rude wakeup call to focus on inventing a simple programming model for building large scale asynchronous and data-intensive Internet service architectures. After many false starts it finally dawned on us that by dualizing the Iterable/Iterator interface for synchronous collections, we could obtain a pair of interfaces to represent asynchronous event streams, with all the familiar sequence operators such as map, filter, scan, zip, groupBy, etc. for transforming and combining asynchronous data streams, and thus Rx was born somewhere in the summer of 2007. During the implementation process we realized that we needed to manage concurrency and time, and for that we extended the idea of Javas executors with virtual time and cooperative re-scheduling.

After an intense two year hackathon where we explored numerous design choices, we first shipped Rx.NET on November 18, 2009. Soon thereafter we ported Rx to Microsoft.Phone.Reactive for Windows Phone 7 and started to implement Rx in various other languages such as JavaScript, and C++, and dabbled with experimental versions in Ruby and Objective-C.

The first Rx user inside Microsoft was Jafar Husain, and he brought the technology with him when he joined Netflix in 2011. Jafar evangelized Rx within the company, and eventually re-architected the Netflix UIs client-side stack to fully embrace asynchronous stream processing. Also, most fortunately for all of us, he managed to pass on his enthusiasm to Ben Christensen who was working on Netflixs middle tier API and since Netflix uses Java on the middle tier, Ben started to work on RxJava in 2012 and moved the codebase to Github in early 2013 for continued open source development. Another early adopter of Rx at Microsoft was Paul Betts and when he moved to Github, he managed to convince his colleagues at Github such as Justin Spahr-Summers to implement and release ReactiveCocoa for Objective-C in the spring of 2012.

As Rx became more popular in the industry, we convinced Microsoft Open Tech to open-source Rx .NET in the fall of 2012. Soon thereafter, I left Microsoft to start Applied Duality and focus 100% of my time on making Rx the standard cross-language and cross-platform API for asynchronous real-time data stream processing.

Fast forward to 2016 and the popularity and use of Rx has skyrocketed. All traffic through the Netflix API relies upon RxJava, as does the Hystrix fault-tolerance library that bulkheads all internal service traffic, and via related reactive libraries RxNetty and Mantis, Netflix is now creating a completely reactive network stack for connecting all internal services across machine and process boundaries. RxJava is also extremely successful in the Android space with companies like SoundCloud, Square, NYT, Seatgeek all using RxJava for their Android apps and contributing to the RxAndroid extension library. noSQL vendors such as Couchbase and Splunk also offer Rx-based bindings to their data access layer. Other Java libraries that have adopted RxJava amongst others include Camel Rx, Square Retrofit, and Vert.x. In the JavaScript community, RxJS is widely used and powers popular frameworks such as Angular 2. The community maintains a website where you can find information about Rx implementations in many languages, as well as fantastic Marble Diagram artwork and explanations by David Gross (@CallHimMoorlock).