Ben Smith - Building Conduit

Welcome to Building Conduit.

In this book you will discover how to implement the Command Query Responsibility Segregation and event sourcing (CQRS/ES) pattern in an Elixir application.

This book will take you through the design and build, from scratch, of an exemplary Medium.com clone. The full source code is available to view and clone from GitHub. As each feature is developed, a link to the corresponding Git commit will be provided so you can browse the source code at each stage of development.

The application will be built as if it were a real world project. Including the specification of integration and unit tests to verify the functionality under development.

By the end of this book you should have a solid grasp of how to apply the CQRS/ES pattern to your own Elixir applications.

You will learn how to:

• Follow test-driven development to build an HTTP API exposing and consuming JSON data.
• Validate input data using command validation.
• Create a functional, event sourced domain model.
• Define a read model and populate it by projecting domain events.
• Authenticate a user using a JSON Web Token (JWT).

This book is written for anyone who has an interest in CQRS/ES and Elixir.

It assumes the reader will already be familiar with the broad concepts of CQRS/ES. You will be introduced to the building blocks that comprise an application built following this pattern, and shown how to implement them in Elixir.

The reader should be comfortable reading Elixir syntax and understand the basics of its actor concurrency model, implemented as processes and message passing.

You will learn an approach to implementing the CQRS/ES pattern in a real world Elixir application. You will build a Medium.com clone, called Conduit, using the Phoenix web framework. Conduit is a real world blogging platform allowing users to publish articles, follow authors, and browse and read articles.

The inspiration for this example web application comes from the RealWorld project:

See how the exact same Medium.com clone (called Conduit) is built using any of our supported frontends and backends. Yes, you can mix and match them, because they all adhere to the same API spec.

While most todo demos provide an excellent cursory glance at a frameworks capabilities, they typically dont convey the knowledge & perspective required to actually build real applications with it.

RealWorld solves this by allowing you to choose any frontend (React, Angular 2, & more) and any backend (Node, Django, & more) and see how they power a real world, beautifully designed fullstack app called Conduit.

By building a backend in Elixir and Phoenix that adheres to the RealWorld API specs, you can choose to pair it with any of the available frontends. Some of the most popular current implementations are:

• React / Redux
• Elm
• Angular 4+
• Angular 1.5+
• React / MobX

You can view a live demo of Conduit thats powered by React and Redux with a Node.js backend, to get a feel for what well be building.

Many thanks to Eric Simons for pioneering the idea and founding the RealWorld project.

Before we start building Conduit, lets briefly cover some of the concepts related to command query responsibility segregation and event sourcing.

At its simplest, CQRS is the separation of commands from queries.

• Commands are used to mutate state in a write model.
• Queries are used to retrieve a value from a read model.

In a typical layered architecture you have a single model to service writes and reads, whereas in a CQRS application the read and write models are different. They may also be separated physically by using a different database or storage mechanism. CQRS is often combined with event sourcing where theres an event store for persisting domain events (write model) and at least one other data store for the read model.

CQRS overview

Commands are used to instruct an application to do something, they are named in the imperative:

• Register account
• Transfer funds
• Mark fraudulent activity

Commands have one, and only one, receiver: the code that fulfils the command request.

Domain events indicate something of importance has occurred within a domain model. They are named in the past tense:

• Account registered
• Funds transferred
• Fraudulent activity detected

Domain events describe your system activity over time using a rich, domain-specific language. They are an immutable source of truth for the system. Unlike commands which are restricted to a single handler, domain events may be consumed by multiple subscribers - or potentially no interested subscribers.

Often commands and events come in pairs: a successful register account command results in an account registered event. Its also possible that a command can be successfully executed and result in many or no domain events.

Domain events from the write model are used to build and update a read model. I refer to this process as projecting events into a read model projection.

The read model is optimised for querying therefore the data is often stored denormalized to support faster querying performance. You can use whatever technology is most appropriate to support the querying your application demands, and take advantage of multiple different types of storage as appropriate:

• Relational database.
• In-memory store.
• Disk-based file store.
• NoSQL database.
• Full-text search index.

Any state changes within your domain are driven by domain events. Therefore your entire applications state changes are modelled as a stream of domain events:

Bank account event stream

An aggregates state is built by applying its domain events to some initial empty state. State is further mutated by applying a created domain to the current state:

Domain events are persisted in order as a logical stream for each aggregate. The event stream is the canonical source of truth, therefore it is a perfect audit log.