Foreword
We started HashiCorp in 2012 to solve the challenges introduced by the rise of public cloud. The products we introduced were built in roughly the same order in which theyd be experienced by a new team building in the cloud. Vagrant was first, since creating a development environment was the first challenge we faced. Packer was second, to translate those development environments into cloud images. With the applications successfully deployed, the next challenge was networking between the multiple deployed images, and thus Consul was born.
The word microservice wasnt used then, and the problem space was admittedly much smaller: we needed a way to find the address of a healthy instance of another application or service. The reality of cloud introduced several new technical challenges: global availability, automation friendliness, and the expectation that application instances came up and went down constantly. The initial release of Consul in 2014 solved all these challenges.
The beauty of new paradigms is that first-order challengesonce solvedgive rise to second-order capabilities. The first-order challenge was service discovery in the world of public cloud. The second-order capability was then microservices, improved monitoring, more dynamic routing, and enhanced security by leveraging this new software-driven networking layer.
Consul was the natural place to enable these new capabilities, and over the years Consul has evolved to solve these difficult, modern networking challenges. From latency-aware routing at the DNS layer to automatic TLS between services to HTTP-aware load balancing and more, Consul has grown into a fully featured service mesh.
These capabilities enable teams to take full advantage of what public cloud has to offer while simultaneously getting more out of on-premises environments. Teams can deliver more applications across more regions safely, and teams that use multiple cloud platforms or on-premises datacenters can communicate across those environments in a consistent manner. And this is all possible without any modifications to the deployed applications.
Today, Consul is downloaded millions of times per year and is deployed into everything from small hobbyist home labs to the infrastructure of the worlds largest companies. It has been proven in challenging production environments time and time again.
Luke Kysow has been part of the Consul engineering team for many years, personally implementing many of its incredible features. He has a particular talent in making complex topics approachable by anyone, and he does so beautifully in this book.
Mitchell Hashimoto and Armon Dadgar
Cofounders of HashiCorp and creators of Consul
Preface
The sheer volume of software required by todays world has necessitated an evolution in how we structure our engineering organizations. Weve learned that smaller, independent teams work better than larger, highly coupled ones. Since Conways lawthat companies will produce systems to match their organizational structureis inevitable, this evolution has precipitated the rise of microservices: smaller, independent services owned by smaller, independent teams. As a result of these forces, companies are now running hundreds and even thousands of services in production.
The rise of microservices has enabled development teams to scale up and ship code faster, but it has also caused an exponential increase in complexity for operations teams. What was once an in-memory function call is now a cross-continent API request that can fail in unexpected and spectacular ways. What was once a single monitoring dashboard is now a byzantine maze of metrics, logs, and traces. A security model that was once a simple firewall now must protect against a myriad of ever-evolving attack vectors and threats. Finally, what was once a single monolithic service is now hundreds of services built using different technologies and deployed on multiple runtimes: virtual machines (VMs), Kubernetes, serverless platforms, and more.
Operations teams, also known as DevOps and site reliability engineering (SRE), thus face a monumental challenge. In the midst of this complexity, they must harden security, increase reliability, simplify observability, and speed application deliveryand they must do so in a way that works across multiple runtimes and languages. Service mesh is an exciting new technology that promises a solution to these problems.
Consul is a fully featured service mesh from HashiCorp, the company that also created Terraform, Vault, Nomad, Packer, and Vagrant. A small operations team can leverage Consul to impact security, reliability, observability, and application delivery across their entire stackall without requiring developers to modify their underlying microservices.
In this book, youll learn to install, configure, and operate Consul in order to tame complexity and take back control of your infrastructure. Im excited for you to start on your service mesh journey with Consullets dig in and get up and running!
Who Should Read This Book
If youre a platform or operations engineer tasked with maintaining a growing microservices environment on Kubernetes or VMs, then this book is for you.
If youre a microservices developer interested in increasing reliability or experimenting with advanced deployment strategies such as blue/green and canarying, this book is also for you.