Lucas - Relayd and Httpd Mastery 2017-05-22

Thanks to Reyk Flter for being the point man on relayd and httpd, and for pointing out my boneheaded goofs in writing this book.

Anthony La Porte, Stephane Guedon, and Henrik Lund Kramshj provided their illuminating production configurations. Slices of these appear throughout this book.

Thanks to Josh Grosse for his help debugging Wordpress on httpd.

I did an experiment with this book, offering an auction to benefit the OpenBSD Foundation and get your name in this book. Bill Allaire, habitual OpenBSD auction bidder, won it for $1300. Bill the Web Developer in this book is named after him. As I was writing this book, though, Bills best friend Mark Allard unexpectedly passed. The website mallard.info is in his memory. For those who think Im going soft; while I make every effort to be respectful to Mr. Allard, Bill still gets treated with the contempt appropriate for someone guilty of being Bill.

Bob Beck was kind enough to send me the script he used to make OpenSSL generate OCSP certificates. He told me that hed write a stand-alone replacement tool one day. I spent a good week in mortal agony understanding the Lovecraftian OpenSSL commands needed to work with OCSP. Right after that, Bob wrote ocspcheck(8), as he apparently thinks one day means as soon as Lucas understands this script. I need to curse Bob for putting me through those tortuous days, but you lot should fall to your knees and offer him your boundless gratitude for preserving you from my fate.

And a special thanks to Liz, for supporting my lunatic way of paying the mortgage.

The World Wide Web holds todays Internet together. Those videos of cats riding robotic vacuums require a whole complicated infrastructure behind them. Reducing that complexity helps maximize meme availability.

The members of the OpenBSD project are masters of reducing complexity. Their signature project, the OpenBSD operating system, is a Unix-like operating system thats only as complex as it must be. They provide several widely-used pieces of critical Internet infrastructure software such as the OpenSSH secure shell server and client, the LibreSSL TLS library, and freely-reusable implementations of other core network protocols. And now theyve turned their attention to web services with the web server httpd, the load balancer relayd, and network redundancy protocols such as CARP. These applications have been ported to most other operating systems.

This book takes you through the OpenBSD web stack. The stack is also available on FreeBSD and some other platforms that provide the necessary features. Well learn how to deploy web sites, balance and manipulate traffic across a server farm, and build redundant servers.

The OpenBSD web application stack has several components: the httpd web server, the relayd load balancer, and CARP.

The OpenBSD web server, httpd, is deliberately designed to provide exactly the most common web site functions. This means that it might not meet your requirementsif you need advanced server features, this server wont work for you. For a typical PHP application, static site, or download server, its perfectly adequate.

Why is a minimal web server important? While the Internet existed long before the Web, the Webs pointy-clicky interface was a key part in popularizing the Internet. The Webs core protocols have evolved since their introduction in 1991. Some things that looked like great ideas on paper turned out to be complete sewage once exposed to actual users. Other protocol components worked well in the age of 33.6 modems, but are nearly irrelevant today. The code to support those features remains in most web servers, however.

The minimal feature set extends to the configuration. The most sensible features are not optional, but mandatory. For example, the smart way to run a web site is to restrict it to a subsection of the servers filesystem (a chroot). Httpd always runs in a chroot. You cannot turn that feature off. If you want to reduce system security, you must run a less secure web server.

Even if you cant use httpd, other components of the OpenBSD web stack can help you make your site more robust.

Once you get beyond httpd into the reliability options, the stack makes heavy use of OpenBSDs packet filter, PF. PF is a general-purpose packet filter that allows you to dictate which traffic a host accepts and rejects based on its TCP/IP characteristics. It performs network address translation (NAT) and can do traffic shaping.

PF includes its own load balancing algorithm. It is completely fault-intolerant, however, so we wont spend any time on it. Youre much better off proceeding directly to relayd.

If you have a large web site, youll probably have to split the load between several web servers. Relayd is a tool for redirecting traffic between servers, letting you divide traffic between multiple hosts. While we mostly discuss web site load balancing, you can use relayd to configure load balancing for any TCP/IP protocol.

In addition to distributing load between servers, relayd can act as a web proxy, letting you intercept and filter traffic from desktop clients.

Relayd has features unrelated to the web stack, such as the ability to dynamically load balance multiple Internet links. Theyre worthy of attention, but less widely useful, so this book wont cover them.

If your site isnt large enough to merit a server farm and load balancer, but you still want redundancy against hardware failures, then the Common Address Redundancy Protocol, or CARP, is your friend. CARP allows multiple hosts to share a single IP address. If one of the hosts fails, the other can take over providing service. This means that as youre replacing one host, your web site stays up. CARP is a great way to provide high availability to a service like the Web.

Server software follows a predictable life cycle. Someone writes a handy tool to provide a service. The tool gains a following. People add features to make the tool even more useful, increasing its popularity. Eventually the software grows a thick manual worth of features, and any grace or elegance found in the original tool is long buried under layer upon layer of software barnacles.

That, of course, is when someone else writes a small tool to easily and simply do the original thing. Its the circle of digital life.

OpenBSD has a long-standing practice of standing athwart software development, yelling Stop! They stick by the original Unix principles of small tools that each do one thing well. Rather than a wish list of features to be added to relayd and httpd, the primary author maintains a list of features that will never be added to the software, as well as a much shorter list of things that could conceivably be added if the right implementation appears. This gives users a realistic ability to decide if httpd meets their needs, or if they should move on to one of the other servers.

This stripped-down approach has limits. While httpd, relayd, and CARP are excellent solutions for many sites, they dont support functions some organizations require. If you need any of these features, youre better off choosing an alternate solution.

The developers have made explicit decisions to not support certain features. Httpd will never support rewrites with Perl-style regular expressions, for example. Perl regexes have grown features for decades, and are now incredibly complicated. The code to support them would vastly increase the size of the httpd code and make it both less maintainable and less secure. Httpd includes Lua pattern support, though, which are simpler but almost as flexible. It