Thomas Hunter Ii - Distributed Systems with Node.js

by Thomas Hunter II

Copyright 2021 Thomas Hunter II. All rights reserved.

Printed in the United States of America.

Published by OReilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472.

OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .

• Acquisitions Editor: Jennifer Pollock
• Development Editor: Corbin Collins
• Production Editor: Daniel Elfanbaum
• Copyeditor: Piper Editorial LLC
• Proofreader: Piper Editorial LLC
• Indexer: nSight Inc.
• Interior Designer: David Futato
• Cover Designer: Karen Montgomery
• Illustrator: Kate Dullea

• November 2020: First Edition

See https://www.oreilly.com/catalog/errata.csp?isbn=9781492077299 for release details.

The OReilly logo is a registered trademark of OReilly Media, Inc. Distributed Systems with Node.js, the cover image, and related trade dress are trademarks of OReilly Media, Inc.

The views expressed in this work are those of the author, and do not represent the publishers views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-492-07729-9

[LSI]

In the past decade, Node.js has gone from novelty to the de facto platform for new applications. During that period, I have had the opportunity to help thousands of Node.js developers from around the world orient themselves and find their paths to success. I have seen Node.js used for everything. Really: someone even built a low-level bootable operating system with Node.js.

At the SFNode meetup I created in San Francisco, we have a star speaker who has spoken more than anyone else. You guessed it: Thomas Hunter II, the author of this book. While you may be able to do anything with Node.js, there are some really practical things that particularly benefit from being done with Node.js. In todays cloud-first world, most systems have become distributed systems. In this book and in the countless talks Ive had the pleasure to see Thomas give at SFNode and around the world, pragmatism reigns supreme. This book is filled with experience-tested, hands-on guidance to get you from where you are today to where you need to be tomorrow.

The JavaScript language enables us as developers to create at the speed of thought. It requires little ceremony, and the code we write is usually simple enough that writing it by hand is more efficient than generating it. This beautiful simplicity of JavaScript is perfectly matched with Node.js. Node, as we frequently refer to it, is intentionally minimal. Ryan Dahl, its creator, wrote Node to build an application server that was an order of magnitude easier and faster than what anyone was used to. The results have exceeded even our wildest dreams. The ease and simplicity of Node.js enables you to create, validate, and innovate in ways that simply werent possible 10 years ago.

Before I had Node.js, I was a full stack developer using JavaScript to build interactive web-based experiences and Java to provide APIs and backend services. I would revel in the creative flow of JavaScript, and then have to completely shift gears to translate all of it into an object model for Java. What a waste of time! When I found Node.js, I could finally iterate efficiently and effectively both on the client and the server. I literally dropped everything, sold my house, and moved to San Francisco to work with Node.js.

I built data aggregation systems, social media platforms, and video chatall with Node.js. Then I helped Netflix, PayPal, Walmart, and even NASA learn how to use the platform effectively. The JavaScript APIs were rarely folks biggest challenge. What confused people most was the asynchronous programming model. If you dont understand the tools you are using, how can you expect to achieve the best results with those tools? Asynchronous programming requires you to think a bit more like a computer system rather than a linear script of consecutive actions. This asynchrony is the heartbeat of a good distributed system.

When Thomas asked me to review the table of contents of this book to make sure hed covered everything, I noticed that the section on scaling starts with an overview of the cluster module. I immediately flagged it as an area of concern. Cluster was created to enable single instance concurrency that can be exposed to a single port on a system. Ive seen folks new to Node.js take this and run with the assumption that since concurrency may be desirable, cluster is the right tool for their needs. In distributed systems, concurrency at the instance level is usually a waste of time. Luck had it that Thomas and I were on the same page, and this led to a delightful talk at SFNode by our top presenter.

So, as you are building your aptitude as a Node.js developer and as a distributed systems developer, take time to understand the constraints and opportunities in your system. Node.js has incredibly performant I/O capabilities. Ive seen downstream systems become overwhelmed when old services were removed and replaced with Node.js implementations. These systems acted as natural rate limiters that the downstream services had been built to accommodate. Adding a simple Node.js proxy can fix most issues until the downstream services are updated or replaced.

The ease of development with Node will enable you to try many things. Dont be afraid to throw out code and start over. Node.js development thrives in iteration. Distributed systems let us isolate and encapsulate logic at a service level, which we then can load balance across to validate whole system performance. But dont just take my word for it. The pages in this book show you how to do this most effectively.

Have fun and share what you learn along the way.

Dan Shaw (@dshaw)
Founder and CTO, NodeSource
The Node.js Company
Always bet on Node.js