Nikhil Anurag - Distributed Computing with Go

Copyright 2018 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Commissioning Editor: Dominic Shakeshaft

Acquisition Editor: Frank Pohlmann

Project Editor: Radhika Atitkar

Content Development Editor: Monika Sangwan

Technical Editor: Nidhisha Shetty

Copy Editor: Tom Jacob

Proofreader: Safis Editing

Indexer: Rekha Nair

Graphics: Tom Scaria

Production Coordinator: Nilesh Mohite

First published: February 2018

Production reference: 1270218

Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.

ISBN 978-1-78712-538-4

www.packtpub.com

V.N. Nikhil Anurag is a Go developer currently working in Berlin. He speaks at conferences about how to use Go in domains such as concurrency, file systems, and distributed systems. He is also trying to bridge the gap between the rich literature on concurrency and the practice of programming goroutines and channels. He did his Bachelor's in Electronics and Instrumentation Engineering from JNTU, India and Master of Science in Control System from University of Sheffield, UK.

Pankaj Khairnar is a cofounder and CTO at Qwentic (A Golang specialized development company). He loves programming, and for the last 10 years, he has been developing highly scalable and distributed enterprise applications using various technologies.

Jinzhu Zhang is a veteran coder, creator/contributor of many open source projects, such as GORM. He is on Github at github.com/jinzhu .

The Go programming language was developed at Google to solve the problems they faced while developing software for their infrastructure. They needed a language that was statically typed without slowing down the developer, would compile and execute instantaneously, take advantage of multicore processors, and make working across distributed systems, effortless.

The mission of Distributed computing with Go is to make reasoning about concurrency and parallelism, effortless and provide the reader with the confidence to design and implement such programs in Go. We will start by digging into the core concepts behind goroutines and channels, the two fundamental concepts in Go around which the language is built. Next, we will design and build a distributed search engine using Go and Go's standard library.

This book is for developers who are familiar with the Golang syntax and have a good idea of how basic Go development works. It would be advantageous if you have been through a web application product cycle, although it's not necessary.

, Developer Environment for Go, covers a list of topics and concepts required to start working with Go and rest of the book. Some of these topics include Docker and testing in Go.

, Understanding Goroutines, introduces the topic of concurrency and parallelism and then dives deep into the implementation details of goroutines, Go's runtime scheduler, and many more.

, Channels and Messages, begins by explaining the complexity of controlling parallelism before introducing strategies to control parallelism, using different types of channels.

, The RESTful Web, provides all the context and knowledge required to start designing and building REST APIs in Go. We will also discuss the interaction with a REST API server using different available approaches.

, Introducing Goophr, opens the discussion on what is meant by a distributed search engine, using OpenAPI specification to describe REST APIs and describing the responsibilities of the components of a search engine, using OpenAPI. Finally, we'll describe the project structure.

, Goophr Concierge, dives deep into the first component of Goophr by describing in detail how the component is supposed to work. These concepts are further driven home with the help of architectural and logical flow diagrams. Finally, we'll look at how to implement and test the component.

, Goophr Librarian, is a detailed look at the component that is responsible for maintaining the index for the search terms. We also look at how to search for given terms and how to order our search results and many more. Finally, we'll look at how to implement and test the component.

, Deploying Goophr, brings together everything we have implemented in the previous three chapters and start the application on the local system. We will then test our design by adding a few documents and searching against them via the REST API.

, Foundations of Web Scale Architecture, is an introduction to the vast and complex topic on how to design and scale a system to meet with the demands at web scale. We will start with a single instance of a monolith running on a single server and scale it up to span across multiple region, have redundancy safeguards to ensure that the service is never down and many more.

• The material in the book is designed to enable a hands-on approach. Throughout the book, a conscious effort has been made to provide all the relevant information to the reader beforehand so that, if the reader chooses, they can try to solve the problem on their own and then refer to the solution provided in the book.
• The code in the book does not have any Go dependencies beyond the standard library. This is done in order to ensure that the code examples provided in the book never change, and this also allows us to explore the standard library.
• The source code in the book should be placed at $GOPATH/src/distributed-go. The source code for examples given will be located inside the $GOPATH/src/distributed-go/chapterX folder, where X stands for the chapter number.
• Download and install Go from https://golang.org/ and Docker from https://www.docker.com/community-edition website