Cohen Marc - Google compute engine: [managing secure and scalable cloud computing]

by Marc Cohen , Kathryn Hurley , and Paul Newson

Copyright 2015 Marc Cohen, Kathryn Hurley, and Paul Newson. All rights reserved.

Printed in the United States of America.

Published by OReilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472.

OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles ( http://safaribooksonline.com ). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .

• Editor: Rachel Roumeliotis
• Production Editor: Shiny Kalapurakkel
• Copyeditor: Jasmine Kwityn
• Proofreader: Amanda Kersey
• Indexer: Angela Howard
• Interior Designer: David Futato
• Cover Designer: Karen Montgomery
• Illustrator: Rebecca Demarest

• December 2014: First Edition

See http://oreilly.com/catalog/errata.csp?isbn=9781449360887 for release details.

The OReilly logo is a registered trademark of OReilly Media, Inc. Google Compute Engine, the cover image, andrelated trade dress are trademarks of OReilly Media, Inc.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed astrademarks. Where those designations appear in this book, and OReilly Media, Inc. was aware of a trademarkclaim, the designations have been printed in caps or initial caps.

While the publisher and the author(s) have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author(s) disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-449-36088-7

[LSI]

Todays ).

Figure P-1. Google Data Center (Photograph by Google/Connie Zhou)

In the mainframe era, if you outgrew the capacity of a single computer, you needed to come up with millions of dollars to buy another computer. Modern data centers achieve scalable capacity by allocating tasks across large numbers of commodity servers. In the data center era, you allocate as many inexpensive servers as you need and then relinquish those resources when youre done.

Until recently, data center resources were accessible by the few engineers fortunate enough to work for a new generation of technology companies. However, over the past few years, a revolution has taken place. Just as earlier revolutions in computer hardware made it feasible for more people to access larger numbers of smaller computers, cloud computing enables even greater access, via the public Internet, to vast clusters of computers in modern state-of-the-art data centers. And just as it did in the past, this expanded accessibility is stimulating tremendous innovation.

In its short history, Google has pioneered many of the techniques and best practices used to build and manage cloud computing services. From Search to Gmail to YouTube to Maps, Google services provide secure, scalable, reliable cloud computing to millions of users and serve billions of queries every day. Now, with Google Compute Engine, the infrastructure that supports those services is available to everyone.

Compute Engine offers many advantages: leading-edge hardware, upgraded regularly and automatically; virtually unlimited capacity to grow or shrink a business on demand; a flexible charging model; an army of experts maintaining computing and networking resources; and the ability to host your resources in a global network engineered for security and performance.

This book provides a guided tour of Google Compute Engine, with a focus on solving practical problems. At a high level, Google Compute Engine is about giving you access to the worlds most advanced network of data centersthe computing resources that power Google itself. Practically speaking, this means providing APIs, command-line tools and web user interfaces to use Googles computing and networking resources.

In succeeding chapters, well explain the detailed product capabilities, along with some best practices for getting the most out of Google Compute Engine. Well provide numerous examples, illustrating how to access Compute Engine services using all of the supported access methods. Although the programming examples in this book are all written in a combination of Python and JavaScript, the underlying RESTful API and the concepts presented are language independent.

Contents of This Book

API offers a way to interact with Compute Engine components and resources programmatically.

Figure P-2. Overview of Google Compute Engines components

Well explore the following Google Compute Engine components and resources:

• Projects, access modes, and the API ()

• Virtual machines ()

• Persistent disk ()

• Google Cloud Storage ()

• Cloud SQL and Cloud Datastore ()

• Firewalls, load balancing, and the private network ()

• A complete)

What Each Chapter Covers

, Getting StartedWell take a look at how to get started using Compute Engine. We begin by creating a Compute Engine project using the Google Developers Console, a web UI. We then explore various means of accessing and managing Compute Engine resources via the Developers Console; gcloud compute, a command-line interface; and the Compute Engine API, a RESTful interface., InstancesInstances covers the basics of working with Compute Engine instances and explains instance attributes in detail., Storage: Cloud SQL and Cloud Datastore

Most applications require a location for storing their data. The storage available to Compute Engine comes in many flavors, including persistent disks, Google Cloud Storage, Cloud SQL, and Cloud Datastore.

As the name implies, a persistent disk stores data beyond the life of any associated instance(s). Cloud Storage allows you to store, access, and manage objects of any size on Googles infrastructure. Cloud Storage offers an excellent option for highly durable, high availability data storage. Your data is accessible both inside and outside the scope of Compute Engine via a variety of mechanisms and tools, including the Developers Console, the gsutil command, and the Cloud Storage API.

Cloud SQL provides a MySQL service in the cloud, managed by Google, while Cloud Datastore provides a Google scale NoSQL data service. Both of these services are available inside and outside the scope of Compute Engine.

explain all of these storage options in depth, and provide detailed examples illustrating how to exercise all supported access methods.

, NetworkingEvery provides a short primer on TCP/IP networking and covers Compute Engines advanced networking features in depth., Advanced TopicsIn , well cover a variety of advanced topics that provide additional ways to customize your Compute Engine resources. Topics include custom images, startup scripts, and the metadata server., A Complete Application