Dan Sanderson - Programming Google App Engine with Python: Build and Run Scalable Python Apps on Googles Infrastructure

by Dan Sanderson

Copyright 2015 Dan Sanderson. All rights reserved.

Printed in the United States of America.

Published by OReilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472.

OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .

• Editors: Meghan Blanchette and Brian Anderson
• Acquisition Editor: Mike Loukides
• Production Editors: Colleen Lobner and Kara Ebrahim
• Copyeditor: Jasmine Kwityn
• Proofreader: Charles Roumeliotis
• Indexer: Judy McConville
• Interior Designer: David Futato
• Cover Designer: Ellie Volckhausen
• Illustrator: Rebecca Demarest

• June 2015: First Edition

See http://oreilly.com/catalog/errata.csp?isbn=9781491900253 for release details.

The OReilly logo is a registered trademark of OReilly Media, Inc. Programming Google App Engine with Python, the cover image of a four-lined snake, and related trade dress are trademarks of OReilly Media, Inc.

While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-491-90025-3

[LSI]

On the Internet, popularity is swift and fleeting. A mention of your website on a popular news site can bring 300,000 potential customers your way at once, all expecting to find out who you are and what you have to offer. But if youre a small company just starting out, your hardware and software arent likely to be able to handle that kind of traffic. Youve sensibly built your site to handle the 30,000 visits per hour youre actually expecting in your first six months. Under heavy load, such a system would be incapable of showing even your company logo to the 270,000 others that showed up to look around. And those potential customers are not likely to come back after the traffic has subsided.

The answer is not to spend time and money building a system to serve millions of visitors on the first day, when those same systems are only expected to serve mere thousands per day for the subsequent months. If you delay your launch to build big, you miss the opportunity to improve your product by using feedback from your customers. Building big early risks building something your customers dont want.

Historically, small companies havent had access to large systems of servers on day one. The best they could do was to build small and hope that meltdowns wouldnt damage their reputation as they try to grow. The lucky ones found their audience, got another round of funding, and halted feature development to rebuild their product for larger capacity. The unlucky ones, well, didnt.

These days, there are other options. Large Internet companies such as Amazon.com, Google, and Microsoft are leasing parts of their high-capacity systems by using a pay-per-use model. Your website is served from those large systems, which are plenty capable of handling sudden surges in traffic and ongoing success. And because you pay only for what you use, there is no up-front investment that goes to waste when traffic is low. As your customer base grows, the costs grow proportionally.

Googles offering, collectively known as Google Cloud Platform, consists of a suite of high-powered services and tools: virtual machines in a variety of sizes, multiple forms of reliable data storage, configurable networking, automatic scaling infrastructure, and even the big data analysis tools that power Googles products. But Google Cloud Platform does more than provide access to Googles infrastructure. It encapsulates best practices for application architecture that have been honed by Google engineers for their own products.

The centerpiece of Google Cloud Platform is Google App Engine, an application hosting service that grows automatically. App Engine runs your application so that each user who accesses it gets the same experience as every other user, whether there are dozens of simultaneous users or thousands. Your application code focuses on each individual users experience. App Engine takes care of large-scale computing taskssuch as load balancing, data replication, and fault toleranceautomatically.

The scalable model really kicks in at the point where a traditional system would outgrow its first database server. With such a system, adding load-balanced web servers and caching layers can get you pretty far, but when your application needs to write data to more than one place, you face a difficult problem. This problem is made more difficult when development up to that point has relied on features of database software that were never intended for data distributed across multiple machines. By thinking about your data in terms of Cloud Platforms model up front, you save yourself from having to rebuild the whole thing later.

Often overlooked as an advantage, App Engines execution model helps to distribute computation as well as data. App Engine excels at allocating computing resources to small tasks quickly. This was originally designed for handling web requests from users, where generating a response for the client is the top priority. Combining this execution model with Cloud Platforms task queue service, medium-to-large computational tasks can be broken into chunks that are executed in parallel. Tasks are retried until they succeed, making tasks resilient in the face of service failures. The execution model encourages designs optimized for the parallelization and robustness provided by the platform.

Running on Googles infrastructure means you never have to set up a server, replace a failed hard drive, or troubleshoot a network card. You dont have to be woken up in the middle of the night by a screaming pager because an ISP hiccup confused a service alarm. And with automatic scaling, you dont have to scramble to set up new hardware as traffic increases.

Google Cloud Platform and App Engine let you focus on your applications functionality and user experience. You can launch early, enjoy the flood of attention, retain customers, and start improving your product with the help of your users. Your app grows with the size of your audienceup to Google-sized proportionswithout having to rebuild for a new architecture. Meanwhile, your competitors are still putting out fires and configuring databases.

With this book, you will learn how to develop web applications that run on Google Cloud Platform, and how to get the most out of App Engines scalable execution model. A significant portion of the book discusses Google Cloud Datastore, a powerful data storage service that does not behave like the relational databases that have been a staple of web development for the past decade. The application model and the datastore together represent a new way of thinking about web applications that, while being almost as simple as the model weve known, requires reconsidering a few principles we often take for granted.