AWS System Administration
by Mike Ryan and Federico Lucifredi
Copyright 2018 Mike Ryan, Federico Lucifredi. All rights reserved.
Printed in the United States of America.
Published by OReilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472.
OReilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com/safari). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .
- Acquisitions Editor: Rachel Roumeliotis
- Editor: Andy Oram
- Production Editor: Melanie Yarbrough
- Copyeditor: Kim Cofer
- Proofreader: Jasmine Kwityn
- Indexer: WordCo Indexing Services, Inc.
- Interior Designer: David Futato
- Cover Designer: Karen Montgomery
- Illustrator: Rebecca Demarest
- August 2018: First Edition
Revision History for the First Edition
- 2018-08-06: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781449342579 for release details.
The OReilly logo is a registered trademark of OReilly Media, Inc. AWS System Administration, the cover image, and related trade dress are trademarks of OReilly Media, Inc.
The views expressed in this work are those of the authors, and do not represent the publishers views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.
978-1-449-34257-9
[LSI]
Foreword
When Amazon created its first cloud, well before anyone had coined the term DevOps, few if any could conceive of how expansive it would become. From its humble roots in Amazon Elastic Compute Cloud (EC2) in 2002 and Amazon Simple Storage Service (S3) in 2006, AWS has expanded to a collection of over 100 services in over 15 regions around the world. This growing platform of capabilities can be intimidating at first glance and many wonder where to even begin. While there have been scores of blog posts and wikis devoted to passing on the tribal wisdom learned through sometimes painful experiences, it has been difficult to find an entry point that meets the needs of the budding cloud practitioner.
But who exactly is a cloud practitioner? Were entering a period where a working knowledge of cloud configuration and administration fundamentals are considered a requirement for a wide series of industries. Were currently in an environment that goes well beyond a concept of self-service IT. We now assume that users will no longer be just consumers of information technology resources, but cocreators. A research scientist developing code to analyze genomic data now not only helps create the data schema but also configures the compute and storage resources necessary to support their analysis. An industrial engineer executing high-performance fluid dynamics calculations has to know how many virtual CPUs and memory to request to best optimize the performance of her application.
We are living in a time of changing infrastructure architectures, serverless applications, and containerization. It takes a complete change in mindset to move from asking how to best configure a rack with an exact number of servers, disks, and network switches to pondering what to do with a potentially unlimited number of configurations of CPUs/GPUs/FPGAs, memory, object, block, and file storage connected through high-speed network connections. The choices can sometimes be overwhelming. The broad expanses of vast computing resources enable us to do things few thought even possible less than a decade ago, and yet the lack of physical constraints can almost induce vertigo at times like a pilot without a horizon, enveloped in an actual physical cloud, not a virtual computing one.
Boundaries help us to make sense of our surroundings; looking over the edge of the cliff lets us know where to stop; staying between the lines on the road helps us to drive safely. We learn how to best operate within our constraints even though we may occasionally try to push our limitations. We optimize our behavior based upon our limitations. But what do we do when those limitations, which have helped define us for so long, are now removed? How do we begin to think about operating in an era of cloud computing?
Mike and Federico have written a book that is approachable, yet not basic; plain spoken, but not simple; technical, but not overly complex. It addresses the needs of anyone looking to learn how to engage with AWS for the first time. It removes some of the intimidation from the alphabet soup of acronyms and new terminology that will be thrown at the new AWS user: EC2, S3, EBS, AMI, instance, IAM, ELB, RDS, Route 53, CloudWatch, CloudFormation, Glacier, and more. It is a welcome starting point for all who wish to learn more about the essentials of getting started using AWS and beginning their journey into the cloud.
Ian Colle, General Manager for AWS Batch and HPC at Amazon Web Services, has a background in economics and philosophy as well as software engineering.
Preface
System administration is a complicated topic that requires practitionersto be familiar with an ever-expanding range of applications and services.In some ways, Amazon Web Services (AWS) is just another tool to add to your toolkit, yetit can also be considered a discipline in and of itself. Successfullybuilding and deploying infrastructure on AWS involves a thorough understanding of the underlying operating system concerns, software architecture, and delivery practices, as well as the myriad components that make up Amazon Web Services.
Mike runs a DevOps consultancy, helping startups and small businesses reap the benefits of tools and processes that were previously available only to organizations with large teams of system administrators. Many of these businesses do not have a dedicated system administrator, and the development team is responsible for deploying and maintaining the architecture.
In working with these clients, Mike noticed patterns in how people were working with AWS. Those who came from a pure development background, without any sysadmin experience, would often build an infrastructure that left out many of the things sysadmins would take for granted, such as monitoring and logging. The lack of monitoring and logging would then makeit difficult to track down issues, leading to more downtime than necessary.
At the other end of the spectrum were clients with a lot of sysadminexperience, but less or no development experience. This group was morelikely to treat AWS as nothing more than a virtual machine hostingprovider, simply using EC2 to run a fleet of static instances withouttaking advantage of any high-availability features such as Auto ScalingGroups and Elastic Load Balancing. This is akin to buying a Ferrari andthen using it only to buy groceries once per week: fun, but not very cost-effective.
Using AWS requires a fundamentally different mindset than when deployinggroups of static servers. You do not simply set up some servers and thenperiodically perform maintenance. Instead, you use the full AWS toolset(automatic instance replacement, scaling up and down in response to demand, etc.) to build a system. In this sense, it is more like programming than traditional system administration.