Srivastava - DevOps with AWS: Implement Continuous Delivery and Integration in AWS Step by Step

Implement Continuous Delivery and Integration in AWS Step by Step
Sachin Srivastava

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Publisher or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Author and Publisher have endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, they cannot guarantee the accuracy of this information.

Sachin Srivastava is a certified DevOps and cloud expert with 15+ years of IT experience. He has expertise in technologies such as OS, source code management, build and release tools, continuous integration/deployment/delivery tools, containers, configuration management tools, monitoring, logging tools, and public and private clouds. He loves to share his technical expertise with audience worldwide at various forums, conferences, webinars, blogs, and LinkedIn.

A big thanks to God who made me capable of sharing my knowledge with the world.
Vikas Singh is working as Sr DevOps Engineer in Amazon Cloud, California. His DevOpstoolchain: Jenkins, CircleCI, ELK, AWS, Docker.

Chapter 1: The Cloud and DevOps Revolution 7
Thinking in terms of the cloud, and not infrastructure 8
Deploying your own hardware versus in the cloud 8
Cost analysis 9
Just-in-time infrastructure 9
The different layers of a cloud 12
Adopting a DevOps culture 13
The origin of DevOps 13
The developers versus operations dilemma 14
Too much code changing at once 14
Differences in the production environment 15
Communication 15
Key characteristics of a DevOps culture 16
Source control everything 16
Automating testing 16
Automating infrastructure provisioning and configuration 17
Automating deployment 17
Measuring everything 18
Deploying in AWS 20
How to take advantage of the AWS ecosystem? 20
How does AWS synergize with a DevOps culture?20
Summary 23
Questions 23
Further reading 23
Chapter 2: Deploying Your First Web Application 24
Technical requirements 24
Creating and configuring your account 25
Signing up25
Enabling MFA on the root account 27
Creating a new user in IAM 29
Installing and configuring the command-line interface (CLI) 32
Installing WSL (Windows only) 33
Installing the AWS CLI package 38
Configuring the AWS CLI 39
Creating our first web server 40
Amazon Machine Images (AMIs) 40
Instance types 41
Security groups 42

Generating your SSH keys 44
Launching an EC2 instance 46
Connecting to the EC2 instance using SSH 49
Creating a simple Hello World web application50
Installing Node.js 51
Running a Node.js Hello World application 51
Turning our simple code into a service using upstart 53
Terminating our EC2 instance 54
Summary 55
Questions 56
Further reading 56

Chapter 3: Treating Your Infrastructure as Code 57
Technical requirements 58
Managing your infrastructure with CloudFormation 59
Getting started with CloudFormation 59
AWS CloudFormation Designer 61
CloudFormer 63
Recreating our Hello World example with CloudFormation 66
Using Troposphere to create a Python script for our template 66
Creating the stack in the CloudFormation console 71
Adding our template to a source control system 72
Updating our CloudFormation stack 74
Updating our Python script 74
Updating our stack 77
Change sets 78
Deleting our CloudFormation stack79
Adding a configuration management system 81
Getting started with Ansible 82
Installing Ansible on your computer 82
Creating our Ansible playground 83
Creating our Ansible repository83
Executing modules 86
Running arbitrary commands 87
Ansible playbooks 87
Creating a playbook 87
Creating roles to deploy and start our web application 88
Creating the playbookfile 92
Executing a playbook 93
Canary-testing changes 96
Running Ansible in pull mode 97
Installing Git and Ansible on our EC2 instance 98
Configuring Ansible to run on localhost 98
Adding a cron job to our EC2 instance 99
Integrating Ansible with CloudFormation 99
Monitoring 102
Summary 103

Questions 103
Further reading 104
Chapter 4: Infrastructure as Code with Terraform 105
Technical requirements 106
What is Terraform? 106
Getting started with Terraform 107
Terraform and AWS for automated provisioning 109
Deployment using AWS Management Console 109
Deployment using AWS CLI 111
Creating our Terraform repository 112
First Terraform template for AWS instance provisioning 114
A second Terraform template for deploying a Hello World application 125
Integrating AWS, Terraform, and Ansible 129
Terraform with Ansible using a push-based approach 129
Terraform with Ansible using the pull-based approach 133
Summary 136
Questions 137
Further reading 137

Chapter 5: Adding Continuous Integration and Continuous
Deployment 138
Technical requirements 139
Building a CI pipeline 140
Creating a Jenkins server using Ansible and CloudFormation 141
Creating the Ansible playbook for Jenkins 141
Creating the CloudFormation template 146
Launching the stack and configuring Jenkins 148
Preparing our CI environment 150
Creating a new GitHub organization and repository 150
Creating a GitHub personal access token 153
Adding the access token to the credentials in Jenkins 153
Creating the Jenkins job to automatically run the builds 154
Implementing the helloworld application using our CI environment 157
Initializing the project 157
Creating a functional test using Mocha 158
Developing the remainder of the application 162
Creating the CI pipeline in Jenkins 163
Productionizing the CI pipeline 167
Building a continuous deployment pipeline 169
Creating new web servers for continuous deployment 170
Importing a custom library to Ansible for AWS CodeDeploy 170
Creating a CodeDeploy Ansible role 171
Creating the web server CloudFormation template 173
Launching our web server 174
Integrating our helloworld application with CodeDeploy174
Creating the IAM service role for CodeDeploy 175
Creating the CodeDeploy application 176
Adding the CodeDeploy configuration and scripts to our repository 177
Building our deployment pipeline with AWS CodePipeline 182
Creating a continuous deployment pipeline for staging 182
Integrating Jenkins to our CodePipeline pipeline 185
Updating the IAM profile through CloudFormation 186
Installing and using the CodePipeline Jenkins plugin 187
Adding a test stage to our pipeline 188
Building a continuous delivery pipeline for production 189
Creating the new CloudFormation stack for production 189
Creating a CodeDeploy group to deploy to production 190

Next page