DevOps - Intro to Infrastructure As Code

# DevOps
## Intro to Infrastructure as Code

Thomas Foulds

SGS Ideation Lab July 7, 2017

---

## Agenda

1. Who am I?
2. Recap: On our last episode...
3. Today's Lesson
4. Auto-Magical
5. Q&A

---

## Who am I?

* 2.5 years @ Sapient
* DevOps for HRSA
* CLI maniac
* Automation enthusiast

---

## Recap: On our last episode...

* What is DevOps?
* Why implement it?
* Who benefits?
* How do we do it?

===

## What is DevOps?

![What is DevOps?](Silicon_Valley_Inquisitive.jpg)

$Blatent slide reuse?$

Note:
* A set of tools and practices born out of Agile that tries to address the
  disconnect between devs and ops in the SD lifecycle
* Much more about the culture than the tools

===

## Why implement it?

Generate business value faster for the client

===

## Who benefits?

Every stakeholder in the SD lifecycle

===

## How do we do it?

That's what I'm here to show you today

Note:
* Act as a primer on the kind of thinking to use when approaching devops
  problems

---

## Today's Lesson

Virtualize and automate your development and deployment environment

===

## Implementing Devops

* *Version Control*
* *Automation*
* *Infrastructure as Code*
* Release Management
* Monitoring and Logging

Note:
* We're going to cover the first 3 today.
* Also going to be including some virtualization tools to speed development of
  our scripts

===

## But Why Those?

* VC is how we track our work
* VMs == faster test cycle
* Infrastructure as code provides reliable environments
* Build Automation removes rote tasks

Note:
* We start with VC because we need somewhere to put all the stuff that we're
  going to be creating for everything else.
* Next, VMs make it so that a new blank slate to test our script against takes
  seconds not hours to build.
* Infrastructure scripting is what we're really after here, because it lets us
  create repeatable deployments
* Automation puts it all together by removing even the need to run the deploy
  manually

---

## Version Control

![XKCD 1597: Git](xkcd-git.png)

Note:
* I'm going to be using Git today, but use whatever makes the most sense for
  your team.
* I had the most familiarity with Git, it's easy to set up a repo any number of
  places, and there are lots of good integrations.
* Just please use it!

===

## What to Commit?

* Roles
* Configuration Scripts
* Build Scripts

Note:
* Really everything you can should end up in version control because then it is
  at least backed up somewhere.
* Recreating a build job becomes just importing the configuration, not trying
  to remember it for a GUI.

===

## Repository Layouts

* Depends on project and potential for role reuse
* All-in-one or configuration separated from roles

===

## Repository - All-In-One

Note:
* Good for quick projects that may not see reuse or whose roles are extremely
  customized
* Everything in one place so it is versioned together

===

## Repository - Split

Note:
* Good if you're building a library of reusable roles for use across projects
* Have to keep from over specializing your roles
* Additional housekeeping to tell Ansible where to find its configuration

===

## Tricks and Gotchas

* Committing credentials/passwords to the repository
* Ansible can't find it's role library or it's out of date
* Single SSH key of failure

Note:
* Over 300k results for GitHub commits saying "removed password"
* Use Ansible Vault or similary encryption for secrets
* Don't want to develop configuration that depends on functionality missing from
  your roles.
* Use all-in-one or train your devs
* Have credentials and the repo backed up somewhere. A lost laptop or broken
  drive shouldn't take down your ability to deploy.

---

## Virtualization 101

Vagrant & (VirtualBox | AWS)

===

## Picking VM Tools

* Client requirements/deploy platform
* What you have available

Note:
* VB is easy to set up and available everywhere
* Everyone is using AWS and gives you a lot of options
* Vagrant makes handling VMs much much less painful to do

===

## VirtualBox

* Install from [virtualbox.org](https://www.virtualbox.org)
* Virtualization extensions enabled in BIOS

Note:
* Can choke up your system because I've found it allocates VM memory in a single
  block so no over provisioning.

===

## Vagrant

Available at [vagrantup.com](https://www.vagrantup.com)

```bash
$ vagrant init debian/stretch64
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
```

Note:
* Tool created to help automate creating and testing VMs.
* Available on all platforms.

===

## Vagrant - Vagrantfile

```ruby
Vagrant.configure("2") do |config|
  # Base box definition
  config.vm.box = "debian/stretch64"
  # Provider specific configuration
  config.vm.provider "virtualbox" do |vb|
    vb.cpus = 1
    vb.memory = 1024
  end
  # VM Definition
  config.vm.define :mimir do |mimir|
    mimir.vm.hostname = 'mimir.aetherith.net'
    mimir.vm.synced_folder ".","/vagrant", disabled: true
    mimir.vm.network :forwarded_port, guest: 80, host: 8080
    mimir.vm.network :forwarded_port, guest: 443, host: 8443
  end
end
```

Note:
* Vagrantfile is just a bit of Ruby code and can be extended as such.
* This is how you define your VMs and their configuration.

===

## Vagrant - The Base Box

```ruby
# Base box definition
config.vm.box = "debian/stretch64"
# Provider specific configuration
config.vm.provider "virtualbox" do |vb|
  vb.cpus = 1
  vb.memory = 1024
end
```

Note:
* To simplify setup Vagrant uses 'base boxes' to work from.
* Many different ones provided by the community and various people on Atlas.
* You can also create your own base boxes if one doesn't already exist.

===

## Vagrant - Defining a VM

```ruby
# VM Definition
config.vm.define :mimir do |mimir|
  mimir.vm.hostname = 'mimir.aetherith.net'
  mimir.vm.synced_folder ".","/vagrant", disabled: true
  mimir.vm.network :forwarded_port, guest: 80, host: 8080
  mimir.vm.network :forwarded_port, guest: 443, host: 8443
end
```

Note:
* Can leave this part out and use config.vm instead of a specific host block.
* This form lets you define more than on VM in a single Vagrantfile.
* This is where you set up synced folders or forwarded ports.

===

## Vagrant - Launching a VM

Time for our first demo!

Note:
* Bring up the Vagrantfile we just went through in a demo folder. Run
  `vagrant up` and then show people how you can SSH around in the box once it
  comes up.
* Destroy the VM after.

===

## Vagrant - Adding More Providers

Add a new virtual machine provider: [vagrant-aws](https://github.com/mitchellh/vagrant-aws).

```bash
$ vagrant plugin install vagrant-aws
```

Specify which one you want at runtime.

```bash
$ vagrant up --provider=aws
```

Note:
* VirtualBox comes built in but dozens of others are available.
* Supports AWS, Azure, KVM, etc.

===

## AWS Hosting - Secrets

```yaml
# .aws_credentials
---
access_key_id: 1234567890
secret_access_key: deadbeef
```

Vagrantfile
```ruby
require 'yaml'

aws_creds = YAML::load(File.read("#{File.dirname(__FILE__)}/.aws_credentials"))

Vagrant.configure("2") do |config|
```

Be sure to add .aws\_credentials to your .gitignore!

Note:
* We don't want to hard code our AWS secrets into our configuration so we put
  them in a file.
* Remember the Vagrantfile is just a bit of Ruby, you can pull in builtins
  really easily.

===

## AWS Hosting - Provider

```ruby
config.vm.provider :aws do |aws, override|
  aws.access_key_id = aws_creds["access_key_id"]
  aws.secret_access_key = aws_creds["secret_access_key"]

override.vm.box = "dummy"
  aws.ami = "ami-27072e31"

aws.instance_type = "t2.micro"

aws.security_groups = ['webserver']
  aws.keypair_name = "aws-web"

override.ssh.username = "ec2-user"
  override.ssh.private_key_path = "~/.ssh/aws-web.pem"
end
```

Note:
* We configure the AWS provider just like we configure VB.
* We have to use the AWS dummy box because we can't actually upload the base
  box image to AWS.
* I've specified the Debian Stretch box in NoVA for this demo.
* You will need to create a security group that allows at least port 22 through
  from the outside world or provisioners won't run.
* Vagrant also assumes that the user it should log in as is 'vagrant', but for
  EC2 images you need the 'ec2-user'.

===

## AWS Hosting - Launching a VM

Now how easy is this to do actually?

Time for another demo!

Note:
* Have AWS console open in another tab and show that there aren't any active
  nodes.
* Open up the AWS enabled Vagrantfile that we just went through and show people
  that it's the same.
* Run `vagrant up --provider=aws` and wait for the machine to come up.
* Use this time to field questions people might have so far b/c AWS takes a hot
  second to come up.
* SSH into the new machine with Vagrant to show people that it works.
* Destroy VM and show that it brings the machine down cleanly.

---

## Building Prod In a Day

Vagrant + Ansible = Environments

===

## Picking Your Configuration Manager

* Firewall restrictions
* Language familiarity
* Experience with similar tools

Note:
* Ansible is agentless and runs over SSH which means a minimum of additional
  software to install on the remote host.
* Written in Python which I prefer over Ruby (Puppet, Chef)
* More declarative style instead of a state description language (Salt, Puppet)
* Downside is UNIX required and limited Windows support

===

## Get Vagrant Playing

```ruby
  mimir vm.network :forwarded_port, guest: 443, host: 8443
end

config.vm.provision "ansible" do |ansible|
  ansible.config_file = "ansible.cfg"
  ansible.verbose = "v"
  ansible.playbook = "site.yml"
end
```

```bash
ansible-playbook site.yml -u vagrant -v
```

Note:
* Setting the verbosity a bit higher helps us debug when things go wrong in our
  playbooks.
* Vagrant supports many different provisioners including simple shell scripts.

===

## Ansible - Basics

```yaml
# site.yml
---
- hosts: all
  become: yes
  tasks:
    - name: Install Git
      package:
        name: git
        state: present
```

Let's look at this in action...

Note:
* This is as simple playbook which executes the steps provided in order.
* The `become` directive instructs Ansible to execute the commands in question
  as an administrator (root) user.
* Tasks can be given descriptive names to help in tracing what is done each run.
* Many different modules created for Ansible which allow you to control any
  number of different aspects of your system.
* For example `package` is actually a meta module which will call the correct
  package manager depending on the system it is run on. This allows for more
  portable playbooks and roles.

===

## Ansible - Basics cont.

Effective

Limited reuse

### Solution: Roles

===

## Roles - DIY VS Community

* [Ansible Galaxy](https://galaxy.ansible.com/) has a great collection of
  prebuilt roles
* Do you have time to write it yourself?
* Do you trust the other guy to do it right?

Note:
* I like writing my own because I know exactly how the role works and exactly
  what assumptions I have made.
* Writing your own also allows you to configure baseline security practices
  which you want to share across your organization in an opt-out fashion.

===

## Roles - Structure

Note:
* In each subdirectory, only main.yml is loaded by default but other files can
  be loaded by a module in a task.
* Tasks - the things you want to do
* Handlers - tasks to run when things change like restarting a server. Run at
  the end of a playbook or when explicitly flushed with a meta task.
* Templates - things like conf files that need variable replacement done.
* Files - files that need to be copied exactly to the remote host
* Vars - lists of variables that tasks can load. Usually used to handle
  differences in package name or install location between different versions.
* Defaults - list of variables which should be used in the role if no override
  is provided anywhere else.
* Meta - for things like authorship information when pushing to Galaxy and for
  listing dependencies on other roles.

===

## Roles - Tasks

```yaml
- name: Install nginx
  package:
    name: '{{ nginx_package_name }}'
    state: present
- name: Restart nginx
  service:
    name: nginx
    state: restarted
```

Note:
* When you apply a role, only the main.yml is included by default.
* You can include other lists of tasks from within main.yml.
* This is helpful to break up your tasks into logical pieces and can help you
  create reusable parts if a certain series of tasks need to be done over and
  over.

===

## Roles - Defaults

```yaml
---
nginx_owner: nginx
nginx_group: nginx
nginx_http_port: 80
nginx_conf_dir: /etc/nginx
nginx_sites_enabled_dir: '{{ nginx_conf_dir }}/sites-enabled'
nginx_site_conf_path: '{{ nginx_sites_enabled_dir }}/site.conf'
```

Note:
* Prefix variables with the role name to avoid name collisions
* Lowest level of presidence. Any redefintion of the variable overrides this one
* Can use previously defined variables to define new ones. Great for software
  config directories

===

## Roles - Templates

site.conf.j2
```jinja
server {
  listen *:{{ nginx_http_port }};

server_name {{ nginx_server_name }};
  root {{ nginx_root_dir }};

server_tokens off;

index index.html index.htm;

location / {
    try_files $uri $uri.html $uri/ =404;
  }
}
```

Note:
* Templates are compiled using the Jinja2 engine.
* Undefined variables, like `nginx_server_name` and `nginx_root_dir` will cause
  the compilation to fail when it runs.
* This is a very simple template and you can include many more conditionals and
  other helpful bits with Jinja.

===

## Roles - Templates cont.

```yaml
- name: Template out the site configuration file.
  template:
    src: site.conf.j2
    dest: '{{ nginx_site_conf_path }}'
    owner: '{{ nginx_owner }}'
    group: '{{ nginx_group }}'
    mode: 0644
```

Note:
* Writing a template out to the remote machine is just as easy as calling a
  module.
* Linux template module also lets you set the file permissions and owner when
  you create the file.
* By default it is created as whatever user Ansible has connected with.

===

## Roles - Customizing

```yaml
- hosts: all
  roles:
    - nginx
```

```yaml
- hosts: all
  roles:
    - role: nginx
      nginx_http_port: 81
```

Let's see this in action...

Note:
* Good for overriding ports and paths if you need to install multiple copies of
  the same server on the same machine.
* Have to make sure your role is written with this in mind, it doesn't come by
  default.

===

## What about testing?

* [ansible-lint](https://github.com/willthames/ansible-lint) for syntax
* [Test Kitchen](https://github.com/test-kitchen/test-kitchen) for state

Note:
* Linter is good for basic syntax but can be a bit picky and opinionated. You
  will need to annotate accordingly to get it to stop complaining about things.
* Kitchen is actually from the Chef project originally and is written/uses Ruby.
  You will need to be fairly familiar to write the spec files it's looking for.

---

## Auto-Magical

How to automate redeploying your infrastructure on command

===

## Jenkins

* Open source automation server
* Tons of plugins
* Easily distributed

Note:
* Go with what is available and has the integrations you need. All Atlassian?
  Try to stay in their stack for best interoperability.

===

## Jenkins - Jenkinsfile

Script and version infrastructure like an application

```groovy
node {
  stage('Preparation') {
    echo 'Download our roles and host configuration'
  }
  stage('Test') {
    echo 'Test our configuration for syntax and correctness'
  }
  stage('Deploy') {
    echo 'Deploy our infrastructure only if it passes our tests'
  }
}
```

Note:
* Part of the new Pipeline feature in Jenkins.
* Written in Groovy.
* You create this versionable file and that defines your build, minimal GUI
  needed.
* Not all plugins are supported yet and some creative thinking may be required
  to get things done.

===

## Jenkins - SCM

```groovy
stage('Preparation') {
  dir('config') {
    git 'https://github.com/aetherith/devops-demo-config.git'
  }
  dir('roles') {
    git 'https://github.com/aetherith/devops-demo-roles.git'
  }
}
```

Note:
* Pull down as many repos as you need into whatever folder structure you require

===

## Jenkins - Testing

```groovy
stage('Test') {
  dir('config') {
    sh 'find . -type d -maxdepth 1 -mindepth 0 -not -path \'./.git\' | xargs ansible-lint'
  }
  dir('roles') {
    sh 'find . -type d -maxdepth 1 -mindepth 1 -not -path \'./.git\' | xargs ansible-lint'
  }
}
```

Note:
* Run your linters that don't have plugins through the commandline. Any nonzero
  return code reads as an error and stops the build.
* This is a bit of a hack to get `ansible-lint` to play nice with the way my
  repo is set up.

===

## Jenkins - Deploy

```groovy
stage('Deploy') {
  dir('config') {
    withCredentials([string(credentialsId: 'Ansible Vault Password', variable: 'ANSIBLE_VAULT_PASSWORD')]) {
      writeFile file: '.vault_password', text: '''#!/bin/sh
      echo $ANSIBLE_VAULT_PASSWORD'''
      sh 'chmod u+x .vault_password'
      ansiblePlaybook credentialsId: 'fc9e437f-31c4-42d9-ba86-d7efa2d59324', installation: 'System Ansible', inventory: 'hosts', playbook: 'site.yml', sudoUser: null
    }
  }
}
```

Note:
* Ansible allows you to provide the Vault password through an executable script.
  This lets us store our Vault password securely in Jenkins and only have it
  exposed during the parts we need it.
* We also have to create credentials in Jenkins for Ansible to use to log in.
  This again lets us keep our credentials somewhere secure while still keeping
  track of what they are.
* The Pipeline snippet generator is your friend.

===

## Jenkins - Automatic Infrastructure

Last demo of the day...

Note:
* Show people the site.yml which uses the nginx role we just made.
* Show them the site on port 80 with the hello message.
* Show them the Jenkins job which uses the Jenkinsfile we just went through.
* Change the port and hello message in site.yml and commit the change.
* Go back to Jenkins and show how it picks up the change and deploys it out.
* Got to demo.aetherith.net:81 and show off the changed page.

===

## Gotchas

* SSH and Vault credentials for Jenkins
* Remote user will need `sudo`
* Vault `ansible_become_password` to provide password for remote `sudo`

---

## Questions?

---

## Thanks For Coming!