#! Dev's Bytes
Random programming stuff
#! Dev's Bytes
Simple orchestration with Ansible

Ansible is an IT automation tool, powerful and simple to use. It helps you with software installation, systems configuration, and app deployment.

Let's suppose that you have just finished your new, shiny web app. This is the big day and you are ready to upload the backend on your remote server, when you realize that the end of your coding sessions is not the end of your work. Now, you need to set up your remote machine, configure Supervisor, nginx, the firewall and tons of other stuff. And what if something, one day, goes terribly wrong and you have to setup all these things again?

Ansible to the rescue

Ansible is an IT automation tool that is powerful and easy to use. It can deal with all of the problems above, make the configuration of your system repeatable, and organize all of the tasks in simple and easy to read YAML files. Ansible can, and is conceived to, configure tens of machines together, differentiating the configuration according to their roles. Have I already mentioned that it is agentless? All it needs to work is SSH and Python. And you can use it even to provision Vagrant!

Installation

If Python and pip are installed, it is as simple as:

$ sudo pip install ansible

Otherwise, install it with your preferred package manager.

Let's start

First things first, to experiment with Ansible we need a remote host, so create a new "remote" host with Vagrant and add it to the known hosts of your system.

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

    # An Ubuntu Trusty Tahr box
    config.vm.box = "ubuntu/trusty32"

    # Forward port 80 of the box on port 4567 of the host
    config.vm.network :forwarded_port, host: 4567, guest: 80

end

Ansible needs to know which systems you want to configure through it. The file with the addresses of the hosts that you want to set up is called inventory. An inventory file can be as simple as an hosts list.

192.168.2.20
a.example.com
b.example.com
c.example.com

The default inventory file is located at /etc/ansible/hosts, but to experiment, you can simply pass the file with your hosts as an argument whit the -i flag.

Maybe, you have tons of remove machines and you want to divide them by tasks.

[webservers]
a.example.com
b.example.com

[databases]
db1.example.com
db2.example.com

Maybe, you need to login with a username different from the current one.

a.example.com       ansible_ssh_user=jekyll
b.example.com       ansible_ssh_user=hyde

For the purpose of this tutorial, create an inventory file called hosts with the following lines.

[vagrant]
localhost ansible_ssh_port=2222 ansible_ssh_user=vagrant ansible_ssh_pass=vagrant

We need to specify the ssh port for the vagrant box, as it doesn't use the default port 22, and the ssh password. DON'T do this in production, instead of a clear text password in your host files, use a couple of cryptographical keys for authentication on SSH.

Now, let's check to see if everything is working well.

$ ansible vagrant -i hosts -m ping

This performs a ping on all the hosts of the group vagrant.

Ok, let's move to something more funny.

Some basic commands

You can use Ansible to send some commands to your remote hosts, as with SSH. Let's suppose that you want to reboot your vagrant box.

$ ansible vagrant -i hosts -a "/sbin/reboot"

Are you saying 'sudo'? Yes, you can do it, even in the case when it requires a password.

$ ansible all -a "/usr/local/bin/foo" --sudo [--ask-sudo-password]

As you can see, I have replaced the vagrant group with the all group. The all group is a special one that targets all of the hosts in your inventory file.

You can also log as a user and sudoing as another one.

$ ansible webservers -a "/usr/local/bin/foo" -u bruce --sudo --sudo-user batman

But I promised you automation and repeatability. That brings me to playbooks.

Your first playbook

From the official documentation:

Playbooks are Ansible’s configuration, deployment, and orchestration language.

These could sound as some difficult tasks, but Ansible make them all pretty easy. A playbook can be as simple as a single .yml file filled with commands to execute on the remote host. Our first playbook installs ntp and cron-atp on our virtual machine. Save the following lines in the playbook.yml file.

# playbook.yml
---
- hosts: vagrant
  sudo: yes 
  remote_user: vagrant
  tasks:
  - name: ensure ntp is at the latest version
    apt: pkg=ntp state=latest
  - name: ensure ntp is running
    service: name=ntp state=started
  - name: ensure cron-apt is at the latest version
    apt: pkg=nginx state=latest

To run your first playbook, just type:

$ ansible-playbook -i hosts playbook.yml

Let's split it in its main parts.

Hosts and users

- hosts: vagrant
  sudo: yes 
  remote_user: vagrant

The host line is a list of one or more groups or host patterns, separed by columns as webservers:databases. The remote_user line specifies the name of the user account. This one can be also defined per task, as the sudo line. In this case, the remote_user line is redundant, indeed we have already set the remote user in the inventory.

Tasks

tasks:
- name: ensure ntp is at the latest version
  apt: pkg=ntp state=latest
- name: ensure ntp is running
  service: name=ntp state=started
- name: ensure cron-apt is at the latest version
  apt: pkg=nginx state=latest

This is where the task that must be performed on the remote hosts, specified by the hosts line, are placed. Tasks are executed in order, one at time. A basic task has a name, which is included in the output of the playbook, and a module, that, once executed, brings the system to the desired state. In the exampe, we can see the apt module, and the service module. Most modules take arguments as "key=value" couples.

A playbook is idempotent, therefore every module is executed only when it is necessary to bring the system in a known state.

Variables and templates

Actually, our playbook is not really interesting, it only installs cron-apt and ntp, and it does't even configure the latter! Let's improve it!

# playbook.yml
---
- hosts: vagrant
  sudo: yes 
  vars:
    ntp_servers:
    - 0.it.pool.ntp.org
    - 1.it.pool.ntp.org
    - 2.it.pool.ntp.org
    - 3.it.pool.ntp.org
  tasks:
  - name: ensure ntp is at the latest version
    apt: pkg=ntp state=latest
  - name: write the ntp config file
    template: src=ntp.conf.j2 dest=/etc/ntp.conf
    notify:
    - restart ntp
  - name: ensure ntp is running
    service: name=ntp state=started
  - name: ensure cron-apt is at the latest version
    apt: pkg=nginx state=latest
  handlers:
  - name: restart ntp
    service: name=ntp state=restarted

This is the ntp.conf.j2 file, copy it in the same directory of the playbook.

{# ntp.conf.j2 #}
# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help

driftfile /var/lib/ntp/ntp.drift


# Enable this if you want statistics to be logged.
#statsdir /var/log/ntpstats/

statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable

# Specify one or more NTP servers.

# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board
# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
# more information.
{% for server in ntp_servers %}
server {{ server }}
{% endfor %}

# Use Ubuntu's ntp server as a fallback.
server ntp.ubuntu.com

# Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for
# details.  The web page <http://support.ntp.org/bin/view/Support/AccessRestrictions>
# might also be helpful.
#
# Note that "restrict" applies to both servers and clients, so a configuration
# that might be intended to block requests from certain clients could also end
# up blocking replies from your own upstream servers.

# By default, exchange time with everybody, but don't allow configuration.
restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1

# Clients from this (example!) subnet have unlimited access, but only if
# cryptographically authenticated.
#restrict 192.168.123.0 mask 255.255.255.0 notrust


# If you want to provide time to your local subnet, change the next line.
# (Again, the address is an example only.)
#broadcast 192.168.123.255

# If you want to listen to time broadcasts on your local subnet, de-comment the
# next lines.  Please do this only if you trust everybody on the network!
#disable auth
#broadcastclient

Now, when the template module is called, it takes the Jinja2 ntp.conf.j2 template, substitutes the variables with the ones in the vars lines and, when it has finished, it notifies an event handler to restart ntp. Event handlers are specified after the handlers: line and are called only after all the other tasks complete and only once per play, even if multiple tasks notify the same handler.

A this stage, you should have the basic knowledge to set up a simple playbook with some basic operations, but when the tasks will grow, or when you have to build a playbook with different tasks for every role, a single file could become very messy. It's time to see a better way to organize your stuff.

A complete playbook

A complete and organized Ansible playbook has a structure that braches out across multiple directories, so I have created a git repository with all the code for this last example. Clone it with:

$ git clone git@github.com:lucachr/playbook_tutorial.git

The best way to organize your playbook is to use roles. An ansible playbook organized by roles has a structure similar to this one

production          # Production inventory 
stage               # Stage inventory
site.yml
group_vars/         
hosts_var/
roles/
    common/         # A role
        defaults/   # Role defaults lower priority variable goes here
        files/
        handlers/
        meta/       # Role dependencies goes here
        tasks/
        templates/
        vars/       # Role specific vars goes here
    webservers/     # Another role
        defaults/
        files/
        handlers/
        meta/
        tasks/
        templates/
        vars/
    databases/
        ...

Our example playbook has this structure:

site.yml
hosts
group_vars/
    vagrant
roles/
    common/
        handlers/
            main.yml
        tasks/
            main.yml
        templates/
            ntp.conf.j2
    webservers/
        files/
            tutorial/
                index.html
            nginx.conf
        handlers/
            main.yml
        tasks/
            main.yml

Group vars

---
ntp_servers:
- 0.it.pool.ntp.org
- 1.it.pool.ntp.org
- 2.it.pool.ntp.org
- 3.it.pool.ntp.org

The only variables that the playbook needs to know are the address of the ntp servers chosen.

The common role

This is the main file for the tasks.

# roles/common/tasks/main.yml
---

- name: Run apt-get update if the latest one is more than an hour ago
  apt: update_cache=yes cache_valid_time=3600

- name: Ensure ufw is at the latest version
  apt: pkg=ufw state=latest
  tags: ufw

- name: Set ufw policy to deny all incoming connections
  ufw: policy=deny direction=incoming
  tags: ufw

- name: Set ufw policy to allow all ougoing connections
  ufw: policy=allow direction=outgoing
  tags: ufw

- name: Set ufw to allow ntp
  ufw: rule=allow port=ntp
  tags: ufw

- name: Set ufw rule to limit connections on ssh/tcp
  ufw: rule=limit port=ssh proto=tcp
  tags: ufw

- name: Enable ufw logging
  ufw: logging=on
  tags: ufw

- name: Start ufw
  ufw: state=enabled
  tags: ufw

- name: Ensure cron-apt is at the latest version
  apt: pkg=cron-apt state=latest
  tags: cron-apt

- name: Ensure ntp is at the latest version
  apt: name=ntp state=latest
  tags: ntp

- name: Configure ntp file
  template: src=ntp.conf.j2 dest=/etc/ntp.conf
  tags: ntp
  notify: restart ntp

- name: Start the ntp service
  service: name=ntp state=started enabled=true
  tags: ntp

- name: Ensure tmux is at the latest version
  apt: pkg=tmux state=latest
  tags: tmux

That's a lot of things! It installs ntp, ufw, cron-apt and tmux, and configures the former two. Surely, you have noticed that tags line between tasks. Tags are a way to run only a specific part of your playbook. Suppose that you want to perform only the actions that regard ufw:

$ ansible-playbook -i hosts -t ufw site.yml

Or, you want to perform only the set up of ntp and tmux:

$ ansible-playbook -i hosts -t "ntp,tmux" site.yml

Or, maybe, you want to perform all the tasks exept the installation of cron-apt:

$ ansible-playbook -i hosts --skip-tags "cron-apt"

The "webservers" role

As with the "common" role, this is the main file for the tasks.

# roles/webservers/tasks/main.yml
---

- name: Add apt signing key for nginx
  apt_key: url=http://nginx.org/keys/nginx_signing.key state=present
  tags: nginx

- name: Add nginx repository into sources list
  apt_repository: repo='deb http://nginx.org/packages/ubuntu/ 
      {{ ansible_distribution_release }} nginx' state=present
  tags: nginx

- name: Add nginx source repository into sources list
  apt_repository: repo='deb-src http://nginx.org/packages/ubuntu/ 
      {{ ansible_distribution_release }} nginx' state=present
  tags: nginx

- name: Ensure nginx is at the latest version
  apt: name=nginx state=latest
  tags: nginx

- name: Configure nginx.conf file
  copy: src=nginx.conf dest=/etc/nginx/nginx.conf
  tags: nginx
  notify: restart nginx

- name: Start the nginx service
  service: name=nginx state=started enabled=true
  tags: nginx

- name: Set ufw to allow http
  ufw: rule=allow port=http
  tags: ufw

- name: Create server files path
  file: path="/var/www/tutorial" state=directory mode=0755
  tags: tutorial

- name: Set /var/www permission
  file: path="/var/www" mode=0755
  tags: tutorial

- name: Copy the site's files
  copy: src=tutorial/ dest=/var/www/tutorial directory_mode
  tags: tutorial

So, you have seen that "{{ ansible_distribution_release }}" variable and now you are asking yourself "Where does that thing come from? Aren't the ntp servers address the only variables around?". Well, actually Ansible can "discover" some information on your system and use them as variables in your playbook. The information discovered in this way are called facts, and these facts can save you a good amount of time while you are setting up your playbooks and come impressively in handy with your systems configuration. For further information on facts, have a look to the official documentation.

Last step

Start your playbook with

$ ansible-playbook -i hosts site.yml

See the generous amount of output in your shell, describing the tasks that gets done, then go to http://localhost:4567 and enjoy your hard work!

Conclusions

Today, you have:

  • Learnt what is Ansible and how it can help you with your everyday work.
  • Learnt how to manage a remote host with it.
  • Written up your first playbook.
  • Set up and configured, in near no-time, a production enviroment with nginx, ufw, cron-apt, tmux and static pages, making everything documented and repeatable!

You can be proud of yourself! Now, it's time to put online that nice app ;).

Further reading

Did you like this article? Do you have any suggestions or just want to give your opinion? Please, leave a comment and let me know what you think about it!


Receive Updates

ATOM

Contacts