Overview

Ansible as remote executor in a Puppet environment

4 Comments

When you are using Puppet you might know this problem:
How can I execute arbitrary commands on all or some of my Puppet nodes?

In this article, I explain how you can do so with Ansible. Ansible it another configuration management tool like Puppet and Chef. My colleague Daniel Schneller gives a great introduction in his article “Ansible, simple yet powerful automation”. There are already a lot of comparisons of Puppet and Ansible available for example [1] and [2]. In this post I focus on the combination of Ansible and Puppet.

An often encountered problem with Puppet is that it does not provide a remote executor by itself but relies on mcollective. Ansible’s advantage over Puppet is that it does not need an agent on each server. It uses ssh to execute commands on remote hosts. In order for this to work, Ansible needs an inventory enumerating all the servers it shall manage. Taking a look at the Ansible documentation, an inventory is usually a file. The simplest form is just a list of hostnames:

web01.example.com
web02.example.com
db01.example.com
db02.example.com
...

In order to facilitate Ansible to run Puppet commands, we need to create such an inventory. As Puppet knows all the servers it manages, it can help us to create the inventory. Puppet uses certificates to secure the communication between Puppet agent and server. It stores these certificates on its master and you can access this information. For example, the output of puppet cert list --all looks like this:

+ "web01.example.com" (SHA256) 11:AB:C5:A3:7D:7B:90:40:57:FC:03:97:D7:73:07:E3:F8:D1:F5:81:3C:80:C0:4A:B2:F4:69:A7:BC:85:C8:59
+ "web02.example.com" (SHA256) 9A:84:F1:14:19:6B:BC:65:4D:29:C9:31:C1:2A:6A:BD:DC:7A:DD:28:09:42:C1:23:59:4F:9C:31:DF:DC:3B:D5
+ "db01.example.com"  (SHA256) D2:00:26:F8:3F:9B:19:CD:1F:AB:D2:74:1A:93:35:11:9C:AB:1E:C2:50:4A:72:A7:81:82:5E:83:AB:3C:50:EF
+ "db01.example.com"  (SHA256) FD:A2:F1:F1:5D:0C:1A:6B:02:3A:66:FA:51:C3:DF:88:2B:83:95:D8:BB:4C:11:CB:50:82:BF:C8:CA:26:3B:90
...

There you see a list of all managed hosts.

The next step is to convert the output of puppet cert to an Ansible inventory. The simplest way is to convert the output into a corresponding Ansible inventory You can do this with a bash command:

puppet cert list --all | egrep "^\+.*" | awk-F"\"" '{ print $2 }' > /etc/ansible/puppet_hosts

You can use the created file as inventory for Ansible:

ansible “all” -i /etc/ansible/puppet_hosts -a "uptime"

This command will produce an output like this:

web01.example.com | success | rc=0 >>
15:27:00 up 238 days, 21:20,  1 user,  load average: 0.58, 0.60, 0.63
 
web02.example.com | success | rc=0 >>
15:27:00 up 176 days, 22:07,  1 user,  load average: 0.83, 0.74, 0.73
 
db01.example.com | success | rc=0 >>
15:27:00 up 310 days,  5:51,  1 user,  load average: 0.00, 0.00, 0.00
 
db02.example.com | success | rc=0 >>
15:27:00 up 210 days, 10 min,  1 user,  load average: 1.19, 0.89, 0.77
…

Ansible calls such an inventory a static inventory. As your managed hosts will change over time you want to have something more dynamic. Therefore, Ansible also supports so called dynamic inventories. To use a dynamic inventory, just replace the inventory file from above with an executable, e.g. a script. The dynamic inventory script in our case looks like this:

#!/bin/bash
 
/usr/bin/puppet cert list --all > /dev/null 2>&1
if [ $? -ne 0 ]
then
  echo "Puppet error"
  exit 1
fi
 
HOSTS="{\n\t\"all\":\n\t\t[\n"
FIRST_HOST="true"
 
while read hostname
do
  # this way i don't get a trailing comma
  if [ "${FIRST_HOST}" == "true" ]
  then
    HOSTS="${HOSTS}\t\t\t\"${hostname}\""
    unset FIRST_HOST
  else
    HOSTS="${HOSTS}, \"${hostname}\""
  fi
done < <(/usr/bin/puppet cert list --all | egrep "^\+.*" | awk -F"\"" '{ print $2 }' | grep "example.com")
 
HOSTS="${HOSTS}\n\t\t]\n}"
 
echo -e "${HOSTS}"

This script creates one long list of hosts with the help of Puppet and prints this list in Ansible compatible JSON to standard output:

{
  "all":
    [
       "web01.example.com", "web02.example.com", "db01.example.com", "db02.example.com", ...
    ]
}

For a broader introduction to Dynamic Inventories and another example take a look at this post by my colleague Lukas Pustina. He describes how to use Ansible features like host facts and host groups with dynamic inventories.

With the dynamic inventory script, the command shown above changes to:

ansible “all” -i /usr/local/bin/puppet_dyn_hosts -a "uptime"

The Ansible command converts the current Puppet certificate list and executes the command on the selected hosts:

web01.example.com | success | rc=0 >>
15:27:00 up 238 days, 21:20,  1 user,  load average: 0.58, 0.60, 0.63
 
web02.example.com | success | rc=0 >>
15:27:00 up 176 days, 22:07,  1 user,  load average: 0.83, 0.74, 0.73
 
db01.example.com | success | rc=0 >>
15:27:00 up 310 days,  5:51,  1 user,  load average: 0.00, 0.00, 0.00
 
db02.example.com | success | rc=0 >>
15:27:00 up 210 days, 10 min,  1 user,  load average: 1.19, 0.89, 0.77
…

This command is just an example. You can replace uptime with any other command. When you have a more complex task at hand, you can even write Ansible playbooks and execute them on the hosts.

Here is an example where you first stop a service, change file permissions and start the service again:

---
- name: Updates splunkforwarder
  hosts: all
  tasks:
  - name: Stop splunk to change uid
    sudo: yes
    service: name=splunk state=stopped

  - name: Change file permissions
    sudo: yes
    file: path=/opt/splunkforwarder owner=splunk group=splunk recurse=yes

  - name: Start splunk
    sudo: yes
    service: name=splunk state=started

You can even integrate calls to Puppet agent into your Ansible playbooks, but this will be the topic of another post.

When you have more questions about how to combine Ansible and Puppet, do not hesitate to contact me.

Kommentare

  • Ches Martin

    Nice ideas. MCollective is a bit of a pain to deploy since it requires managing yet another agent process all over the place, so Ansible might be an appealing alternative for some people.

    A couple of other ideas for consideration:

    The EC2 dynamic inventory plugin that’s maintained in the official Ansible repo has support for a file-based cache with configurable expiration time built in. For anyone really using a script like your example for Puppet in earnest, it might be worth cribbing from that.

    Also, since Foreman essentially provides an API for Puppet host inventory, it’d be quite cool if an org that uses Foreman would write a dynamic inventory plugin for Foreman and share it with the community (I see you’ve written about Foreman on this blog, maybe that could be you =]). As a quick-and-dirty start it could initially depend on the hammer CLI‘s hammer host list command and parse output, then iteratively drop that requirement with direct API calls.

    Just an idea 🙂

    Also worth mentioning is that Ansible ships with a module for executing Facter on hosts, thereby incorporating Puppet’s facts into Ansible’s knowledge for automation/orchestration tasks.

  • Paolo

    simpler to use pssh, no?

    • Christian Zunker

      1. October 2014 von Christian Zunker

      In some cases it might be.
      But with pssh and other cluster shells, you are missing some features of modern configuration management/remote execution tools. For example Ansible offers a host filter for commands, in case you do not want to execute the command on every host. I could not find this feature for pssh.

      In my opinion Ansible’s advantage over tools like pssh is the orchestration part. You can write ad-hoc commands with Ansible but can also orchestrate them into playbooks. You might argue this is also possible with shell scripts, but that is the never ending story about shell scripts vs. configuration management. At the end, every team has to decide for its own which tool suits their environment and needs best.

      This is just a quick glance at the topic. There are blog post all over the internet going into much more detail.

Comment

Your email address will not be published. Required fields are marked *