Trifork Blog

Ansible from the Trenches

August 13th, 2013 by

Although its name might suggest something different on Wikipedia, Ansible is an open source framework that is capable of automating various system tasks like setting up a server, deploying and configuring applications and command execution (See Ansible can run across many virtual machines without the need to augment it with other tools. For distribution on various machines Ansible by default relies on SSH as a transport layer. So aside from an SSH server, Ansible doesn’t require any additional daemons to be running on the remote servers. Also Ansible doesn’t leave any traces on remote systems after its done running. Ansible  depends on the python interpreter that comes with the majority of Linux distributions. And it uses a YAML format to describe configuration and execute commands, which makes it readable.

So why not use Ansible to deploy an application across several virtual machines?
For our project we needed to deploy and configure the following packages on Ubuntu 12.04:

  • JDK 7
  • Tomcat 7
  • Elasticsearch
  • Riak
  • Application war’s

Our approach

As a non maintainer Ansible is a very lightweight framework that let’s you get started quickly with deployment and configuration. For our project we decided to use playbooks. Playbooks are the basis for configuration management and multi-machine deployment. In a playbook you can declare configurations, and orchestrate the set of steps a.k.a. tasks that need to be performed against all hosts. We started writing a playbook per package. So for our project we had to write 5 playbooks. Each playbook contains its own set of variables and tasks. Tasks are defined in a separate tasks module. Dependent configuration files like empty.policy for Tomcat are contained in a separate files folder. This keeps the playbook simple and maintainable.
For elasticsearch we’ve used an open source project on github as basis. This worked out pretty fine.

And then…

We managed to start fast with the playbooks, but it is the simplicity of the framework that is misleading; thinking you’ve done the steps necessary for deployment but in fact you haven’t.

Lack of feedback…

One of the first things we ran into is the fact that Ansible doesn’t warn if a host’s key changes. You’ll have to make sure that all machines are populated with the proper known-hosts file.

Another major drawback we experienced is the fact that Ansible doesn’t give much feedback when executing steps. So it is not very clear if steps have been performed properly or not. That you’ll have to check yourself. And if something goes wrong you barely have a clue what went wrong and where you should start looking making it very cumbersome.

We also had to download several packages like the Oracle tar.gz, a 90 Mb file, which takes quite some time to complete, even on a high speed network. Unfortunately Ansible doesn’t provide any feedback on the progress of the download.

It should come as no surprise then that we were not worried when a sub task in the Tomcat playbook could sometimes take quite some time to finish. But instead the sub task never completed because it was waiting for user input. This was not visible when running the playbook. It seems to be a conscious decision not to do this (see ansible issue 472)

Figure it out yourself….

Compared to frameworks like Puppet, with Ansible you have to write everything yourself. There is limited support from the framework for OS variables or settings. On one hand lots of freedom, but you do need to have a good and complete idea of what steps need to be performed and how. It also depends on the packages you install. Installing OpenJDK 7 with Ansible is much simpler than installing the Oracle JDK on Ubuntu. why? Because installing OpendJDK on Ubuntu already sets many environment variables as with Oracle you have to set all the variables with the Ansible script. So with the Oracle JDK you’ll have to set many variables manually. To get around this we’ve used the Oracle JDK installer from the WebUpd8 team.

In the end Ansible took us way more time than expected. Although it looks simple, the learning curve is quite high. It can also take quite some time to test and run a playbook, so don’t underestimate the amount of time necessary. We certainly won’t next time, also thanks to my co-writer and colleague in this project Thomas Zeeman. For those of you interested, we’ve also written some other blogs about Ansible in the past.

2 Responses

  1. August 13, 2013 at 14:43 by Michael DeHaan

    Hmm, there’s a lot of confusing things here, I’m not sure you stopped by to ask any questions but they are easily solveable.

    For starters, run your playbooks with “-v” and you’ll see a lot more feedback!

    I’m not sure what you mean about OS variables and settings. Post a question on the mailing list and we’d be happy to help.

    • August 13, 2013 at 16:31 by Monique Dewanchand

      Hi Michael,

      We’ve tried several options but in some cases the feedback could be improved like in the case of issue 472. That would really help.

      Regarding OS variables; when downloading and installing the Oracle JDK7 from the Oracle website, using something like:
      wget –no-cookies –header “Cookie:”$1 -O $1
      we discovered that several variables like the JAVA_HOME, the java and jar links and more are not set. This we had to do manually from the Ansible script. So our script was growing large.

      But using the installer of the WebUpd8team made it much easier. Just somehing like:
      – name: Add WebUpd8 repository
      apt_repository: repo=’ppa:webupd8team/java’

      – name: Accept the license for Oracle JDK
      action: raw sudo echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | sudo /usr/bin/debconf-set-selections

      – name: Install Oracle JDK 7 package
      action: $ansible_pkg_mgr pkg=’$item’ state=installed
      – oracle-java7-installer
      – oracle-java7-set-default

      The OpenJDK task, was even simpeler:
      – name: Install OpenJDK 7 package
      action: $ansible_pkg_mgr pkg=’openjdk-7-jdk’ state=installed

      So if you have any options regarding the Oracle JDK7 public download, let us know.