Automated virtual test-environments with Vagrant and Puppet

Almost every developer knows the problem:
You work in a team, maybe even a distributed one, on a complex application which encompasses several infrastructure components. In most cases you already have the need for a databse and a application server. When it’s getting more complex, stuff like a messaging middleware or a NoSQL store may get added. Most times developers will install all components on their local machine and if you’re lucky, installation and configuration is at least aided by some scripts. But often enough, this “art” is passed on from mouth to mouth or you have to work through some poorly maintained wiki pages. Even in the best cases, the downside is that your test-environment differs from test, staging and production environments in that on your local machine all services are on one host and effects that will happen in a distributed environment are more likely not to occur. But things can be put right with Vagrant, a tool to create and distribute virtualized environments and Puppet, a tool which helps you with your configuration management. This article will illustrate their use with an example of how to construct a simple Java web application stack.

Update (June 2012): The code on Github is updated to work with Vagrant versions 1.x now. Puppet manifests are fixed (no more Sun JDK, update of package lists).

Installation and first steps

The following examples have been tested on Ubuntu 11.10 but should be easily adoptable for other environments. Due to the fact that Vagrant is built upon Ruby and VirtualBox, these should be installed onto your system. Then the installation with RubyGems is the most simple way:

> gem install vagrant

Note: If you encounter errors during the installation, invalid timestamps in the gemspec file are a good bet. After the successful installation, we can start right through:

> mkdir vagrant-test && cd vagrant-test
> vagrant init lucid64

With ‘vagrant init’, we create a default Vagrantfile in the working directory, which is the basis for each Vagrant environment. The last two parameters are the name and URL of the base box on which this environment should build. Each Vagrant project builds upon such a base box and from there on other steps like custom configuration or software provisioning are done. The Vagrant guys offer Ubuntu 10.04 (Lucid Lynx) directly from their site. Other boxes can be found on the community site, or you may want to build your own base box.

The above command now creates the file Vagrantfile, which looks like this: do |config|
  # All Vagrant configuration is done here. The most common configuration
  # options are documented and commented below. For a complete reference,
  # please see the online documentation at
  # Every Vagrant virtual environment requires a box to build off of. = "lucid64"
  # The url from where the '' box will be fetched if it
  # doesn't already exist on the user's system.
  config.vm.box_url = ""
  # Boot with a GUI so you can see the screen. (Default is headless)
  # config.vm.boot_mode = :gui
  # ...

Now our first automated virtual environment is just one command away:

> vagrant up

Console output of 'vagrant up'

Console output of 'vagrant up'

The warnings concerning the VirtualBox GuestAdditions can be ignored in most cases. However, if you get in troubles with it, you should update and repackage your base box.

Now Vagrant offers you several commands to ssh into your box, halt it or destroy it completely if it’s no longer needed or you want to restart with the initial configured state.

 > vagrant ssh
> vagrant halt
> vagrant destroy

This initial configured state is what interests us now. Within a project, their should be one defined state for the test environment, even the one locally used by the developers. This helps getting rid of the classical “I got it to run this way …” situations and everybody has the option to generate this state from the versioned configuration. This is where Vagrant helps you again, as it offers several provisioning mechanisms to be configured to manage installation and configuration of software and the system itself. Currently you can use either plain shell scripts, Chef in its solo or server flavour, or Puppet. Pavlos article Provisioning of Java web applications using Chef, VirtualBox and Vagrant describes how to work with Vagrant and Chef, this article will look into Puppet.

Mutli-VM Environments

On big advantage of Vagrant is its capability to not only define one box in one Vagrantfile, but a whole environment of boxes, if you like. In the course of this article we will build a simple, tiny although typical stack for a Java web application:

  • one database server, running MySQL
  • one application server, running tomcat

Most configuration details are found in this article, but if you like to have the whole setup right away, you can find it on GitHub.

After we destroyed our previously built default box, we start with editing our Vagrantfile like this: do |config|
  # base box and URL where to get it if not present = "lucid64"
  config.vm.box_url = ""
  # config for the appserver box
  config.vm.define "appserver" do |app|
    app.vm.boot_mode = :gui ""
    app.vm.host_name = "appserver01.local"
    app.vm.provision :puppet do |puppet|
      puppet.manifests_path = "manifests"
      puppet.manifest_file = "appserver.pp"
  # config for the dbserver box
  config.vm.define "dbserver" do |db|
    db.vm.boot_mode = :gui ""
    db.vm.host_name = "dbserver01.local"
    db.vm.provision :puppet do |puppet|
      puppet.manifests_path = "manifests"
      puppet.manifest_file = "dbserver.pp"

We define two separate boxes for our severs and assign them aliases and static IPs. We also enable that the backing virtual boxes are started in GUI mode, because due to a VirtualBox problem, sometimes Vagrant hangs on startup. If that happens, and you see that the virtual box has already booted, login in with vagrant/vagrant, run ‘sudo dhclient’ and Vagrant will carry on.

Configuration Management with Puppet

We create Puppet manifests for both machines in the manifests directory. The basic unit in a Puppet manifest is a resource, which consists of a type, a title and several parameters:

resource_type { 'resource_title':
  param1 => 'value1',
  param2 => 'value2'

Resources can stand for quite a lot, e.g.

  • simple files, directories, symlinks
  • users and groups
  • packages
  • services
  • cronjobs

A complete list of all builtin types with all their possible parameters can be found in the Puppet documentation.
Now, let’s first take a look at our database server manifest:

group { 'puppet': ensure => 'present' }
class mysql_5 {
  package { "mysql-server-5.1":
    ensure => present
  service { "mysql":
    ensure => running,
    require => Package["mysql-server-5.1"]
  exec { "create-db-schema-and-user":
    command => "/usr/bin/mysql -u root -p -e \"drop database if exists testapp; create database testapp; create user dbuser@'%' identified by 'dbuser'; grant all on testapp.* to dbuser@'%'; flush privileges;\"",
    require => Service["mysql"]
  file { "/etc/mysql/my.cnf":
    owner => 'root',
    group => 'root',
    mode => 644,
    notify => Service['mysql'],
    source => '/vagrant/files/my.cnf'
include mysql_5

First, we define a class ‘mysql_5′. Classes are a way to group together several resources into bigger units and then include them later on. The first resource in our class tells the underlying package manager (apt-get in this case) to check if the package ‘mysql-server-5.1′ is already installed and to install it if not. The ensure parameter describes the desired state of the package, so in this case we expect the package to be present on the machine.

The second resource is about the MySQL service. Again, our expected state is that it’s running, and if not, Puppet will start it for us. As Puppet does not guarantee a execution order, we need to tell it that this resource depends on the package to be handled first. We can do this with the require parameter, whose value is a reference on our package resource. Note that references on resources are started with a uppercase letter instead of the resources definitions, which start with a lower case letter.

The next resource is the execution of a command. We use the MySQL CLI to create the initial schema and user for our application. Again, we use require to ensure that the resource we depend on, the starting of the server process, will be executed before this one. In a optimal case we would not put the SQL statements with all the credentials directly into our manifest, but would maintain it in some sort of VCS or artifact repository and pull it from there. But for the sake of simplicity we keep it there for these examples.

The last source replaces the default MySQL configuration with ours, which configures our server to listen on remote connections. The notify parameter does – you might have guessed it – notify the MySQL service to restart if it was already running at the time the configuration changed.

Now, we can start our database server with

> vagrant up dbserver

and take a look at the output.

Puppet output for our DB server

Puppet output for our DB server

The output shows that puppet changed the package status of the MySQL package from ‘purged’ to ‘present’, simply meaning that it instructed the package manager to install it. Furthermore we see that our configuration file has been copied and that the service has been refreshed after that.

Now we can check out if everything went well and try to log into our database server (password: dbuser):

> mysql -h -u dbuser -p

In a similar way, we write a manifest for our application server:

group { 'puppet': ensure => 'present' }
class sun_java_6 {
  $release = regsubst(generate("/usr/bin/lsb_release", "-s", "-c"), '(\w+)\s', '\1')
  # adds the partner repositry to apt
  file { "partner.list":
    path => "/etc/apt/sources.list.d/partner.list",
    ensure => file,
    owner => "root",
    group => "root",
    content => "deb $release partner\ndeb-src $release partner\n",
    notify => Exec["apt-get-update"],
  exec { "apt-get-update":
    command => "/usr/bin/apt-get update",
    refreshonly => true,
  package { "debconf-utils":
    ensure => installed
  exec { "agree-to-jdk-license":
    command => "/bin/echo -e sun-java6-jdk shared/accepted-sun-dlj-v1-1 select true | debconf-set-selections",
    unless => "debconf-get-selections | grep 'sun-java6-jdk.*shared/accepted-sun-dlj-v1-1.*true'",
    path => ["/bin", "/usr/bin"], require => Package["debconf-utils"],
  package { "sun-java6-jdk":
    ensure => latest,
    require => [ File["partner.list"], Exec["agree-to-jdk-license"], Exec["apt-get-update"] ],
class tomcat_6 {
  package { "tomcat6":
    ensure => installed,
    require => Package['sun-java6-jdk'],
  package { "tomcat6-admin":
    ensure => installed,
    require => Package['tomcat6'],
  service { "tomcat6":
    ensure => running,
    require => Package['tomcat6'],
    subscribe => File["mysql-connector.jar", "tomcat-users.xml"]
  file { "tomcat-users.xml":
    owner => 'root',
    path => '/etc/tomcat6/tomcat-users.xml',
    require => Package['tomcat6'],
    notify => Service['tomcat6'],
    content => template('/vagrant/templates/tomcat-users.xml.erb')
  file { "mysql-connector.jar":
    require => Package['tomcat6'],
    owner => 'root',
    path => '/usr/share/tomcat6/lib/mysql-connector-java-5.1.15.jar',
    source => '/vagrant/files/mysql-connector-java-5.1.15.jar'
# set variables
$tomcat_password = '12345'
$tomcat_user = 'tomcat-admin'
include sun_java_6
include tomcat_6

The class ‘sun_java_6′ includes a few steps which lead to Sun/Oracle Java being installed on our box. First the Ubuntu partner repository is added, then a ‘apt-get update’ is run to ensure that the package is available. Then the package itself is installed (source:

The second class is responsible for installing and starting Tomcat and the Tomcat Manager application. Furthermore we use a template to replace the tomcat-users.xml file. In the template, the line of interest is


The variables get replaced later with the values we set further down in the manifest. To be able to connect to our database server later on, we also add a MySQL connector to Tomcats classpath. Again, for simplicity reasons, we just put it into the files directory but the above argument still holds that you would normally have it somewhere central, e.g. in your artifact repository and pull it from there. With the subscribe parameter, we tell the tomcat service to restart if any of the referenced resources changes.

Now we can also start our application server:

> vagrant up appserver

If everything has gone right, you’ll be able to got to and login in with the above configured credentials (tomcat-admin/12345).

With that, our test infrastructure is complete, and now we can connect to booth the Tomcat and the MySQL instance from within our IDE, we may use Maven profiles to integrate them, just deploy stuff by hand or whatever we like.

If the environments are not of use for the moment, we can halt them to restart them later, or we can destroy them completely if we need them no more or want to restart from scratch with our configured state. If you run destroy without a box name, all boxes in the current Vagrantfile will be destroyed.

> vagrant halt
> vagrant destroy


The advantages of such a setup are obvious. Given you use freely available base boxes or have a central repository in your organization which contains your base boxes, it easy for a new developer to set up a test environment on demand, as needed. Also, changes to the environment can be distributed in an easy way. Simply put the Vagrant project under version control, so your developers just need to update, rebuilt the boxes and everybody is guaranteed to have the same setup, without the risk of having several different states on several developer machines. Your environment is always just on command away.

If you consequently think ahead there’s another advantage. You can also manage test, QA, staging and even production environments with Puppet. Put your configurations into a central place, versioned and parameterized for several environments, maintained collaboratively by development and operations (well, the DevOps buzzword had to pop up, right? 😉 ). This eliminates all the small and not so small differences between your environments that lead to all these “… but it used to work well in environment X” situations you really want to avoid. The next article on the topic will show the client/server setup of Puppet and how to work with it to target several environments.


  • Facebook
  • Delicious
  • Digg
  • StumbleUpon
  • Reddit
  • Blogger
  • LinkedIn
Bastian Spanneberg

12 Responses to Automated virtual test-environments with Vagrant and Puppet

  1. Humber says:

    Very nice post! Thanks
    Something useful would be having Vagrant file and Puppet manifest versioned in a SCM (E.g. Git, Subversion, etc) and let the team “checkout” (clone for Git) the proper files for a project.

  2. Hendrik Ebbers says:

    wenn ich das ganze ausführe bekomme ich folgende Fehlermeldung:
    * The network type ‘’ is not valid. Please use
    ‘hostonly’ or ‘bridged’.

    Nachdem ich die IPs einfach mal auskommentiert hab, starten die VMs. Allerdings kann Java nicht installiert werden:

    notice: /Stage[main]/Sun_java_6/Package[debconf-utils]/ensure: ensure changed ‘purged’ to ‘present’
    notice: /Stage[main]/Sun_java_6/Exec[agree-to-jdk-license]/returns: executed successfully
    notice: /Stage[main]/Sun_java_6/File[partner.list]/ensure: defined content as ‘{md5}6b40260fb1c76397bef841ad845cd77e’
    notice: /Stage[main]/Sun_java_6/Exec[apt-get-update]: Triggered ‘refresh’ from 1 events
    err: /Stage[main]/Sun_java_6/Package[sun-java6-jdk]/ensure: change from purged to latest failed: Could not update: Execution of ‘/usr/bin/apt-get -q -y -o DPkg::Options::=–force-confold install sun-java6-jdk’ returned 100: Reading package lists…
    Building dependency tree…
    Reading state information…
    Package sun-java6-jdk is not available, but is referred to by another package.
    This may mean that the package is missing, has been obsoleted, or
    is only available from another source
    E: Package sun-java6-jdk has no installation candidate
    at /tmp/vagrant-puppet/manifests/appserver.pp:35

    Ich hab mir das Projekt aus GIT ausgecheckt, kann also eigentlich nix falsch gemacht haben.
    Ich hab das ganze auf meinem MacBook ausprobiert und mir jeweils die neusten Versionen der Tools besorgt:
    Puppet 2.7.16
    Vagrant 1.0.3
    VirtualBox 4.1.16

    Hat irgendjemand eine Ahnung was ich falsch mache? Wär für jede Hilfe dankbar. Ich meine mich zu erinnern, dass Orales Java aus dem Paketmanager von Ubuntu entfernt wurde. Allerdings war das deutlich bevor dieser Artikel geschrieben wurde…

    • Bastian Spanneberg says:

      Hi Hendrik,

      das Beispiel wurde noch mit einer älteren Version von Vagrant erstellt. In der aktuellen Syntax muss es an der betroffenen Stelle heissen: :hostonly, “”

      Was Java angeht: Ich erinnere mich gelesen zu haben dass die Java 6 Paktete aus den Packages entfernt wurden, d.h. die Lösung funktoniert leider so nicht mehr. Ich werde zusehen den Artikel demnächst mal auf den neuesten Stand zu bringen und eine neue Lösung für die Java-Installation einzubauen.

      Danke jedenfalls für die Kommentare und die Erinnerung dass ich es mal aktualisieren muss 😉

  3. Hendrik Ebbers says:

    Den DB-Server bekomm ich auch nicht ans laufen. Hier kommt folgende Fehlermeldung:

    err: /Stage[main]/Mysql_5/File[/etc/mysql/my.cnf]/ensure: change from absent to file failed: Could not set ‘file on ensure: No such file or directory – /etc/mysql/my.cnf.puppettmp_3157 at /tmp/vagrant-puppet/manifests/dbserver.pp:25
    err: /Stage[main]/Mysql_5/Package[mysql-server-5.1]/ensure: change from purged to present failed: Execution of ‘/usr/bin/apt-get -q -y -o DPkg::Options::=–force-confold install mysql-server-5.1′ returned 100: Reading package lists…
    Building dependency tree…
    Reading state information…
    The following extra packages will be installed:
    libdbd-mysql-perl libdbi-perl libhtml-template-perl libmysqlclient16
    libnet-daemon-perl libplrpc-perl mysql-client-5.1 mysql-client-core-5.1
    mysql-common mysql-server-core-5.1
    Suggested packages:
    dbishell libipc-sharedcache-perl tinyca mailx
    The following NEW packages will be installed:
    libdbd-mysql-perl libdbi-perl libhtml-template-perl libmysqlclient16
    libnet-daemon-perl libplrpc-perl mysql-client-5.1 mysql-client-core-5.1
    mysql-common mysql-server-5.1 mysql-server-core-5.1
    0 upgraded, 11 newly installed, 0 to remove and 23 not upgraded.
    Need to get 24.2MB of archives.
    After this operation, 61.0MB of additional disk space will be used.
    Err lucid-updates/main mysql-common 5.1.62-0ubuntu0.10.04.1
    404 Not Found [IP: 80]
    Get:1 lucid/main libnet-daemon-perl 0.43-1 [46.9kB]
    Err lucid-security/main mysql-common 5.1.62-0ubuntu0.10.04.1
    404 Not Found [IP: 80]
    Get:2 lucid/main libplrpc-perl 0.2020-2 [36.0kB]
    Get:3 lucid/main libdbi-perl 1.609-1build1 [801kB]
    Err lucid-updates/main libmysqlclient16 5.1.62-0ubuntu0.10.04.1
    404 Not Found [IP: 80]
    Get:4 lucid/main libdbd-mysql-perl 4.012-1ubuntu1 [137kB]
    Err lucid-security/main libmysqlclient16 5.1.62-0ubuntu0.10.04.1
    404 Not Found [IP: 80]
    Err lucid-updates/main mysql-client-core-5.1 5.1.62-0ubuntu0.10.04.1
    404 Not Found [IP: 80]
    Get:5 lucid/main libhtml-template-perl 2.9-1 [65.8kB]
    Err lucid-security/main mysql-client-core-5.1 5.1.62-0ubuntu0.10.04.1
    404 Not Found [IP: 80]
    Err lucid-security/main mysql-client-5.1 5.1.62-0ubuntu0.10.04.1
    404 Not Found [IP: 80]
    Err lucid-security/main mysql-server-core-5.1 5.1.62-0ubuntu0.10.04.1
    404 Not Found [IP: 80]
    Err lucid-security/main mysql-server-5.1 5.1.62-0ubuntu0.10.04.1
    404 Not Found [IP: 80]
    Failed to fetch 404 Not Found [IP: 80]
    Failed to fetch 404 Not Found [IP: 80]
    Failed to fetch 404 Not Found [IP: 80]
    Failed to fetch 404 Not Found [IP: 80]
    Failed to fetch 404 Not Found [IP: 80]
    Failed to fetch 404 Not Found [IP: 80]
    Fetched 1086kB in 7s (136kB/s)
    E: Unable to fetch some archives, maybe run apt-get update or try with –fix-missing?

    notice: /Stage[main]/Mysql_5/Service[mysql]: Dependency File[/etc/mysql/my.cnf] has failures: true
    notice: /Stage[main]/Mysql_5/Service[mysql]: Dependency Package[mysql-server-5.1] has failures: true
    warning: /Stage[main]/Mysql_5/Service[mysql]: Skipping because of failed dependencies
    notice: /Stage[main]/Mysql_5/Exec[create-db-schema-and-user]: Dependency File[/etc/mysql/my.cnf] has failures: true
    notice: /Stage[main]/Mysql_5/Exec[create-db-schema-and-user]: Dependency Package[mysql-server-5.1] has failures: true
    warning: /Stage[main]/Mysql_5/Exec[create-db-schema-and-user]: Skipping because of failed dependencies
    notice: Finished catalog run in 9.00 seconds

    Ich habe mal ein wget ausprobiert, das schlägt auch fehl:
    –2012-06-18 20:04:06–
    Resolving…,,, …
    Connecting to||:80… connected.
    HTTP request sent, awaiting response… 404 Not Found
    2012-06-18 20:04:16 ERROR 404: Not Found.

    Echt schade. Find den Artikel absolut super, die Skripte laufen nur irgendwie mom. nicht…

    • Bastian Spanneberg says:

      Danke für den Hinweis, ich schaue mir das die Tage mal an und werde auch hier aktualisieren. Der Fehler ist mir neu und scheint mir auch mit geänderten Packages zu tun zu haben. Ich werd’ der Sache auf den Grund gehen.

  4. Nils says: ist eine IP Adresse die dem US Verteidigungsministerium zugewiesen ist ( Besser ist es sich ein Netz aus den folgenden Bereichen zu nehmen:

    je nachdem wie das lokale netzwerk aufgesetzt ist.

  5. Hi Bastian,
    da du dich ja für Vagrant / Java interessierst, könnte folgendes sehr interessant für dich sein:

  6. Patrick says:

    I got 404 errors when provisioning. To solve that I added:

    exec { ‘apt-get update':
    command => ‘/usr/bin/apt-get update’

    to the top of my puppet config.

    Thanks for a great tutorial!

    • Bastian Spanneberg says:

      Thx for the hint Patrick. That’s a common problem with Vagrant boxes, especially when the are a bit older already, as the package sources/info might change, compared to the state that is packaged in the box.

      An alternative to putting the update into Puppet is to use a shell provisioner ( before your Puppet provisioner, in order to keep the manifests clean. If you put more than one provisioning declarations into your Vagrantfile, they get executed in the order of appearance.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>