Using Vagrant and Chef For Reproducible, Isolated Rails Development Environments

Ever come late to the party in an ongoing Rails project? You start rolling through the Getting Started section in the README (if you’re lucky enough to have such a section) and then you’re faced with some obtuse error or even worse find yourself knee deep in Solr XML configuration. You reach out to one of your colleagues and they’re like “I dunno?! Works on my machine.” Let this article relieve you of the vocabulary “Works on my machine.” Let’s start our journey down the path of creating reproducible development environments with help from our friends Vagrant and Chef.

Vagrant

Vagrant is a tool developed by HashiCorp that takes advantage of existing tools such as VirtualBox, VMWare or even AWS to create virtual environments specifically for developing your application. You define your Vagrant configuration in a single file named Vagrantfile. The Vagrantfile just uses a simple Ruby DSL so if you’re already familar with Ruby you should feel right at home. In your Vagrantfile you can specify things like what base image you want to use, what provisioner you want to use to setup your application and some other various network and file system settings. Vagrant also provides a command line interface for interacting with your virtual environments, here’s an overview of some specifically helpful vagrant commands.

• vagrant up - Downloads the inital Vagrant image that you specified in your Vagrantfile and runs any provisioners defined.
• vagrant destroy - Shutsdown and destroys your Vagrant box.
• vagrant provison - Runs your defined provisioner (in our case this will be Chef) on an existing Vagrant box.
• vagrant ssh - SSHs into you Vagrant box.
• vagrant halt - Shutdowns your Vagrant box.

Chef

When it comes to reproducible environments Vagrant is great, however, things start to get really interesting when Chef gets thrown into the mix. Chef is a configuration management tool maintained by Chef (the company, previously Opscode). Vagrant supports a variety of provisioners, but at FullStack our tool of choice happens to be Chef. So what is Chef’s role in reproducible environments? When you run vagrant up without a provisioner configured it will download a base image that you specify and that’s it. OK, so now you have a virtual Ubuntu box now what? SSH in, install Ruby, Postgres, Node right? Maybe a few years ago, but never again. This is about reproducible environments. Chef allows you to programatically configure your base Vagrant box to run your application. Chef manages the installation of Ruby, Postgres, Node and whatever other obscure technology is required you run your application. The end goal: vagrant up -> vagrant ssh -> rails s -> develop features.

Cocoon

Recently I worked on a new Rails project and was determined take the high road, the Vagrant road. This project was your pretty typical Rails 4 + PostgreSQL application, at least pretty typical for FullStack. I started off with a pretty basic Vagrantfile, looked something like this:

Vagrant.configure('2') do |config|
  config.vm.box = 'chef/ubuntu-14.04'

  config.vm.network :private_network, type: 'dhcp'
  config.vm.network :forwarded_port, guest: 3000, host: 3000
  config.vm.synced_folder './code', '/home/vagrant/code', nfs: true

  config.vm.provision :chef_solo do |chef|
    chef.cookbooks_path = ['chef/cookbooks']
    chef.add_recipe 'recipe[cocoon::default]'
  end
end

You start off with calling Vagrant.configure which takes the Vagrant API version number and a block. config.vm.box specifies the base VM image, in this case we’re using Chef’s Ubuntu 14.04 image. This image will be downloaded from the Vagrant Cloud, a central repository for Vagrant images. Next some networking settings, I’ve configured it to use a dynamically assigned local addresss and am forwarding port 3000 on the virtual machine to port 3000 on my local machine. This essentially allows to you fire up your normal browser, go to http://localhost:3000 and it will be forwarded along to the Rails appication running in the VM. config.vm.synced_folder will keep a local folder on your computer synced with a folder on the VM. This allows you to make changes with your favorite editor locally and it will sync the changes to your VM.

Take note that I’m supplying the nfs: true option, If you’re using VirtualBox this is a must to have a productive development experience. NFS stands for Network File System and it’s supported out the box on Mac OS X and is a package manager install away on Linux machines. The last piece of configuration tells Vagrant to provision with Chef and points the provisioner to the correct directory to find recipes to provision with and what recipes to run.

The next piece to have a fully functioning Vagrant Rails development environment is Chef. Chef is the tool we’ll use to manage all of our applications dependencies, Ruby, PostgreSQL, Node, various development headers and some Ruby gems.

The main building blocks of Chef are cookbooks, recipes and resources. Within our Vagrantfile we’ve specified the provisioner to run the default recipe from the cocoon cookbook, so let’s start there.

include_recipe 'cocoon::_ruby'
include_recipe 'cocoon::_postgres'
include_recipe 'cocoon::_node'

In the default recipe we’re just using include_recipe as a mechanism to seperate our cookbook into logical pieces. You could also just include these individual recipes within your Vagrantfile with chef.add_recipe, but I find accomplishing it this way to be a bit more flexible. The advantage being that you could potentially have several recipes similar to the default recipe that accomplish slightly different things. Using these sort of “container” recipes makes it easy to understand the usecase at a glance of the recipes name. Let’s take a look at the _ruby recipe.

package 'libxslt-dev'
package 'libxml2-dev'
package 'build-essential'
package 'libpq-dev'
package 'libsqlite3-dev'

package 'software-properties-common'

execute 'apt-add-repository ppa:brightbox/ruby-ng -y' do
  not_if 'which ruby | grep -c 2.1'
end

execute 'apt-get update' do
  ignore_failure true
end

package 'ruby2.1'
package 'ruby2.1-dev'

gem_package 'bundler' do
  gem_binary('/usr/bin/gem2.1')
end

gem_package 'rails' do
  gem_binary('/usr/bin/gem2.1')
end

As I mentioned one of the building blocks of Chef is resources. A Chef resource is basically just a set of Ruby DSLs to describe configuration that will happen. The first part of the _ruby recipe uses the package resource to install some various development dependencies for Ruby (mostly used to compile C extensions for various Ruby gems). The package resource will use whatever the native package manager is for the system you’re running chef-client on, apt, yum, etc. to install the package you define. If the package is already installed then chef-client continues on its merry way.

Next we again use the package resource to include software-properties-common, an Ubuntu package which includes the apt-add-repository tool. I should note that this cookbook is specific to Ubuntu 14.04. While most Chef resources are agnostic to what OS they’re running on, we only care about provisoning our Vagrant box so we take some liberties with how we construct this cookbook, using apt-add-repository is one of them.

Taking advantage of the execute resource we add the brightbox/ruby-ng repository to apt and also execute apt-get update to fetch the package information from said repository. You’ll notice that for the apt-add-repository execute block we’re using the not_if guard to prevent this from running during every chef-client run. Resources like package have built in guards to prevent chef-client attemtping to install packages over and over again. However, the execute block, since it could potentially do anything, does not. You must use a guard that is evaluated before the execution and if it returns a status code of 0 it will not be executed.

Now that the package manager knows about modern Ruby versions we can use the package resource to install ruby2.1 and ruby2.1-dev. Finally we’ll install some various essential Ruby gems using the gem_package resource. gem_package works very similarly to the package resource but using gem instead of apt or yum. Ubuntu 14.04 comes with an older version of Ruby so we’re telling the gem_package resource to use the newly installed gem2.1 binary to build these gems. Ruby should be all set to handle our Rails application, the next thing we need is a database, on to the _postgres recipe.

package 'postgresql'
package 'postgresql-contrib'

execute 'createuser' do
  guard = <<-EOH
    psql -U postgres -c "select * from pg_user where
    usename='vagrant'" |
    grep -c vagrant
  EOH

  user 'postgres'
  command 'createuser -s vagrant'
  not_if guard, user: 'postgres'
end

By now you know the drill, we need to first install postgresql and postgresql-contrib using the package resource. That will get us 90% of the way there, the only other thing our application needs is a PostgreSQL user. Again we use the execute block with the createuser tool to create a superuser named vagrant. We define a guard by querying the PostgreSQL database for a user named vagrant ensuring this execute block will only run once.

We are almost there! The final piece in the equation to have our application up and running is Node, required by the Rails asset pipeline. Don’t blink or you’ll miss this recipe though.

package 'nodejs'

Yep, that’s it, not much to say here. Ubuntu 14.04’s default apt configuration comes with a sufficient version of Node, so all we need to do is install it using the package resource.

I ended up abstracting the Vagrant and Chef setup that I’ve described here into an open source project called Cocoon. It’s a bit naive in that it will only really work well with a Rails 4 + PostgreSQL project, but I think it’s a great starting point. I hope to continue to iterate upon it as needs change. It will also serve as a good resource if you’re looking to learn more about the nitty gritty details of how to provision Vagrant with Chef.

I hope this has at least opened the door for you to continue to pursue the Vagrant lifestyle and I hope it sparks your interest in learning more about Chef. There is so much more to Chef that I haven’t covered here but if you’re looking to learn more I highly suggest the Chef Fundamental Series and the Docs are pretty great as well.