Using Vagrant and Chef For Reproducible, Isolated Rails Development EnvironmentsSep 18, 2014 • Tristan O'Neil
Ever come late to the party in an ongoing Rails project? You start rolling through the Getting Started section in the README (if you’re lucky enough to have such a section) and then you’re faced with some obtuse error or even worse find yourself knee deep in Solr XML configuration. You reach out to one of your colleagues and they’re like “I dunno?! Works on my machine.” Let this article relieve you of the vocabulary “Works on my machine.” Let’s start our journey down the path of creating reproducible development environments with help from our friends Vagrant and Chef.
Vagrant is a tool developed by HashiCorp that
takes advantage of existing tools such as VirtualBox, VMWare or even AWS to
create virtual environments specifically for developing your application. You
define your Vagrant configuration in a single file named
Vagrantfile just uses a simple Ruby DSL so if you’re already familar with Ruby
you should feel right at home. In your Vagrantfile you can specify things like
what base image you want to use, what provisioner you want to use to setup your
application and some other various network and file system settings. Vagrant
also provides a command line interface for interacting with your virtual
environments, here’s an overview of some specifically helpful vagrant commands.
vagrant up- Downloads the inital Vagrant image that you specified in your Vagrantfile and runs any provisioners defined.
vagrant destroy- Shutsdown and destroys your Vagrant box.
vagrant provison- Runs your defined provisioner (in our case this will be Chef) on an existing Vagrant box.
vagrant ssh- SSHs into you Vagrant box.
vagrant halt- Shutdowns your Vagrant box.
When it comes to reproducible environments Vagrant is great, however, things
start to get really interesting when Chef gets thrown into the mix. Chef is a
configuration management tool maintained by Chef (the company, previously
Opscode). Vagrant supports a
provisioners, but at FullStack our tool of choice happens to be Chef. So what
is Chef’s role in reproducible environments? When you run
vagrant up without
a provisioner configured it will download a base image that you specify and
that’s it. OK, so now you have a virtual Ubuntu box now what? SSH in, install
Ruby, Postgres, Node right? Maybe a few years ago, but never again. This is
about reproducible environments. Chef allows you to programatically configure
your base Vagrant box to run your application. Chef manages the installation of
Ruby, Postgres, Node and whatever other obscure technology is required you run
your application. The end goal:
vagrant up ->
vagrant ssh ->
rails s ->
Recently I worked on a new Rails project and was determined take the high road, the Vagrant road. This project was your pretty typical Rails 4 + PostgreSQL application, at least pretty typical for FullStack. I started off with a pretty basic Vagrantfile, looked something like this:
Vagrant.configure('2') do |config| config.vm.box = 'chef/ubuntu-14.04' config.vm.network :private_network, type: 'dhcp' config.vm.network :forwarded_port, guest: 3000, host: 3000 config.vm.synced_folder './code', '/home/vagrant/code', nfs: true config.vm.provision :chef_solo do |chef| chef.cookbooks_path = ['chef/cookbooks'] chef.add_recipe 'recipe[cocoon::default]' end end
You start off with calling
Vagrant.configure which takes the Vagrant API
version number and a block.
config.vm.box specifies the base VM image, in
this case we’re using Chef’s Ubuntu 14.04 image. This image will be downloaded
from the Vagrant Cloud, a central repository for
Vagrant images. Next some networking settings, I’ve configured it to use a
dynamically assigned local addresss and am forwarding port 3000 on the virtual
machine to port 3000 on my local machine. This essentially allows to you fire
up your normal browser, go to http://localhost:3000 and it will be forwarded
along to the Rails appication running in the VM.
keep a local folder on your computer synced with a folder on the VM. This
allows you to make changes with your favorite editor locally and it will sync
the changes to your VM.
Take note that I’m supplying the
nfs: true option, If you’re using VirtualBox
this is a must to have a productive development experience.
NFS stands for Network
File System and it’s supported out the box on Mac OS X and is a package manager
install away on Linux machines. The last piece of configuration tells Vagrant
to provision with Chef and points the provisioner to the correct directory to
find recipes to provision with and what recipes to run.
The next piece to have a fully functioning Vagrant Rails development environment is Chef. Chef is the tool we’ll use to manage all of our applications dependencies, Ruby, PostgreSQL, Node, various development headers and some Ruby gems.
The main building blocks of Chef are cookbooks, recipes and resources. Within our Vagrantfile we’ve specified the provisioner to run the default recipe from the cocoon cookbook, so let’s start there.
include_recipe 'cocoon::_ruby' include_recipe 'cocoon::_postgres' include_recipe 'cocoon::_node'
In the default recipe we’re just using
include_recipe as a mechanism to
seperate our cookbook into logical pieces. You could also just include these
individual recipes within your Vagrantfile with
chef.add_recipe, but I find
accomplishing it this way to be a bit more flexible. The advantage being that
you could potentially have several recipes similar to the default recipe that
accomplish slightly different things. Using these sort of “container” recipes
makes it easy to understand the usecase at a glance of the recipes name. Let’s
take a look at the
package 'libxslt-dev' package 'libxml2-dev' package 'build-essential' package 'libpq-dev' package 'libsqlite3-dev' package 'software-properties-common' execute 'apt-add-repository ppa:brightbox/ruby-ng -y' do not_if 'which ruby | grep -c 2.1' end execute 'apt-get update' do ignore_failure true end package 'ruby2.1' package 'ruby2.1-dev' gem_package 'bundler' do gem_binary('/usr/bin/gem2.1') end gem_package 'rails' do gem_binary('/usr/bin/gem2.1') end
As I mentioned one of the building blocks of Chef is resources. A Chef resource
is basically just a set of Ruby DSLs to describe configuration that will
happen. The first part of the
_ruby recipe uses the
package resource to install
some various development dependencies for Ruby (mostly used to compile C
extensions for various Ruby gems). The
package resource will use whatever the
native package manager is for the system you’re running chef-client on, apt,
yum, etc. to install the package you define. If the package is already installed
then chef-client continues on its merry way.
Next we again use the
package resource to include
software-properties-common, an Ubuntu package which includes the
apt-add-repository tool. I should note that this cookbook is specific to Ubuntu
14.04. While most Chef resources are agnostic to what OS they’re running on, we
only care about provisoning our Vagrant box so we take some liberties with how
we construct this cookbook, using
apt-add-repository is one of them.
Taking advantage of the
execute resource we add the
brightbox/ruby-ng repository to apt and also execute
apt-get update to
fetch the package information from said repository. You’ll notice that for the
apt-add-repository execute block we’re using the
not_if guard to prevent
this from running during every chef-client run. Resources like
built in guards to prevent chef-client attemtping to install packages over and
over again. However, the execute block, since it could potentially do anything,
does not. You must use a
guard that is evaluated
before the execution and if it returns a status code of 0 it will not be
Now that the package manager knows about modern Ruby versions we can use the
package resource to install
ruby2.1-dev. Finally we’ll
install some various essential Ruby gems using the
gem_package works very similarly to the
package resource but using
yum. Ubuntu 14.04 comes with an older version of Ruby so
we’re telling the
gem_package resource to use the newly installed
binary to build these gems. Ruby should be all set to handle our Rails
application, the next thing we need is a database, on to the
package 'postgresql' package 'postgresql-contrib' execute 'createuser' do guard = <<-EOH psql -U postgres -c "select * from pg_user where usename='vagrant'" | grep -c vagrant EOH user 'postgres' command 'createuser -s vagrant' not_if guard, user: 'postgres' end
By now you know the drill, we need to first install
postgresql-contrib using the
package resource. That will get us 90% of the
way there, the only other thing our application needs is a PostgreSQL user.
Again we use the
execute block with the
createuser tool to create a superuser
named vagrant. We define a guard by querying the PostgreSQL database for a user
named vagrant ensuring this
execute block will only run once.
We are almost there! The final piece in the equation to have our application up and running is Node, required by the Rails asset pipeline. Don’t blink or you’ll miss this recipe though.
Yep, that’s it, not much to say here. Ubuntu 14.04’s default apt configuration
comes with a sufficient version of Node, so all we need to do is install it
I ended up abstracting the Vagrant and Chef setup that I’ve described here into an open source project called Cocoon. It’s a bit naive in that it will only really work well with a Rails 4 + PostgreSQL project, but I think it’s a great starting point. I hope to continue to iterate upon it as needs change. It will also serve as a good resource if you’re looking to learn more about the nitty gritty details of how to provision Vagrant with Chef.
I hope this has at least opened the door for you to continue to pursue the Vagrant lifestyle and I hope it sparks your interest in learning more about Chef. There is so much more to Chef that I haven’t covered here but if you’re looking to learn more I highly suggest the Chef Fundamental Series and the Docs are pretty great as well.