Go Project Structure For Rubyists

Ruby has been my language of choice for the past five years or so. When I write Ruby I’m usually building web applications using the Rails framework, you may have heard of it. One of the things that I value from Rails is its fairly rigid structure. One thing that I don’t value from the Rails framework is its tendency to be dog slow. It’s getting faster but as it does other languages and frameworks catch my eye.

One such language that I’ve been dabbling in recently is Go. Go is a statically typed, C style, compiled language created and maintained by Google. Go was created because of Google’s need for a fast but still productive and modern language. Initially I was drawn to Go because of its raw speed but since using it, I’ve started to appreciate its simplicity, tooling and growing community. This is not an article about why you should be using Go though. This is an article about my journey trying to understand the GOPATH, Go dependency management and Go packages. The end goal is to be able to organize a Go web project in a sane way.

$GOPATH

When you install the Go language it comes with a tool called go. The go tool expects you to have $GOPATH exported as a folder on your local file system. This is called a Go workspace. My Go workspace is located at ~/go. When you execute go get some-go-repo it fetches the Go package and places it into your Go workspace. When you run go run your-go-app that imports a third party go package it looks for the package in your $GOPATH. This is well and good except that if you want to work on a project across multiple computers or with multiple developers you have to vendor these third party packages. Luckily there are tools that help solve this problem. We’ll be taking a look at one called Goop.

Goop

Goop is a dependency manager inspired by Bundler created by the folks over at Nitrous.IO. Rather than using go get to manually install third party packages required by your application. Goop allows you to define a manifest with all of your dependencies, then install them with the goop tool. To get started with Goop you create a file in the root of your application called Goopfile. A Goopfile just consists of a list of the package repos that you would use with go get, you can also pin a dependency to a specific commit revision or tag. Here’s an example Goopfile that specifies two dependencies, one of them pinned to a commit sha.

github.com/mattn/go-sqlite3
github.com/gorilla/context #14f550f51af52180c2eefed15e5fd18d63c0a64a

Once you have your dependencies defined you can install them with goop install. This fetches the defined packages and places them in .vendor within your application directory. The first time you run goop install it also creates Goopfile.lock which pegs your dependencies to a specific version, running goop update will alter the lock file if there are new versions. One major caveat is if you run or build your application with go it won’t be able to find these dependencies. go is still at the whim of whatever $GOPATH you’ve defined. In order to run or compile your application you must use goop exec followed by any go command you wish to run. goop exec temporarily alters your $GOPATH and $PATH to include the source and bianary packages installed to .vendor.

Packages

When you compile a Go application it requires you to have your code defined in a package. The package main, which defines a function called main is the entry point of all Go applications. When you look at a lot of Go web application examples freqently everything is just dumped into this main package. This paradigm is madness. Coming from the world of Ruby on Rails I’m used to structure. I find it leads to greater productivity when you know exactly where to look to find the functionality you need to change. Recently I built a simple web service using Go and I broke it up into five separate, logical packages: main, db, model, route and test. Within the main package I initialized the database, mapped the routes and started the HTTP server.

package main

import (
  "./db"
  "./route"

  "github.com/go-martini/martini"
)

func main() {
  db.Init()
  m := martini.Classic()

  m.Get("/", route.GoalsIndex)
  m.Run()
}

Notice I’ve imported the db and route packages in order to initialize the database and assign route mappings to Go functions. One file that I have defined in theroute package looks like this:

package route

import (
  "encoding/json"
  "log"

  "../model"
  "../db"
)

//
// GoalsIndex returns an array of goals as JSON.
//
// GET /
//
func GoalsIndex() []byte {
  goals := []model.Goal{}
  db.Db.Find(&goals)
  goalsJSON, err := json.Marshal(goals)

  if err != nil {
    log.Fatal(err)
  }

  return goalsJSON
}

The implementation details aren’t importantant for this article, but notice I’m defining this file within the route package and defining a function named GoalsIndex. Take note that the capitialization of GoalsIndex is very important. We want to import the route package into our other packages and have access to this function via route.GoalsIndex. Making the function name uppercase tells the Go compiler that the function is exported, similar to public, and that other packages can use it. This holds true for other Go keywords as well.

When you’re defining packages Go expects the source files that live in that package to be within a sub-directory with the same name as the package. In the case of our route package source file it lives in a file called goals.go in a folder called route.

The other packages I have defined in my application are similar to the above example so there’s no need to dive into the details. However, when it comes to the test package there are some details worth covering. go comes with a command called test that will run your _test.go files within the directory you’re currently in. Since our test files are all within our test package and therefore in a sub-directory called test you must tell go test where to find you tests like so.

$ goop exec go test ./test/*

Another major caveat of having your tests live in a separate package is since you’ll need to import anything you’re testing, only exported (public) functions, variables, etc. may be tested. I generally come from the school of thought that you shouldn’t be testing private things so it hasn’t been a problem for me so far but only time will tell.

Conclusion

I hope this has shed a little light on the world of Go project structure. When I first started using Go I had a hard time getting over the fact that it seemed like the community embraced a mish-mash style of project organization. I get the sense that the way I’ve described Go project organization here is a bit unconventional but so far its worked for me and I’ll keep rolling with it until I hit a roadblock. The web service that I reference in this article can be found on GitHub. I encourage you to clone it down and take a look around. I’d be curious to hear from any readers if they’ve found a better way to organize Go projects or if the way I’m approaching it has any major drawbacks. I’m still new to this whole Go thing so any feedback would be welcomed with open arms.