Project Focus: Chef Supermarket - An Overview

Oct 8, 2014 • Mike Biang

Beginning in 2014, FullStack began a partnership with Chef (formerly Opscode) to redevelop Chef’s community site. The Chef community site was a destination for the open source community to browse, download, and share Chef cookbooks. The existing community site had a vibrant and involved user-base, however it had started to become outdated, and was missing several key features. Chef desired to open-source their community site in order to allow contributions to the functionality, but the existing codebase contained proprietary libraries and source code, making open-sourcing it impossible. In order to quickly and efficiently deliver an enhanced open-source community site, Chef reached out to us at FullStack. The new project took on the codename “Supermarket.”

Chef had several goals in mind for Supermarket:

1. Make the process of finding cookbooks easier.
2. Allow users to continue to share their cookbooks on Supermarket using existing tools.
3. Update the process of becoming a contributor to Chef’s open source projects in order to automate the process of authorizing contributions.
4. In addition to preserving the cookbook directory functionality, create a directory of other open source Chef tools.

In this article, we’ll take a look at the core features of Supermarket, and how we accomplished those goals.

Finding Cookbooks

The main purpose of Chef’s community site was to provide a single directory of the wealth of open-source cookbooks available within the Chef community. From installing MySQL on a Linux distribution to setting up a url rewriting module for IIS on Windows, Supermarket makes it simple to find a specific cookbook. It supports both Full Text Search of the cookbook directoy, as well as the ability to browse cookbooks by most downloaded, most followed, and most recently updated.

One deliberate departure from the previous Chef community site was the removal of cookbook categories. The initial alpha versions of Supermarket allowed cookbooks to be filtered by category. While categories provide a nice hierarchical structure to the directory, the Chef community ultimately questioned their usefulness in making finding cookbooks easier. The breadth of platforms and types of cookbooks meant that many cookbooks simply were lumped into an “Other” category. Consequently, cookbook categories where not included in the initial release of Supermarket.

In addition to being able to search for cookbooks, Supermarket provides the ability to view the cookbook’s supported platforms, its README file, its list of dependencies (other cookbooks), its CHANGELOG, and a rating of the quality of the cookbook.

Sharing Cookbooks

Supermarket, as the community site for Chef, was designed to showcase open source cookbooks. A key element of Supermarket is the ability to share a cookbook with the rest of the community. In order to share a cookbook, the author uploads the cookbooks source files to the community site using a tool such as Knife or Stove. Knife uploads the cookbook to the Supermarket API, which then processes the cookbook, parses relevant information out of the cookbook’s source files, and adds the cookbook to the directory.

Once a cookbook exists in the directory, Supermarket also gives the author the ability to upload new versions of the cookbook. Supermarket maintains a version history, and will even email the CHANGELOG file, if one is provided, to any users following the cookbook.

Additionally, the cookbook author can designate other users as “collaborators” on a cookbook. Being a collaborator gives them the ability to also upload new versions edits to the community site.

Becoming a Chef Contributor

Another goal of the community site was to streamline the process of contributing back to Chef. As the majority of Chef’s products are open source, there is a large number of engineers and developers that contribute back to their projects. Contributing to Chef requires that developers first sign a Contributor License Agreement (CLA). Prior to Supermarket, signing a CLA was a semi-manual process that was cumbersome for all involved. With Supermarket, that process became automated and much more seamless.

Supermarket now provides a simple process to become a Chef-authorized contributor. In order to sign your CLA, you simply click the “Become a Contributor” button on Supermarket , after signing in with your Chef account. After linking your GitHub account to your user profile, you’ll be prompted to read and accept the Contributor License Agreement. Simply agree to the license, and you immediately are authorized to contribute back to Chef’s open source projects.

Supermarket also integrates with Chef’s open source workflow. Whenever a pull request is opened in Github on one of Chef’s open source projects, Supermarket will verify that all of the commit authors have signed a CLA. If any of the authors are not authorized to contribute, Supermarket will leave a comment on the pull request prompting the authors to sign the CCLA before the contribution can be accepted.

Finding Other Chef Tools

Finally, Supermarket contains a directory of other Chef open source tools in addition to cookbooks. Supermarket makes it easy for authors to share the different plugins that they’ve written to enhance the Chef ecosystem. These tools and plugins come in many different flavors, but mostly are enhancements to Chef’s existing suite of projects, such as Knife, Ohai, and Chef Server.

Clarifying the Vision

Building Supermarket also allowed Chef to evaluate and focus the vision and features of the community site. Several features from the previous community site were deliberately left out of the new version, including cookbook ratings, IRC logs, and cookbook categories (as mentioned above). While these features could have been ported over to Supermarket, the process of rebuilding the community site gave us the opportunity to evaluate their effectiveness and implementation.

In Conclusion

We’ve had a great time working with the Chef team to build Supermarket. It has a wealth of other features not documented here, so to learn more about it, we’d recommend creating an account, and using it yourself!

During the rest of this series of posts on Supermarket, we’ll be exploring some of the challenges we faced while building this platform, and some of the exciting things we’ve learned. Stay tuned!