Here at RJMetrics, we’ve been expanding, refining and refactoring our software since the early days, but in the past year or so, we’ve made some big changes.

Our code has seen two fundamental and interrelated changes:

  1. The introduction of Clojure into our codebase.
  2. Moving our software to an architecture composed of microservices. We’re extremely pleased with the direction this is taking us, but as always, there have been new problems to solve along the way.

When we began to think about how to break our system down, we saw that decoupling our databases from data access logic was going to be a roadblock early on. In order to be of any use, new services would need to be able to interact with the data models that live at the heart of our application. This includes entities such as charts, dashboards, users, and a whole host of others that define the state of our system.

At the time, all of the logic that understood how those entities were represented in databases was written in PHP. Any new services we built on another platform had no means to reuse it. We decided that logic needed to move out on its own and expose a clean, consistent, platform-agnostic interface to allow reuse. On the way to realizing that goal, we developed a library.

Meet Sweet-Liberty the Library

Sweet-Liberty is a Clojure library designed to simplify the process of building services that:

  1. Manage data stores and,
  2. Expose a consistent RESTful API.

You can think of it as a means to build and configure components that translate (both ways!) between REST and SQL. Or, you might say that it helps you wrap a REST interface around a relational database. You can find it here on github

Here’s a very simplified explanation of what it can do.

1. A service built with Sweet-Liberty receives an HTTP request.

2. Sweet-Liberty translates the request into a SQL query.

3. Executing the SQL yields a result set.

4. Sweet-Liberty translates the result set to a JSON-formatted response and returns it to the calling code.

But, wait! There’s more!

Endpoints

An endpoint powered by Sweet-Liberty provides a number of operations by default. These can be accessed via query parameters in the request URL. Operations can by used in conjunction.

Examples of each operation are below. Each shows a URL, the resulting SQL and (for the first two) a result set in json.

Request only a subset of fields using the _fields argument.

Filter results to only return those that meet some condition.

The paging query parameters can be used to sort and divide the collection of records into pages and return only the page requested.

Additionally, the query parameters above can be used in combination.

You can find a full example application here for a service that manages dogs. Below are two excerpts from the core.clj in that project. These two snippets fully specify everything that is unique to the service — the configuration and the routes. In total, these clock in just under 50 lines of code, not too bad.

Configuration Map

The configuration map is where you specify each of the tables your service will manage, as well as their respective field lists. There are several other options that can also be configured such as default filters, name transformations, and pre- and post- execution conditions, but I’m sticking with the default behavior to keep this clear and simple.

Routes

The other critical piece is the routes. This example uses the Compojure library to provide an interface for defining routes. The code below defines three routes. The first two handle GET and POST requests to the collection endpoint (ie /dogs). The last one handles all requests to the entity endpoint (ie /dogs/1).

Ultimately, we hand a liberator resource value over to compojure. This resource is built by the make-resource function, which take a sweet-lib configuration map as its only argument. The expression threads the initial base configuration through a series of sweet-lib functions, each one making changes along the way.

Each change specifies some additional details about how the route will respond to requests. For example, “(sl/add-exists :id)” adds a handler that will use the “id” field of the resource to determine whether it already *exists* in the database before any further processing.

Because the amended map is exposed after each successive call, you are able to make other, custom modifications if necessary. The door is open for extensibility here.

How to use it

The library is available on clojars at:

The example application is a good place to start. Check it out on github. It includes a namespace that will examine the Sweet Liberty configuration and build an in-memory database on the fly. This isn’t appropriate in a production scenario, but it makes it easy to jump in and play around.

I take great pleasure in slashing boilerplate code through abstraction. This is our attempt to do that for RESTful services written in Clojure. I’m excited by our early progress so far. I hope others will find it useful in their own projects and maybe even join in to help grow this into something even better.

How to contribute

We are happily accepting pull requests. If you have questions or comments, head over to the google group, or submit as an issue on github.

  • Dave Bryand

    Very cool stuff, thanks for sharing.

    FYI, it looks like steps 1 and 2 are off from the expected query in step 3.

    Also, GET /dogs_fields=id&_fields=name -> GET /dogs?_fields=id&_fields=name

    • Bill Piel

      Fixed. Thanks!

  • Grubnut

    really nice work guys! How did you find working with the Liberator lib? I see you pass in a sweet-lib configuration map, does this mean Liberator can take an optional map of stuff, or do you use your make-resource fn to convert that map into a resource? If that makes sense? 🙂

    • Bill Piel

      Sweet Liberty would not have been possible without Liberator. I (we, if I can speak for the rest of the team) were very pleased with the library. There was only one limitation that we found necessary to hack around.

      I *think* I understand your question regarding the configuration map and liberator resources. At a high level, it would be accurate to say that the sole function of sweet lib is to produce a resource specification that can be handed over to liberator. So, yes, sweet lib’s make-resource fn boils the sweet-lib config down to a resource and nothing more.

      However, a liberator resource specification has many slots for different hooks that allow for custom behavior. Some of those hooks are populated with functions constructed by sweet-lib. When a liberator resource is executed (ie when a request comes in), embedded sweet-lib logic is in there doing the real processing work that results in a response.

      Did I succeed in answering your question?

      • Grubnut

        Yes thanks! Very helpful 🙂