As programmers, we create code and libraries for others to use. There’s no getting around this fact: if nobody else ever reads your code, you probably aren’t going to get very far. This gives rise to a question: how can we tell if the code we’ve designed is useful for others, and also ourselves? The classic text Structure and Interpretation of Computer Programs (http://mitpress.mit.edu/sicp/) spends a lot of time delving into this issue. The first section of Chapter 4 contains a few sentences that I can safely say have had a bigger impact on how I write code than anything I’ve ever encountered. The text reads:

Metalinguistic abstraction — establishing new languages — plays an important role in all branches of engineering design. It is particularly important to computer programming, because in programming not only can we formulate new languages but we can also implement these languages by constructing evaluators. An evaluator (or interpreter) for a programming language is a procedure that, when applied to an expression of the language, performs the actions required to evaluate that expression.

It is no exaggeration to regard this as the most fundamental idea in programming:

The evaluator, which determines the meaning of expressions in a programming language, is just another program.

To appreciate this point is to change our images of ourselves as programmers. We come to see ourselves as designers of languages, rather than only users of languages designed by others.

Admittedly, this is very general and abstract! Your eyes might have glazed over at the phrase “metalinguistic abstraction”, but try to bear with me. How can we take this idea and use it in our day-to-day work? The next paragraph offers an example from earlier in the text:

In fact, we can regard almost any program as the evaluator for some language. For instance, the polynomial manipulation system of section 2.5.3 embodies the rules of polynomial arithmetic and implements them in terms of operations on list-structured data.

Ah, so what they’re saying can apply to any program you write! This takes a shift in mindset to appreciate, but once you make that shift, you’ll find that your programs are clearer and more helpful. If you follow these principles, your code will give you the ability to:

• Work with the code instead of having it work against you.
• Solve problems in new and interesting ways that you couldn’t fathom when you did your initial design work.

How does this apply to your domain?

You might be thinking: “This is fine for somebody creating small systems that are obviously reusable, like mathematics packages. However, my application in <your domain here> doesn’t facilitate that kind of reusable design!” Let’s consider a practical example: the RJMetrics application. This is a large system created for gathering data and performing analytics on it to provide useful business knowledge to our clients.

I’m going to propose a different separation of concerns. Instead of only creating an application for people to use and analyze their business data, we should design a language for solving business analytics problems, and then build an application on that foundation. You can do this for any domain–of course, the previous sentence can always be rewritten to include your problem domain instead of business analytics.

An example will probably help to clarify what I mean. One of the fundamental building blocks of our system is the “structure” of a client’s database. Assume, for the sake of this example, that this system only supports relational databases (though this is not the case for the RJMetrics platform). This is made up of your standard database objects: tables, columns, schemata, etc. Now, what happens when you decide you no longer want to support only relational databases? You want to move into the NoSQL realm, so you need ways of representing the structure of a NoSQL store that can be used in the same fashion as your current structure building blocks. We should be able to assemble the existing pieces (or introduce new ones transparently) so that NoSQL data stores can be manipulated and used in the same fashion–you certainly don’t want to reimplement your fundamental building blocks to accomodate a new way of structuring data. If this is an easy task (and you’ll know when you go to implement it if that’s the case!), you might have designed your system in a language-oriented way.

Note I’m using the word “language” in a different sense than we usually mean when discussing code: I don’t mean a programming language, and I’m certainly not advocating that you implement a new programming language for your specific problem domain! Rather, I’m talking about the API you use to build the application–and if you consider these API design principles, it will be much harder for you to end up with a poorly-designed interface.

How can you tell if your platform has these qualities?

This is a difficult problem, but here are some thought exercises to make it easier. Try considering this: assume that tomorrow, you’re leaving the application business and want to focus solely on creating a platform for others to use and create their own solutions. First, would people be able to comprehend the language you’ve provided for them? Do the primitives and means of combination make sense? Second, if somebody comes up with a new way of using your platform, are your current tools going to work with their new ideas, or are they going to break? On a more implementation-specific level, any side effects in the code will make this impossible–someone will inevitably call a function with side effects at the wrong time and it will blow up in their face. Functional solutions are the only option here.

Or, to put it more succinctly:

• Could you open source your platform tomorrow?
• Would you be embarrassed by what people saw?

The RJMetrics codebase isn’t there yet (personally, I’ve seen very few production codebases that are), but as we consistently refactor and notice patterns, the language evolves and becomes more useful. A nice side effect is that these changes stack on top of each other: every incremental change makes it easier and easier to improve it even further.

On that note, this is the most difficult part of getting to this point with your code: you can never neglect refactoring. No codebase gets it right the first time, so you have to be willing to find patterns and exploit them. That’s the only way you can ever evolve your platform to the level of a reusable language.