In an effort to improve the effectiveness of our website, we recently conducted our first A/B test. This test allowed us to compare two options for the content that users see after clicking the big orange button on our homepage. Thanks to these tests, our visitor-to-lead conversion rate has almost tripled.

Choosing a Goal

We used Google Website Optimizer (GWO) to run the tests, and our first step was deciding on a goal. We decided that a good goal needs to be achievable, measurable, and a meaningful step of the funnel that ends in RJMetrics making money. We also wanted the goal to be as close to the end of the funnel where we make money as possible.

For most businesses, a sale or other revenue-generating event is an obvious goal to use. In our case, however, prospective customers need to talk to a member of our team before making a purchase. This means our website doesn’t generate revenue directly, leaving us no revenue-based actions to use as goal completions. There are several reasons for this set-up, but the important thing is that it is not going to change for this A/B test.

Another goal we considered was the successful education of our visitors about our product. Educating potential customers about RJMetrics and the problems we solve is certainly an achievable goal that drives new customer acquisition. However, it is difficult to reliably measure. We could track how many people watch our promotional video to completion or how many pages they visit. But these specific events are not always meaningful, and frankly I don’t really care about them. Additionally, they are way over on the wrong side of the funnel.

A third option for our goal was the completion of our “get a free demo” form. It’s trivially easy to measure and provides prospects with a feel for what our software can do (by analyzing Vandelay Industries’ data). It also provides us with contact information for these now somewhat-qualified sales leads. This places it relatively far down the funnel. While it would be nicer to further qualify the leads before considering one a “goal completion,” that would be much more difficult to reliably monitor and would reduce our already-small sample size.

We decided that the demo form completion was the right goal for us to be optimizing for right now.

Conducting the Test

The old version of our landing page is still available at our How It Works page. It looks like this:

The page has a Flash movie that gives an overview of our target customers, the problems they face, and how RJMetrics can help. Below the flash movie, we have some marketing copy that gives additional color for people who would rather read than watch. We did not explicitly talk about a goal when creating the page, but if I had to describe what it does, I would say that this page gives information on RJMetrics and gives prospects a way to try out our free demo.

The winning version of the page is our new landing page. It looks like this:

We removed the movie and marketing copy and increased the size of the form fields. We also added a description of what will happen after filling out form. The goal was to minimize the commitment required and make the reciprocity explicit. This page is unambiguous about its goal: driving demo form completion.

The Results

We did a total of 3,519 trials. The original version yielded a conversion rate of 2.1%, and the winning yielded a conversion rate of 6.1%, providing an improvement of over 190%.

We learned a few valuable lessons from this experiment:
1) We need to set an explicit goal for our website, or at the very least for individual pages.
2) Remove anything that does not explicitly drive our goal forward. In this case, more content meant fewer conversions.
3) We need to be testing a lot more.