blog post cover image-01

A/B testing has long been a favorite tool of growth hackers, and the practice is catching on among marketers everywhere. As companies invest more in creating a seamless online experience, they’re willing to invest more in making sure that experience is fully optimized.

Yesterday we teamed up with the data nerds at Optimizely to talk about how companies can move toward a scientific approach to their A/B testing. If you missed it, you can download the slide deck or watch the full event here:

Here’s what you missed

Optimizely kicked it off with some…

Screen Shot 2014-07-23 at 3.09.21 PM

They addressed some of the common misconceptions that people have about A/B testing:

  • Validation of guesswork (i.e., “Design thinks A, marketing thinks B. Let’s do an A/B test to see who’s right!”)
  • Consumer psychology gimmicks (i.e., “Red buttons get more people to click.”)
  • Meek tweaking (i.e., “A series of incremental improvements will grow my business.”)

A/B testing is the practice of conducting experiments to optimize your customer experience. On high-impact pages, the return on time can be huge and more and more marketers are tapping into the power of A/B testing.

Step 1: Analyze data

Anyone who has done any amount of A/B testing knows that the disappointment doesn’t come from having your assumptions proven wrong, but rather from high numbers of inconclusive tests. Asking the right questions is surprisingly difficult.

The good news it that your data can point you to the tests that will have the highest impact. Quantitative data in the form of web traffic, email marketing, order history, etc. is useful in helping you identify where your test will have the great impact on business results. Qualitative data in the form of user testing, heat mapping, or survey data is great for helping you identify what elements of a page should be tested.

Step 2: Form a hypothesis

Once you know what needs to be tested, the second step is forming a good hypothesis. A good hypothesis is made up of three parts:

  1. Variable: the element being modified
  2. Result: the predicted outcome
  3. Rationale: what assumption will be proven wrong if the experiment is a draw or loses?

Forming a good hypothesis is foundational for effective A/B testing. If you want to get into the details on this topic, it’s worth reading this post.

Step 3: Construct an experiment

Once you know where your test will have the most impact and have determined your hypothesis, it’s time to get your hands dirty and construct an experiment. Every website test will contain at least one of these three core elements:

  • Content: what you’re saying
  • Design: how it looks
  • Tech: how it works

The most effective tests often combine all three elements.

While A/B testing is often used for simple things like copy changes:

Screen Shot 2014-07-24 at 5.36.38 PM

It can be used for complex business processes as well. Currently, we’re running an A/B test to identify the sales process that delivers the optimal experience for prospects:

Screen Shot 2014-07-24 at 5.36.52 PM

Step 4: Evaluate results

Now, for just a little bit of statistics 101. For every experiment you run, you want to be sure that the observed change was not due to chance. Statistical significance provides that indicator. For example, test results with 95% statistical significance have only a 5% chance of the change being due to chance.

What this means for the tester is that significance is a matter of risk. Higher confidence means a lower chance that you’ll implement the winning test result and realize the A/B test didn’t predict actual outcomes. It works something like this:

Screen Shot 2014-07-23 at 4.47.55 PM

If you’re running A/B tests manually, Optimizely has a handy calculator that any one can use to analyze test results.

Getting your team on board with A/B testing

A/B tests focused on website optimizations will get results, but the impact of tests grow with greater investment.

Screen Shot 2014-07-23 at 4.53.08 PM

Some ways to get people in your organization excited about testing (and willing to pitch in some resources) include:

  • Documenting your test results in a central repository where anyone can catch up with what your team has been learning from its tests
  • Building excitement by sharing your wins with the company. Experimentation is fun, the more you share, the more other people will care.
  • Introduce some competition by having people vote on variations. Find out who in your organization has the highest accuracy of predicting winners.
  • Next Steps

    A/B testing is a powerful tool to improve your customer experience. Several attendees had questions about how they could keep learning about A/B testing. We recommend the following:

  • Building Your Company’s Data DNA
  • How to Use Data to Choose Your Next A/B Test
  • The Ultimate Guide to A/B Testing
  • From Testing Newbie to Conversion Optimization Beast
  • Stop Wasting Time on A/B Tests That Don’t Matter
  • Of course, we would also recommend scrolling back up to the top of the page and watching the webinar you missed. These people agree.

    birdies

    If you’re a content marketer, you’ve probably noticed that content marketing is growing. If it seems like everyone is doing it, that’s probably because pretty much everyone is.

    The Crowded Content Landscape

    crowded content landscape

    Here’s what Google Trends tells us about content marketing. You can see that people have been talking about content marketing for several years, but we start seeing a definite uptick toward the end of 2012 and through 2013.

    Which makes sense, since 91% of B2B marketers use content marketing today. In addition to the number of marketers using content marketing, there’s also been an increase in the kinds of content, blogs, videos, white papers, ebooks, social media messaging…but it still isn’t enough. 58% of marketers plan to increase the amount of content they’re producing over the next 12 months.

    reality

    That’s a whole lot of content, and a really crowded space. With so much competition, and knowing your intended audience has a limited amount of time for research, you need more than just great content to attract people’s attention. You have to actively work to get your content in front of them.

    Continue reading

    PredictableRevenueCover

    No matter if you’re an embedded member of a long established company, a single entrepreneur just striking out, or if you’re in the first, hopeful stages of start-up-dom — you need a way to reliably predict your revenue.

    Last week, we teamed up with the fantastic Aaron Ross, author of “Predictable Revenue.” Aaron played a significant role in the behemoth success of Salesforce.com, and his book has been dubbed the Silicon Valley Sales Bible.

    Aaron’s webinar was a crash-course covering Predictable Revenue basics like:

    • The three fatal mistakes sales leaders make
    • How to build an outbound sales machine that can triple your pipeline
    • Why salespeople shouldn’t prospect

    Continue reading

    arrows

    Jeff Clavier, Founder and Managing Partner of SoftTech VC, estimates that he’s heard about 10,000 pitches in his life. Ask him what makes the best ones stand out and he’ll tell you: data. Recently Jeff joined our CEO, Robert J. Moore, for a Q&A webinar, Raising Venture Capital with Data. It’s worth watching all 53 minutes of the recording, or you can take the shortcut and catch some of the highlights here.

    Full Disclosure: SoftTech VC is an investor in RJMetrics.

    Choosing Where to Invest

    On an annual basis SoftTech VC hears several thousand pitches, closing only about 20 of them. One of the most surprising data points that Jeff shared speaks to the importance of a strong network: over 9 years and the 143 closed deals SoftTech VC has made, there are 0 that came without an introduction.

    How Founders Should Look at Data

    Obviously, data is fundamental to what VCs are doing. The challenge for both investors and founders is that in early-stage companies there can be very little data to look at. Jeff shared a few things investors are always looking for:

    Continue reading

    The Social Commerce Accelerator asked Jake to present to their members about how you can use data to optimize your customer experiences and become more effective at acquiring new customers. The highlights of the presentation were:

    1) Use different categories of data depending on what you are trying to achieve.

    Depending on what you want to measure and improve, the categories can include:

    • Revenue
    • Life-cycle analysis
    • Segmentation

    2) Extend data driven optimization of the business to your social media initiatives.

    Use data to understand which initiatives and channels are generating the best returns.

    Social media intersects with data in the areas of:

    • Acquisition of new customers
    • Retention of existing customers
    • Referrals to friends and followers – help your customers market for you

    3) More data is not always better. Focus on what is most effective.

    Use actionable metrics. If seeing the data then makes you take action, then it is worthwhile.

    • Know what you are trying to accomplish, and focus on the end goal.
    • Know whether the data you need is accessible.

    4) Where do companies see the biggest gains from data:

    Companies see the biggest gains from data in making better customer acquisition resource decisions. RJMetrics’ customers (more than 100 e-commerce sites) see 5x difference among best and worst channels. Know your lifetime value (LTV) by channel.

    5) Tactics you can use – targeting lapsed customers:

    • Remarketing: repeat purchased from existing customers are much more cost-effective than acquiring new customers.
    • Reactivation: send a targeted offer to your customers to keep them engaged.
    • Segmentation Marketing: tailor your offering so it is more interesting, and give different offers to different segments.

    Examples of how to segment your customer base:

    • Those who haven’t bought in the past 90 days
    • People referred by a friend
    • People who have referred friends
    • Customers with the highest order values

    Keep it simple to start. Split customers by multiple segments and keep testing.

    The webinar is posted here on inSparq and is part of the inSparq Social Commerce Accelerator (SCA). SCA is a 10 week program for established retailers, brands & e-commerce sites aimed to revive their social commerce strategy to drive sales life.