A/B testing

A/B testing, or split testing is the worst kept secret of the big players in tech.  Basically you randomly serve your visitors one version or another of a page. You then define a metric for success, such as click throughs, subscriptions, or sales.  Then you test which one does better.  Switch to serving only that.  You can rinse and repeat this and evolve your marketing material into the most effective form.

There are lots of tools for split testing, but I’m using the landing page split testing built into mautic.  For beginners, especially those of us on a shoestring budget, it’s important to be aware of statistics.  We typically have low conversion rates, and so we need to be able to calculate when we have a significant result.

I’m recommending this tool: https://www.peakconversion.com/2012/02/ab-split-test-graphical-calculator/

We’ll look at the math behind it at some point, but basically the bayesian approach is more sound than most significance testing out there.  There are lots of approaches to calculating confidence intervals.  A lot of them break down pretty badly when you have low probability events.

Leave a Reply

Your email address will not be published. Required fields are marked *