What is an A/B test and how do you deploy it effectively?

Online marketing is constantly about measuring, monitoring and changing campaigns, ads and pages. Nevertheless, it sometimes remains difficult to measure the effect of a particular change in an ad or page, and it is not always clear how to optimise. So how does it become clear in one respect, for example, which changes to an ad achieve an increase in conversion rate? The A/B test offers important insights into this. After all, as the name suggests, this is a way of testing which variant (A or B) of a page or ad proves effective for achieving your goals. This blog therefore explains what an A/B test is, how you design it and what you use the split test for.

How an A/B test works

Among other things, online marketing is concerned with increasing click-through rates and conversions. For this reason, various means are used to achieve this goal, including A/B testing. Take, for example, an ad for a specific product, which you want to promote online. To find out which design and text are most effective with regard to conversion optimisation, two variants of the ad are put online. Ad A serves as the control variant, while ad B has undergone all kinds of modifications. The two versions are then alternately shown to potential customers so that the variants are viewed by the same number of people. Meanwhile, the click-through rate and conversion rate are measured. Indeed, this data will show which ad is best to deploy or which adjustments need to be made for the ad to work effectively. Not only is A/B testing applied to ad optimisation, but you can also use this form of experimentation to improve website elements or adjust marketing via e-mail.

Designing a split test

While it seems obvious that the two variants for the test should not be the same, this is not the only thing you should pay attention to when designing a split test. After all, where is the priority for experimentation, what are your KPIs and what sample size do you use for the test? Below are the most important four tips for designing an A/B test.

1. Don't change more than one aspect

A/B testing is thus a form of split testing, as you are comparing two seemingly identical pages or ads. To measure the number of conversions therefore, version B must differ from version A. You can achieve this by duplicating the page or ad and making a change to version B. However, when doing this, make sure you only change one component so that it is immediately visible whether and which change is effective. After all, if you change more than one aspect, such as price, title and button colour, it is no longer clear which change caused a change in the number of conversions.

2. Where is the priority?

You don't just run an A/B test because you want to find out whether your page or ad is achieving its goal. Does a change result in a higher click-through rate; does a change lead to more interaction or do you realise more conversions with a change in an ad? So make sure the goal of a page or ad is clear, so you can better determine where the priority lies when running an A/B test and which adjustment to test.

3. Set clear KPIs when A/B testing

Not only is prioritising an important part of making a split test effective, but also formatting KPIs is crucial. By setting up KPIs, you will avoid endless testing. After all, if it is not clear why testing is needed and what outcome is expected, the test result will give you little. After all, which test will come out of the results better if you don't know exactly what to look out for? Predefined KPIs will therefore help you considerably here.

4. Test on a representative number of visitors

While there is no guideline for the number of visitors required to effectively run an A/B test, it is important to establish a representative sample size. After all, with ten visitors, you will probably not be able to make many effective adjustments to a page or ad.

In addition, you may also fail to achieve the desired results due to a limited sample. This can be the case, for instance, when the conversion rate should have been higher than it actually is due to a disappointing sample size. Do not adjust the final results in that case, however, because in that case the test was successful, but look closely at what the result means and what should have been done differently.

A/B testing with Google Optimize

Once you have realised the two different versions of an ad or page, it is time to initiate test with a tool like Google Optimize. This is a free tool, which needs to be linked to Google Analytics and to your own website. By then importing the two variants of your page into this, Google alternately directs visitors to one of the two pages. This is done by means of a code, which must be placed on the page to trigger this guidance process. Visitors themselves do not realise that they are being led to different versions of the page, but for you this will become visible in Google Optimize. This is because this is where the results of the A/B test are displayed. Other tools you could use for an A/B test are ABtasty, VWO.com and Optimizely.

The A/B test, or split test, provides an opportunity to check which changes to an ad or page prove effective. The goal is often to achieve more conversions with the change, but the online marketer may also want to achieve more interaction with potential customers. Here, however, it is important that you do not make more than one change and test it on a representative number of visitors. By then linking the two variants to Google Optimize, you can monitor the results and draw a conclusion from this.