A/B (or split) testing is one of the best ways to improve the success rate of push notification campaigns
A/B (or split) testing is one of the best ways to improve the success rate of marketing campaigns, landing pages, websites, social media posts, and now, your web push campaigns!
As you probably already know, A/B testing involves testing two versions of something in order to see which performs better. In this case, you are now able to test two versions of a web push campaign to see which performs better- which may be an improved click through rate or increased conversion rate. All you have to do is create two versions of a campaign and choose your segments. We will handle splitting your campaign 50/50 across the users you are sending to in order to run the test.
Setting up an A/B test is incredibly easy. Head to your dashboard and create a new campaign as you always would. You’ll notice a blue toggle at the top right of your screen that says ‘Enable A/B Testing.’ Turn this toggle on in order to set up two versions of your campaign.
Click on the button at the top right of the screen that says 'Enable A/B Testing'
Once 'Enable A/B Testing' is set to 'On', the content page of your notification will now give you the ability to click between 'Version A' or 'Version B'. Both versions are completely separate from one another.
You can fill in the content for 'Title', 'Body', 'Destination URL', 'Icon' and 'Image'. Fill in the required fields with your desired content on both Versions. Above is an example of 'Version A' and below is an example of 'Version B'.
Notice how the content is different for both versions.
Once you have filled in the required fields and are ready to send, just go through the rest of the tabs as normal (Choosing subscriber segment through recipient tab, enabling conversions etc.) and finally, schedule your notification campaign to go out.
You can also change the scheduled time of the A/B test as the modified item for the purposes of determining the best time to send notifications to your audience:
Your notification campaign will be sent out to the segment of subscribers that you selected and Version A and Version B of your campaign will be evenly split and distributed between these subscribers.
In the example above, I am subscribed to my website on both Chrome AND Opera. I received the notification campaign that I sent out on both browsers, however, they only received one notification and they both got a different version.
Viewing the Results
Once your notification campaign has been sent, you can view the results by going to your Manual Campaigns page and clicking 'View Results'. From here you can see how your campaign performed. To toggle between the results of Version A and Version B, just click on which version you would like to view at the top of the page.
Above is an example of the test campaign for Version A and Below is an example of Version B.
You also have the option of viewing your daily breakdown in two different ways, a line and bar chart. You can also download these results.
A/B Testing Icon
You also have the ability to easily see which campaigns you have set up an A/B test for on your campaign overview page. Make sure the ‘A/B test’ column is enabled (from the ‘Show/Hide' menu, make sure A/B Test is checked) and you will see an icon next to any campaign that has an A/B test running.
FAQ's
- Question: Can you split the weight of an A/B test differently than 50/50 (for example, 80/20)? Answer: Currently, the split will always be 50/50.
- Question: Can you do A/B/C testing? Answer: Not yet - this is something we are working on.
- Question: I made a campaign an A/B test in Draft/Scheduled, can I turn it off? Answer: Once you enable A/B testing on a campaign you cannot turn it off.
If you have any other questions please reach out to us at support@aimtell.com