Long-anticipated, we are finally introducing A/B & Multivariate testing campaigns for Push, Email and SMS. Learn more about this oft-requested feature and what you can expect from this new functionality now available for all our plans.
Although this is a feature that a lot of you have been requesting for some time, it only made sense to introduce it after the recent launch of multiple new channels. As promised, as soon that happened, our engineering team quickly set off to design and deploy our new A/B & Multivariate testing feature.
What can you do?
There are 2 types of tests you can perform with this new feature. You can either split your audience by the number of variations you create (min. 2 and max. 10), or you can decide to create content variations, send them to a smaller sample group first and wait for results, after which we will automatically send the best performing content to the remainder of your audience.
You start by deciding which medium you want to use. You can choose between Push, Email or SMS:
You can then define the audience you want to reach in your test by using our familiar criteria selection interface. If you choose not to select any criteria, we will use your full audience:
You then choose the type of test you want to create, Split or Winner. Choosing Split will simply split your selected audience into equal parts according to the number of variations you create:
If instead, you choose Winner, you will have to provide a few more options:
First of all, you will have to allocate a sample group size. This means you can define the percentage of the selected audience we should use to perform the test. You should also provide the amount of time to wait to gather results and which metric you care about to determine your winner content ((Unique) Opens or (Unique) Replies).
After defining your test criteria, it is time to create the content you want to test. This is done by adding variations of your content:
To quickly add content to your test, we will allow you to automatically create several variations (at least 2, but up to 10) and re-use templates if needed:
After you have designed your content variations, you are ready to send your test:
As soon as you roll out your test into the wild, we will start collecting data:
If you've selected Winner as your type, you can also check the results of your test:
Ready to start testing?
As you can see, creating testing campaigns for your Push, Email and SMS content is this easy! Make better design decisions when rolling out your campaigns. As always, if you have any questions, we are available via our Support Channel to further guide you.