How to do automatic A/B testing of e-mails?

[JanMatthes]

Since 1605 release SAP Hybris Marketing Cloud not only supports manual A/B testing (aka multivariate testing) but also automatically sending multiple mail variants to a small test group (e.g. 5%) before sending out automatically the most successful mail to the remaining recipients.

In detail you need to do the following to create a campaign with automatic A/B testing:

1. Create an e-mail campaign with a target group


2. Select A/B testing as initial action


3. Select at least two mail variants you want to be testing


4. Define the size of your test group in percent


5. Select the winner determination metric e.g. highest rate of opened or clicked messages


6. Enter the test period e.g. 1 day


7. Additionally you can define any trigger to start follow-up actions which shall happen (e.g. mail, call...)

Below you find a demo on how this works in 1702:



Additionally you can also use the random split and control group capabilities to do a manual A/B testing.

You can do this as follows:

1. Use segmentation to create a target group and a separate  control group using the random split features


2. Assign the control group to the target group


3. Create a campaign for the target group or assign it to an existing one


4. Copy the campaign exchange control and target group in the new campaign


5. Execute both campaigns


6. After the test period is over create a new campaign with the most successful variant and assign a third target group which excludes the participants of the test campaigns.