English
Support

07.03.2022

· Whitepaper

Successful with A/B testing

Table of contents

xx chapter(s)

Easy way to increase the performance of your newsletters

In order to easily maximize opening and click rates of your email marketing newsletters, simply choose the right subject line or sender address. However, is it as easy as desired? How can you find the right subject line or sender? Based on the numerous requirements made to a subject line (the shorter the better, inviting, curious, personally appealing, etc.) you can use the practical eworx Marketing Suite feature to test your variant diversity with an A/B test. You simply identify two possible subjects and mailworx will decide for you which one is more successful.

By using the A/B split test you can gain up to 2.5 times higher click rate and a 50% higher opening rate. That’s what measurements have shown over millions of emails – try it for your own and convince yourself.

If your question hasn’t been answered in this whitepaper, please let us know. We’d gladly catch up on this.

What is an A/B split test and how does it work?

Subject and sender are two of the most important influencing factors of your email campaigns and both can decide over whether your newsletter will be opened or not. These two parameters can be optimized via A/B testing. This checks which variant of subject or sender leads to the highest opening or rather click rates.

Thereby, the target group of the sending is devided into three segments: two little testing groups and one group which contains the remaining subscribers.

The recipients of the test group get – depending on the group assignment – either subject line A or subject line B (or the variants of the sender). After the test period, the determined winning subject (or sender) will be used for sending to the remaining subscribers.

Which variants are there for an A/B test?

In the eworx Marketing Suite you can use A/B spli tests either for the sender or for the subject line. For doing this you simply have to activate the check box ‚Perform A/B Split Test‘ in step 2 ‚Sender‘ or step 3 ‚Subject‘. Another input field will open in which you can enter the second variant.

Optimize sender

You may think that this information is already given by your organisation. Nevertheless, think about how you could optimze the sender. If it contains the name of a well-known contact person – which you can personalize to the respective recipient by the way – the interest in your email will increase enormously.

A/B testing email sender

Optimize subject

The subject line of a newsletter should whet the appetite for more! It decisively influences the opening rate of your email campaign. An A/B split tests holds a great opportunity to optimize your results with a minimal effort.

What works out best: short or long? Concrete (‚Download the new study‘) or exciting (‚The best day to send emails‘)? With or without emojis? As you can see, there are a lot of variants that you can test. Nevertheless we want to give you the following advices:

 

  1. Always performe an A/B split test for the subject line.
    Always! Except it really makes no sense.
  2. Put important key words to the beginning of the subject line.
  3. Do not forget the preheader.
    It is lengthening the subject but is not shown by all email programs. This is why you should put the most important content in to the subject line.
A/B testing subject line

What should I watch out for regarding the test group?

In most cases the useful size of both of the testing groups for subject A or rather subject B is between 10 % and 30 % of all recipients of the respective target group. The recipients of this groups are chosen automatically as well as randomly 
by the eworx Marketing Suite. They don’t know anything about a test, that’s why they behave like they normally would do, too.

If the test would emerge as not significant afterwards, it can be that the testing group was too small. In this case we recommend to choose a larger one the next time.

How long should the test take?

Make sure to give your recipients enough time to react to your sending. A too short period of time between the dispatch to the test groups and the remaining subscribers can lead to a not representative test result. Taking sufficient big distributors, the results will still be significant, even though the duration of the test was short (e.g. 1 hour). We recommend to take at least one day for the test.

When should the test start?

When performing an A/B split test with mailworx, you also have the opportunity to choose the time for dispatching your email campaign. The time then means the start of the A/B test. Consider that the time of the dispatch to the test group as well as the time of the dispatch to the remaining recipients should be as good as possible.

A/B test settings

Was the A/B test significant?

During the duration of the test, you see the current status of the A/B test in the overview of the eworx Marketing Suite live statistics.

The results are shown as easy and transparent as possible, because we want to give you the perfect tools in order to lead your email marketing campaign to success.

The success of the A/B split test mainly depends on three factors:

  • Is the size of the test group sufficient?
  • Is the testing criterion (subject/sender) really meaningfully different enough?
  • Is the waiting time between test and real dispatch sufficient?

You will find out in the detailed view of the test in your statistics. whether the A/B split test of your sending lead to a statistical sufficient result.

Result

What should you do if the test was not successful? You have different opportunities: Raise the duration of the test, enlarge your test group in the future or vary the formulations of your subject lines more significant.