Email testing: How and Why

What do we mean by ‘testing’? Assessing the quality of an email, be it in its concrete or more abstract elements e.g. the words/format or its ‘readability’, and seeking to improve them. Of course, improvement is determined by your own set of metrics e.g. the open rate and/or clickthrough rate. However the choice of metrics may also change depending on the type of test. It is crucial to set out before an email is sent, which metrics you would use to calculate the campaign’s success.

Email testing is still being neglected by many email marketers. According to Econsultancy’s ‘back to basics’ Email Marketing Census 2011, only a third of companies (32%) conduct regular testing. The reasons why include: tighter budgets reducing its priority, time constraints, and perceived complexity.

You can significantly improve your emails without falling into these traps. For example, a simple subject line test can alert you to which subject line achieves 10% more opens. For an email sent to 1,000 people, that’s an extra 100 viewers, which could potentially be an extra 100 conversions. In email marketing, incremental changes can have huge consequences.

So, how do you test? There are three options:

• A/B testing
  How? Email version A is deployed with a version B, usually with one contrasting variable e.g. subject line, both sent to a different small selection of the total recipient volume (commonly 5%). The winner of the two is determined, and sent as version C to the remaining majority (commonly 90%).

• Full Factorial testing
  How? A multivariate system, you test several different elements - or ‘factors’ - in an email. A considerable number of email versions are required, since every combination is tested. Success or failure of each email version is determined by their overall results.

• Fractional Factorial testing
  How? A multivariate system similar to Full Factorial, instead of the winning email version being dependent on overall results, the individual elements across tests are analysed. The final email is built to incorporate the consistently strongest individual elements.

Multivariate testing does require a thoroughness that takes time, including the creation of many email versions. It is therefore very difficult to advocate for short-term projects. This is where A/B testing is most useful.

Now, what do you test? Well, if it’s in the email, you can test it. Here is a list of the most essential elements to consider, and why you should seek to improve these:

1.) Subject line
Why? A good one can be the difference between the email being viewed or deleted.

2.) From line
Why? It’s vital in first communicating your business brand.

3.) Landing page optimisation
Why? A clickthrough to an unsuitable page could greatly undermine the user experience.

4.) Personalisation
Why? Now more frequent at a basic level, it’s important to reach out directly.

5.) Drop day
Why? A send on your optimal day will see the best chance of immediate response.

6.) Drop time
Why? Similarly, a send at your optimal time will have the same effect.

7.) Creative
Why? Minor improvements in design can dramatically affect engagement.

8.) Content
Why? Exactly what is communicated is obviously paramount and must be relevant.

9.) Link placement
Why? The positioning of key links and buttons strongly influences clickthroughs.

10.) Link design
Why? If you get it right, you will entice interaction which may convert.

11.) Link frequency
Why? Too many or too little can cause confusion, so this should be balanced.

12.) Social media integration
Why? The right exposure may lead to greater email readership.