Even the greenest of marketing rookies have heard the mantra “Test, test, and test again” a few dozen times. But when it comes to e-mail marketing, there are so many variables to test, the prospect can be overwhelming. We asked Adam Q. Holden-Bache, CEO/managing director of internet marketing agency Mass Transmit, for some guidance.
E-mail Essentials: There are so many elements of an e-mail marketing program that lend themselves to testing. How does a marketer determine which ones are the most important ones to test?
Holden-Bache: You need to start by understanding the goals of your campaign. For most marketers it’s probably a purchase or conversion of some sort—download, opt-in, registration, etc. For others, it may be generating site traffic (ad-supported sites) or providing information (newsletters or b-to-b communications).
Once you know that, you can look at what you need to optimize. For conversions, you need the open and click. For newsletters, you may just be looking at open rates. In either case, you'll need a strong subject line that encourages an open, and then you may also need to test creative, copy, call-to-action elements, etc.
So in the end, subject lines matter to everyone, and then beyond that it depends on your goals.
E-mail Essentials: How many elements should a marketer test per campaign?
Holden-Bache: Provided you have a decent list size, you can do as many as you're comfortable with. However, I find that the overwhelming majority of the time, marketers test one element with two or more options. If you send a lot of campaigns, it makes the testing process more manageable, and it’s easier to quickly analyze the results.
Multivariate testing can be useful when you know you need to test multiple items and when you have a large enough list to allow for sufficient sample sizes. If you go this route, make sure the time and effort required to organize and develop the content and creative doesn't outweigh the value of the information you receive from the test results.
E-mail Essentials: Many smaller organizations say that they don’t have the resources to test their e-mail. Is it possible to test on a tight budget, and if so, how?
Holden-Bache: At this point, any small business serious about e-mail marketing should be using an ESP [email service provider]. The barrier to entry is very low, and the return on investment from using an ESP should cover the cost of the service. You have to know you are reaching the inbox, and you have to be able to track your campaign metrics.
Nearly all ESPs will provide the ability to test your campaigns. This process may be different between services but should be achieved with minimal effort. So once you have your ESP chosen, just make sure you know how to test within that system. After getting over a small learning curve, [testing] should have a minimal impact on time and resources.
E-mail Essentials: Are there best practices for testing?
Holden-Bache: I feel the value of a control group is frequently overlooked by marketers who just test A vs. B. Make sure you're able to determine if your test results provide in increase or decrease against the control-group content.
E-mail Essentials: Do you have any anecdotes to share regarding tests that paid off big or resulted in unexpected results, or any cautionary tales of tests gone wrong?
Holden-Bache: I've seen instances of marketers testing subject lines once, seeing small lifts, and then sending campaigns with a similar subject line at a later date and then experiencing results lower than pretesting. What they ended up realizing was that the time of day their campaigns were delivered influenced the results more than the subject line. So in the end the testing highlighted this, but originally the client didn't factor time of day as an element to test.
I've also seen lately that making a major shift in a campaign can reactivate interest. One client went from using a long subject line to using a two-word subject line and saw their open rates double. Another b-to-b client added a video thumbnail into their e-mail creative and saw a 300% increase in clicks. When testing, try thinking about standing out more from the common campaign, and you're likely to be rewarded.
