Scaled-Down Testing: Tips for DMers with Small Target Audiences

Business-to-business direct marketers find it hard to test, and for a good reason: B-to-B campaigns are complicated, with multiple touches and long sales cycles. Simple split testing can’t provide the most important information, like which touches are essential and which are fluff.

Furthermore, B-to-B campaigns target multiple contacts in single company accounts, so testing can raise thorny policy issues and upset customers — and the sales force, too. It’s tough when a rep comes into your office screaming that your test offer has created confusion in her sales territory.

But the target’s size is the major reason why this kind of testing causes problems. When your prospective audience is smaller than 10,000 accounts — and that’s not unusual — test-and-rollout strategies are likely to be pointless.

So how do you use the power of testing when your targets are too small to support traditional split methods? Let’s look at four work-arounds.

1. Eliminate outliers

   When you can’t test to a sample and roll out to the whole, you’re essentially campaigning to the whole every time. So what can you do to improve the odds of going out to the target with your best shot? Use pre-testing to eliminate the least-powerful offers and creative approaches.

   Some of the pre-testing techniques that work for B-to-B marketers include:

   • Using e-mail and the telephone to test offers and creative, then applying those results to other media, like mail.

   • Focus groups, whose opinions can eliminate the least-promising ideas while providing deeper insight into respondents’ motivations.

   • Online surveys using low-cost tools like Zoomerang and SurveyMonkey.

2. Take manageable risks

   When you can’t test with statistically projectable samples to provide reliable campaign predictions, one alternative is to adjust expectations, accept an increased amount of risk, and seek ways to mitigate that risk.

   Here are some approaches that can work:

   • Lower the confidence level required for your tests

     Loyalty Builders’ CEO Mark Klein points out that reducing the expected confidence level from DMers’ traditional 90% to about 60% can make dramatically smaller test quantities productive. “This kind of testing, while more prone to error, still improves the chances of hitting a winner with a small target universe, which is the whole point,” he says.

   • Campaign sequentially

     When you’re running campaigns to the same audience over time, the results will mirror what you’d get from a split test. There’s some risk since you’re introducing time as a variable. But, says Andrew Drefahl, director of customer insight and technology at Hunter Business Group, “Rapid iteration becomes a learning process. You learn by failure, developing rules of thumb that you can apply to other campaign situations.”

   • Make reasonable assumptions

     For example, take what works in one medium, like the telephone, and assume that it also will work in the mail.

3. Get the most out of what you have

   Even if you don’t have a large enough target, some tools can take advantage of what little you have.

   • Power Test (pioneered by The Hacker Group) provides a methodology for generating a direct mail control package from scratch using just 50,000 names. Power Test eliminates the need for each test cell to be statistically projectable on its own, but lets users read aggregate results across several lists. This way, with only two test runs of 25,000 each, a control package can be culled from two offers, three creative approaches and five or more test lists. A very neat technique indeed.

   • Enterprisewide control groups. As advocated by Richard N. Tooker, author of “The Business of Database Marketing,” a company can set up a single control group for multiple campaigns over time by establishing a control consisting of a small portion of the customer base. This corporate control group, which receives no marketing communications at all, assesses a DM program’s overall value. But it also may be used as a benchmark for particular campaigns.

   • Steal from your competitors. Jim Obermayer, founder of the Sales Lead Management Association, suggests that B-to-B marketers take a page from DM creative directors, who’ve learned over decades the value of the “swipe file,” which stores competitive control ads based on how often they see the piece in the market. “If your competition has figured out what’s working, why not reap some benefit from the other guy’s investment,” he says.

   • Offer the hottest premium. Businesspeople respond well to free stuff, and new trinkets are coming on the market all the time. If you build a strong relationship with an ad specialties vendor, you can be first in line to use some hot new item. Obermayer says, “Remember those fans a few years back, where the brand name was spelled out in the air via electrodes? Everyone had to have one. There’s no staying power in this strategy, but you’ll certainly be giving the market your best shot.”

   • Use your house file as a testing medium. As Lett Direct president Stephen R. Lett notes, “The higher response rate from your house file enables reliable testing with a smaller universe. You can then take these results and apply them to your prospecting campaigns. It’s not perfect, but it’s better than no testing at all.”

   • Run an A/B split test in print. Trade publications often have a circulation large enough to permit valid split testing — offer, headline, or copy, for example. Results can then be applied to other media.

   • Swing for the fences. You don’t have the luxury of testing nuances with small universes. So test the changes that will have the most impact, advises veteran copywriter Lee Marc Stein.

4. Leverage the Internet

   Online media offer B-to-B marketers several brilliant new options for improving campaign results.

   • Pre-testing using e-mail and banner ads

     A/B splits in Internet media do little to enhance the size of your universe, but they can provide speed and convenience, making testing cheap and easy. So take advantage of split testing in headlines, offers, subject lines and other variables that can then be applied to campaigns in other media.

   • Web-site-based multivariate optimization

     Thanks to free tools like the Website Optimizer in Google Analytics, marketers can set up multivariate tests on critical Web pages like registration forms, order forms, and landing pages — anywhere a response is requested.

RUTH P. STEVENS ([email protected]) consults on customer acquisition and retention, and teaches marketing to graduate students at Columbia Business School. She is the author of “The DMA Lead Generation Handbook” and “Trade Show and Event Marketing.”