Loose Cannon: Which Version of This Column Are You Reading?

If Andi Emerson has suddenly developed psychic powers, we are all in a lot of trouble.

Emerson runs the John Caples International Awards, which reward creativity in direct marketing campaigns. During a recent chat, she railed against the direct mail industry’s lack of creative testing before launching new efforts.

A few days after our conversation, MarketingExperiments.com began flogging a paper decrying the industry’s underuse of split tests. Coincidence or premonition?

When Emerson initially posited that the industry was moving away from testing, I skeptical. My first job after college was as a circulation assistant in a magazine company. The firm’s very talented circulation director conducted mailings – this was in the days before e-mail had taken root – of upward of 20 panels. Tests ranged from price to featured artwork to list source to package design, and doubtless a few I’ve forgotten.

So my orientation was that marketers tested like demons. I mentioned this to Emerson, who agreed that this was once the case, but informed me “you’re old school.” Which is a hell of a designation for someone not yet 40.

She claimed that instead of testing, today’s marketers demand the single best effort from their creative shops. These are then used as the creative for mail campaigns, without any variations. What “testing” there is comes from focus groups, or perhaps the whim of a C-level executive.

In an unsettling echo of Emerson’s sentiments, MarketingExperiments.com’s paper urged marketers to explore A/B split tests. But the paper focused on Web sites and e-mail efforts: Traditional mail, which is still the industry’s workhorse, wasn’t mentioned.

Direct marketing is supposed to be the scientific branch of advertising, the one built on testing and modifying efforts. But if testing really is on the wane, then marketers are careening madly from one campaign to another, without regard for having a long-standing control package – and a test package that could ultimately supplant it.

Emerson said that publishing is one of the few industries that continues to embrace widespread testing. I’d like to hear from non-publishing direct mailers regarding whether they are testing extensively or not, and why. Could it really be that testing’s day has passed?

As for warding off Emerson’s magic powers, creative types might consider entering their efforts – tested or not – in the current Caples’ competition. As an additional incentive, any entry received by the 50th anniversary of the debut of CBS-TV’s “Gunsmoke” will receive an early entry discount. Entry forms are available at http://www.caples.org/.

The rest of us might consider burnt offerings.

To respond to the opinions in this column, please contact e-mail: [email protected]