E-mail Industry Lacks Consensus on Most Basic Definitions: Study

Ten years after e-mail began to emerge as a serious commercial marketing channel, the e-mail marketing industry still has no consensus on the definitions of even the most basic metrics, according to a survey due out today.

Even the most fundamental metric of them all, the delivery rate, has no standard definition, according to a survey conducted by the deliverability roundtable of the Email Experience Council, a for-profit organization aimed at promoting standardization in the e-mail industry.

For example, 79% of e-mail service providers surveyed defined “delivered” by deducting all failures from total mailed, while 21% calculated it by deducting hard bounces — where the address no longer exists — from total mailed.

E-mail marketers were in even further discord on the “delivered” metric as 63% defined delivered as total failures subtracted from total mailed, 11% defined it as simply total mailed and 10% defined “delivered” as only those e-mails that made it into the recipients’ inboxes versus their spam folders.

“That was the scariest finding of all,” said Deirdre Baird, roundtable chair and chief executive of deliverability consultancy Pivotal Veracity. “There’s a wide variation in how delivered is calculated and yet it’s central to everything we do in this business.”

Just as concerning was the finding that there is also no consensus on how to calculate the number of messages that were mailed, according to Dave Lewis, project co-chair and vice president, market development, StrongMail.

“Even on ‘how many did you mail?’ there’s no consensus,” he said. “Are you talking about the size of your list and how many attempts you made? Or are you talking about how many times you tried to mail? If you’ve got a million-record list, are you mailing a million? Or are you mailing 3 million because you tried each one three times?”

A lack of consensus on such fundamental metrics has a seriously damaging ripple effect across the industry, said Baird and Lewis. “It has a cascade effect,” said Lewis.

For one thing, it makes measuring the rest of any e-mail program accurately impossible. “You can’t manage what you can’t measure,” said Lewis.

Also, clients sometimes switch or consider switching e-mail service providers because the marketer has been led to believe one ESP has a better deliverability track record than another when the one with the seemingly better rate is really just measuring it differently.

Baird and Lewis added that that lack of consensus on basic metric definitions makes it practically impossible to compare results from different vendors. “You’re comparing apples to oranges,” said Baird.