Get in Control

One of the most prominent features of the database marketing discipline is the ability to experiment and evaluate. Testing involves the manipulation of one or more controllable factors to determine their impact on various outcomes. It’s a process in which results are measured in an environment where controls serve to identify the causes of behavioral differences among respondents.

Valid experiments are characterized by:

• Random assignment of data subjects to groups so distinctions occur by chance alone (the test group).

• The presence of an identical second group where the experiment isn’t conducted (the control group).

Over the years, many novice marketers have dispensed with control groups. One manager explained that the program he designed resulted in more than $2 million of additional sales, a robust figure. He showed me the tidy spreadsheet he’d prepared with the relevant figures. Indeed, it did appear the group he had attempted to influence did buy more.

However, he failed to consider what would’ve happened if he hadn’t done anything.

The importance of controls can be demonstrated by a non-marketing-related illustration, perhaps one of the first recorded instances of control experimentation. Three hundred-fifty years ago, a mathematician by the name of Blaise Pascal hypothesized that atmospheric pressure declined with increasing altitude. Pascal initially took a barometric reading in his village. Some time later, after he had climbed a 3,000-foot mountain, he took another reading and discovered that this top-of-the-mountain measurement showed a considerable drop in atmospheric pressure. Was Pascal’s hypothesis correct? (What Pascal didn’t know, however, was whether the atmospheric pressure had become lower at the bottom as well as the top of the mountain during the time that it took to climb to the higher elevation.)

Pascal modified his experiment so that a reading was made on the top and the bottom of the mountain simultaneously. This controlled experiment showed that although both barometers had lower readings at the end of the time period, the measurement on top of the mountain had a significantly lower reading than the one at the bottom.

Most seasoned marketers now incorporate control groups in their testing. Typically, a sub-population is selected. This test group is presented with certain stimuli in order to elicit a response of some sort. Simultaneously, another segment that is a