Model Behavior

Want to drive higher responses and find a wealth of viable prospects in your database?

You'll have to brave a bit of math. That is, you must segment your file and apply predictive models to it. And that's a complex task. Above all, you have to know what models can and can't do.

Let's say you've done your segmentation analysis. An assignment model has put your customers into buckets.

What's next? You've got to decide whether to build different models for each segment or one general model for all.

Take the case of a marketer that's ranked its house file using a sole model and received a 2% response to a mailing. If analysis has revealed truly different groups, separate models might be the right choice.

Unfortunately, there's no way of knowing without trying both approaches and seeing which one works best. Chances are the results would look similar to those in the chart above.

Assuming that modeling has been done on the entire file, the left side of the chart offers a hypothetical example of decile-by-decile response. The right side shows how the groups identified by the segmentation analysis responded.

As you can see, each of the first eight deciles performs better. The model has pushed the clunkers in all segments to the last two deciles. Cumulative results are the same only when the entire file is targeted.

WHAT DO THEY DO?

What does a predictive model predict? Most assign a probability of action to each prospect. This usually is based on logistic regression, although there's another tool — “regular” (or ordinary) least-square regression. The latter is used when the behavior to be predicted is continuous (for example, anticipated lifetime value), not categorical (targets will or won't respond).

But here's where the confusion creeps in. Categorical models don't predict the response rate, even though they're frequently referred to as predictive models.

Let's return to the mailing that pulled 2%. If I asked you to predict the response rate of a single individual and you knew little about that person, your answer would be 2%. Why? Because you have no reason to believe that this consumer is any better or worse than average.

But if you have lots of information about the prospect and a competent modeler armed with good tools, you can assign a higher or lower probability.

Yes, the average probability across the entire campaign will remain 2%.

Now consider this: The drop that pulled 2% was done in May, an average month for this marketer. But what if it was sent out in August, the mailer's best month?

You would re-score the file in July. Assuming that the makeup remained relatively constant from May to July (that is, the marketer hadn't done anything radical like use a new medium), the average prediction would be around 2%.

Shouldn't the scores be higher?

Take a very good prospect from the May model, one with an expected probability of 4% based on his individual model score. May's average response was 2%. This customer's index number would be 2 — he's twice as likely to respond as the average.

August is a better month, so we'll assume a 3% response. Again, the index number for this consumer and those who look like him is 2. And so, if the average response rate is 3%, their likelihood of responding is 6%.

DAVID SHEPARD is president of David Shepard Associates Inc., a direct marketing and database consulting firm in Dix Hills, NY.

Segmenting Works