Why One Model Is Probably Not Enough

IN THE PAST we have considered whether or not different segments of a customer file will perform differently depending on their relationship to well-known predictor variables like recency, frequency, monetary value and product purchase data.

While all customer segments might show a negative relationship to recency (the longer the time since the last purchase, the lower the probability of response to a follow-up offer) and a positive relationship to frequency and some measure of monetary value, such as sales to date, the equation that describes this relationship probably isn’t the same for each customer segment.

A solution we have offered was to first identify segments in the database using CHAID, cross tabs or commonsense observation. The next step involved creating different models for each segment, or – through the use of dummy and interaction variables – developing a single equation that would capture the effect one is attempting to measure.

Now we’re more convinced than ever that this approach has considerable merit. In fact, we think it’s necessary to expand the notion beyond the relatively simple breaking apart of the key performance variables. We believe there’s a need to identify variables that may work differently with different segments, but also might be relevant to certain segments and not to others.

Let me suggest a couple of situations where this approach can be relevant.

Customer Tenure A good place to begin a search for segments that will show differences in their relationships to behavioral and demographic variables is tenure, or the length of time an individual has been a company’s customer. Essentially, longtime customers will respond differently to a firm’s offers than those who’ve been customers only a moderate amount of time.

For example, demographic variables will probably not be important in predicting the behavior of customers with relatively long tenure, but may be very significant in forecasting how new customers will act. Similarly, product purchase data will be important for long-term customers, but not critical for newer ones. Or it may be that this transaction data is essential for all customers, though different people won’t necessarily buy the same items.

Additional Model Sources Other variables warranting their own models might include important variables such as age or income, or segments based on a more complete demographic breakdown. Or, because we know how important the original source code and offer are in predicting future behavior, different combinations of major media and offers could provide a basis for different models.

The key idea here is that while there are some variables that are important in predicting the behavior of all conceivable segments, there are other variables that might apply only to particular segments of a company’s customer database.

Correlation tables, which measure the linear strength of a variable across all customers, can easily miss identifying a nonlinear variable or one that applies only to a single market segment. CHAID might provide some clues as to different market segments. However, the best way to use CHAID in this hunt for segments is to force the first one or two breaks using the variables one thinks might make a difference, and look at the variables that appear under each branch of the tree. If the variables in each tree are different, or if they are the same but show very different shapes, then separate models or the extensive use of interaction variables will be necessary.

In theory, as long as all potential interactions are captured, there’s no need for separate models. But the single-model solution can get ugly pretty fast.

An Exercise for Disbelievers If there’s any doubt about the power of this approach, try this exercise using any spreadsheet program.

Create 100 records that behave according to the following formula: Y = a + b1 superscript *X1 + b2 superscript *X2 + b3 superscript *X3. Make up a set of coefficients and use a random number generator to calculate the values of the X’s for each of the 100 records.

Then do the same thing for another 100 records using this equation: Y = a +b1 superscript *X1 + b2 superscript *X4 + b3 superscript *X5. Use different values for the “a’s” and the “b’s” and a different set of random numbers for the values of the X’s.

Now put the 200 records together and run the spreadsheet’s regression program. The result will be a fivevariable model (X1 is in both equations) with a very low Rsquared.

Next, create a dummy variable to identify the members of each set and make up five new variables that represent the interaction of the original variables with each segment. This will result in an 11variable model with an Rsquared of 100% as it should be.