Modeling Toward True Optimization
The new marketing technology called finite state machines can provide a flexible way to manage complex, longitudinal customer management programs. Using state machines, marketing programs can be modeled using a combination of states, transition rules that move customers between states, and various actions, such as sending a direct mail piece.
As powerful as they are, state machines have two limitations. First, they require us to determine and implement all possible transition rules, which for large marketing programs can be time-consuming. Second, they do not truly permit us to optimize marketing around particular strategic goals and take into account real-world constraints. Let’s consider both of these problems.
If you meet someone who claims to be optimizing all of their customer communications you can draw one of two conclusions. They’re either misusing the term optimization, or, they have several of the world’s most powerful supercomputers working around the clock.
If by optimization we mean tailoring timing, frequency, channel and other aspects of each promotion to each customer, then we create an enormous problem current computing power can’t solve. For example, grab a calculator and start multiplying the following figures: 10 million customers times 52 weekly campaigns times 3 versions for each campaign times 4 communication channels times any other variation you might introduce.
The number of possible combinations rapidly reaches gigantic proportions. And, you still need to find the right one for each customer, never mind the economic obstacles to produce all these variations cost-effectively.
Fortunately, we can extend the concept of state machines with a technique that is used for optimizing processes in many other industries – a Markov Decision Process (MDP). A decision process also has a series of states and transitions between states. But there’s a key difference: where we had to determine and specify each transition in our finite state machine, here the transitions are a set of probabilities.
For example, if on average 60% of our customers purchase only once, then we can create two states: first-time buyer and second-time buyer. The transition probability between first-time and second-time is 60% and the reverse transition has a 0% probability.
In a typical MDP, we might create between 50 and 100 states, although it is possible to use several thousand. Because we use these states in place of the much larger number of customers, we are able to reduce the size of our optimization problem by several orders of magnitude.
In order to optimize our customer management programs, we need to add two more elements to our MDP.
The first is a set of actions for each state. This is relatively simple, since the campaigns and programs we execute are the “actions” we have taken over time
The final element is a reward system. Again, this can be constructed relatively simply from our marketing data. The rewards can be our financial data from past responses and other customer behavior. If we took a particular action – such as a mailing piece – our reward is the amount of the purchase less our promotional and fulfillment costs for those who responded (let’s say $25 on average) and the cost of the mailing for those who did not purchase (let’s say $-1).
We can, of course, have more sophisticated reward structures. For example, we might want to take the profit of the purchase, since some customers might respond by buying higher margin goods. We can also use a global reward structure, so that the lifetime value can be used (in which case the customer’s retention may contribute to the increase in value).
Once we have these four elements – our states, our transition probability matrix, actions, and rewards – the entire process can be solved for an optimum outcome. That is, the MDP will tell us which treatment at each state yields the maximum reward. This also means that if your company decides to change strategy, we can adjust the MDP with a new reward structure, e.g., maximize acquisition of new customers, and rapidly implement the new strategy across the portfolio.
MDPs help solve a couple of additional problems that traditional campaign management promises, but falls short of delivering.
The first is marketing automation. In campaign management, the automation is fairly trivial: we can set up campaigns and re-use them. We also showed that finite state machines offer even better automation, but like traditional campaign managers, need strategies formulated through outside analysis.
With MDPs, we can do something more radical and important: if we know the best action to take, we can automatically take that action each time a customer enters that state. Naturally, we want to continue to perform testing of new concepts, since nothing is truly static, but we can now do our testing within a customer management framework, rather than as a set of isolated events.
This is particularly important because certain promotions can decrease customer value. Let’s revisit our first-time and second-time buyers. If we wanted to increase our new customer acquisition, we could choose a strategy of offering a discount offer to new customers, for example, $5 off. The promotion is supposed to significantly increase acquisition of new customers. Naturally, our profit falls to $20 because of the additional promotional expense assuming that the additional customers purchase the same amount.
In many cases, we would worry about two negative consequences for such a strategy: the average purchase amount might be lower; and/or the repurchase rate is lower. Or, these two factors may not be severe, but we may see higher attrition rates in six months’ time. These effects, as well as others, will be apparent within the MDP once customers begin to migrate through the system. We can evaluate the effects of the new strategy from a wider perspective.
Finally, our two customer management approaches – finite state machines and MDPs – can also be used together. For example, we can set up the MDP to provide the overall strategic framework and use our rules-based state machines to implement the tactical marketing programs. We then have the best of both approaches: the analytically driven power of a Markov process coupled with the execution capabilities of state machines.
Whether used separately or in combination, these two technologies are finally enabling marketers to realize the full promise of data-driven marketing.
David King is chief executive officer of New York-based marketing solutions provider Fulcrum.