From Good to Great: Best Practices in Analytics

What does it take to produce great analytics? Some would say it’s all about the right data, tools and techniques. But let’s assume that you have all that. What do you do then?

The answer is to follow best practices—in people and processes.

First, let’s define “analytics,” a term with a variety of different interpretations. It generally covers:

Predictive modeling: The application of statistical modeling techniques to predict prospect or customer behavior.

Data-driven segmentation: The use of clustering or other techniques to organize prospects or customers into meaningful groupings for business management.

Strategic data analysis: Monitoring and analysis of trends observed in data to produce conclusions with significant business meaning and strategic impact.

What are the best practices? In my opinion there are three key conditions that have to be satisfied in order to consistently performing at a high level:

Alignment

This starts with the need to fully understand the business issue or opportunity at hand and then selecting the data and techniques most appropriate to ensure a continuous focus on producing what your stakeholders need to make real-world business decisions. Alignment also requires an analytics process, which enables close and continuous contact to be maintained with business sponsors so that course corrections can be made and new information assimilated throughout. Like the best system development projects, good analytics does not focus on documenting business requirements only to submerge them in an opaque process. Instead, analytics requires continual input and validation from those who asked for the work to be done in the first place.

Business Impact

Analytics is a means to an end and not an end in itself. Too often, it produces work which looks and feels technical to business people who therefore struggle to meaningfully link it to the challenges they face in the real world. Knowing that “10-fold validation was used to produce a good model” is an irrelevant detail, which should be consigned to an appendix at best. In good analytics environments, the technical quality and robustness of the work should be a given. It is the applications and the impact that are much more relevant for business people. This means stakeholders must be guided through often detailed deliverables to help them focus on those results relevant to the financial bottom-line.

Ease of Application

For analytics to have maximum impact, analytics personnel must go beyond simply producing a model or analysis. They have to work closely with sponsors to ensure that their work is correctly and easily interpreted and that the business case for using analytic findings is thoroughly documented and expressed in terms of meaningful business metrics. That enables informed decisions to be made about when, how, and to what extent the results should be employed in business planning.

It also requires a clear implementation plan, which clearly states what business activities need to happen—or be stopped or changed—in order to take maximum advantage of the analytics output.

So what are the best practices? Here they are.

Analytics Process and Methodology

It may sound trite, but a well laid out and consistently followed process for analytics can go a long way towards ensuring that good work is consistently produced. It will also help your business partners monitor progress and understand when and how they need to participate. Unlike other areas, there are very few “industry-standard” methodologies or processes that can be used as a template. One which I have found useful is what is commonly referred to as “CRISP”—an approach to analytics that has its origins in work sponsored by SPSS amongst others. The attached chart gives an overview of the key steps in the process. To see it, click here. http://directmag.com/table.pdf”

Having the methodology is one thing—consistently applying it and knowing where the potential pitfalls and areas of emphasis should be is quite another. It is sometimes said that the bulk of the work required to produce good analytics lies in the business understanding, data understanding and data preparation areas. In my experience, that is absolutely true. Selecting the right modeling technique and building candidate models or analyses is where many people quickly gravitate because these are the parts of the process that are most exciting or look most like “data mining.” That is a mistake, however. Without a clear understanding of the business problem—not just as initially articulated or documented by your business partner, but appropriately probed and challenged—you risk producing work that will ultimately sit on the shelf, or worse—be misinterpreted and misapplied, causing much more harm than good.

Data Preparation

Some of the least glamorous (but most important) work in the process quickly follows: Auditing and appropriately transforming the data so that it is as complete as possible and ready for application of the appropriate techniques. All other things being equal, finding the right data and appropriately preparing it is ultimately more important that your selection of analytic technique. The latter is important, of course, but too often people get caught up in the technical merits of one approach versus another and don’t spend enough time on the fundamentals of data selection and preparation.

Modeling and Evaluation

Now the fun begins: Applying the appropriate analytical approach. I recommend that you settle on a few techniques that consistently produce reliable results as opposed to continually worrying about whether one technique would produce marginally better results than another. The most obvious example is logistic regression—the tried and trusted technique which has proven to be a mainstay in marketing and other areas. Some purists would argue there are more “sophisticated” techniques, and there definitely are. What you have to remember as an analytics person, however is that you work in the real business world where time is money and the academic value of one approach versus another is the last thing on a business person’s mind. Much more important is your ability to consistently produce reliable and actionable results.

Deployment

Towards the tail end of the process, the focus should be on communication, deployment and, if appropriate, “operationalization” of the output if it has repeated application potential. The communication step is where many great analytics projects fall down. I have lost track of the number of times a great piece of modeling work is met with blank stares, or worse—extreme disappointment— because the typical technical deliverables of a model (gains charts, decile analyses. etc). do not always easily translate into the specific business actions that the work should be suggesting. Too many modeling presentation decks are focused mainly on the methodology used to build and validate the model with only a passing reference to the specifics of how the work should be applied. The best modeling projects go one step further, talking not just about the specific applications of the analytics work, but also quantifying the impact of the changes in business approach, which application of the work suggests.

People

The best analytics people possess a hard to find skills mix. It combines the advanced technical skills they need to do good work, combined with curiosity, business savvy and a consulting mind-set. That mix enables them to proactively anticipate future needs as well as to constructively probe and re-shape the thinking around what they may initially be asked to focus on. Finding such people is tough, but not impossible. My advice is to focus on the following three characteristics when hiring:

1. A proven track record in direct marketing —I would happily trade a highly academically qualified PhD in Statistics for somebody with a more basic academic education who has a track record in marketing of consistently delivering business value from their analytics work.

2. A commercial mind set and accompanying work ethic—Time is money. That means that is often necessary to sacrifice what might be an interesting intellectual approach to solving a problem in favor of a more grounded, but also more pragmatic approach that can be more easily applied in the real world.

3. Results orientation and communication— Finally, the best analytics people understand that their work is a means to an end and are therefore sensitive to deadlines and dependencies. They judge their work, not so much in terms of its technical merits, but in terms of what it enables their business partners to do – They are skilled in communicating the benefits of their work and the best ways to use it.

Proactivity and Partnership

Good analytics people will try to influence the broader learning agenda by proactively shaping the processes and decisions that give rise both to requests for analytics work as well as governing how that work is used on an ongoing basis. That can sometimes be a smart defensive move on the part of analytics people to ensure that they are not on the receiving end of a non-stop stream of uncoordinated tactical requests. However it can also be a proactive stance involving continuous communication to business partners about the trends and patterns they are seeing in the course of data analysis and how those trends present both opportunities and risks to business performance. Time and again the refrain from business users of analytics services is that they would love to have a more proactive partnership relationship with their analytics colleagues as opposed to trying to guess when and how best to use them.

Analytics is a mixture of art and science. That unique combination means that while tools and data are critical enablers, it is the people and processes that really make the difference between average and great results. Organizations which lack the right analytics skills mix and / or a consistent approach are doomed to produce poor or at best, average results. By contrast, companies focused on having the right people using a repeatable process will consistently produce the kind of work that makes a significant quantifiable difference in business performance.

Niall Budds, Vice President of Marketing Effectiveness, Quaero, has extensive corporate and consulting experience in the application of best practice techniques to deliver optimal customer relationship marketing approaches.