Consider this the golden age of data in marketing.
Until recently, the data available to us in marketing has actually been relatively limited. Historical performance reports, old-school market research, and intuition born from experience have been the primary tools in our decision-making toolbox.
But the world has gone digital, and that has changed everything.
Everything digital generates data: website visits, email touchpoints, mobile app usage, Google searches, ad impressions, social media interactions, video views, online community discussions, and more. It’s a gushing fire hose of information of customer behaviors, intents, and preferences.
We can combine that natively digital data with data from other channels — call center logs, transaction histories, and in-person interactions. We can mix in publicly available data on demographics, firmographics, seasonal cycles, the economy, and even the weather.
Stir this all together, and the result is “big data.” Marketing is now the proverbial kid in a candy store with these overflowing barrels of sweet data laid before our feet.
Making Big Data Work
In theory, insights that will dramatically improve our marketing effectiveness are somewhere in that vast sea of data. The promise of data-driven decision-making is ascending as a more reliable alternative to traditional experience-driven decisions.
But there are a couple of hurdles between here and nirvana.
First, marketing teams need to adopt new technologies and to learn — or hire — new skill sets to work with data at this scale. New roles such as data scientists and marketing technologists are arising to meet these challenges. Training and cross-discipline collaboration can also help to raise everyone’s data IQ.
Once you’re prepared to wrangle big data, then the next hurdle appears.
In digging through that data, you will uncover a plethora of fascinating correlations: shared characteristics or common sequences of touchpoints that are associated with different clusters of customers.
The question for each such discovery: does it matter?
Is that correlation something that actually influences customer behavior? Or is it merely a coincidence or an effect of some other factor that we haven’t included in our analysis? In statistics, this is the age-old warning: correlation is not necessarily causation.
But in marketing, we need to go even further. It’s not enough to establish a cause-and-effect relationship. We must also be able to operationalize it and demonstrate that the resulting increase in business performance is worth the effort.
There is only one surefire way to achieve this: we have to run an experiment.
Test, Test, Test
Experimentation is the gold standard of determining cause-and-effect, as almost any scientist will tell you. We must take the insight garnered from our data, turn it into a hypothesis — a testable question — and run a controlled experiment. We can then measure the results and thereby prove (or disprove) our theory.
Luckily, in a digital world, it’s never been easier to run such marketing experiments. We can quickly and cheaply try many different versions of ads, landing pages, web applications, email messages, and other digital touchpoints — running A/B tests between our original “control” versions and new “challenger” versions that try out hypotheses.
What’s particularly valuable with digital marketing experiments is that it’s usually feasible to limit the exposure of the test to a very small number of customers. If the hypothesis proves to be wrong, very little was risked. You only keep the winners.
Because it’s so easy to run tests like this in the digital medium, companies such as Google and Amazon — businesses that grew up natively in the digital world — have deeply embraced experimentation as part of their culture. Hal Varian of Google has said that on any given day they have 100-200 experiments in progress.
Take note, however, it was reported that out of the 12,000 experiments that Google ran in 2009, only about 10% of them resulted in adopted changes.
It’s that relatively low success rate that may have held back many companies from harnessing the power of marketing experimentation. A study of Fortune 1000 marketers by the Corporate Executive Board last year found that only about half of them agreed with the statement “my team accepts that some experiments must fail in order for us to learn from them.”
It’s a risk aversion legacy from pre-digital days, where experimentation was much more costly and big bets were either spectacular successes or disastrous failures.
But that’s a legacy that modern marketing must shed.
From Big Data to Big Testing
As Greg Linden, who led a set of experiments at Amazon, stated in an article on big data in The Atlantic, “To find high impact experiments, you need to try a lot of things. Genius is born from a thousand failures. In each failed test, you learn something that helps you find something that will work. Constant, continuous, ubiquitous experimentation is the most important thing.”
Such ubiquitous experimentation in search of big wins can be labeled “big testing,” the natural complement to big data.
Big testing gets its bigness from three things:
1. A willingness to experiment with big, bold ideas — to try something that may fail. Risk is mitigated by testing such ideas on a small scale, not by avoiding daring new concepts.
2. The pursuit of experimentation is championed by management — it’s a “big deal,” an integral part of the culture. High failure rates of experiments, as long as they’re run well, are not treated as failures of the individuals running them. On the contrary, aggressive testing is recognized and rewarded.
3. A large number of people throughout the organization are empowered to run experiments in their work. This is what gives big testing its scale, allowing a large number of experiments to be run across different facets of the company.
Big data is good. It’s an amazing source of new hypotheses for marketing. But to truly unlock the value from big data, marketers must embrace big testing.
Like big data, big testing is a native approach to marketing in a digital world.
Scott Brinker is co-founder and CTO of ion interactive.
Network
