Marketing professionals around the world are facing a very similar problem. Measuring the impact of marketing and quickly identifying specific ways to improve the ROI of your marketing spend has never been easy – even when most of our marketing was limited to a few television networks, radio and newspapers. The problem has only gotten more challenging with the explosion of television networks, satellite radio, digital advertising, email, social networks, and the other millions of modern day distractions competing for a customer’s attention.
It’s become clear over the past few years that the old ways of measuring the performance of single channels are no longer enough. Today’s marketing pros are looking to optimize their entire marketing mix and need to understand how each component part may magnify or hinder the impact of others.These insights need to be actionable and based on what is actually happening today – Not what happened months ago.
Rex Briggs, CEO and Founder of Marketing Evolution, recently sat down with Mike Eichorst, Senior Vice President of Marketing Measurement at Citibank. Mike shared his perspective based on a 35-year journey through the world of analytics, marketing measurement, and market mix modeling. Here’s what he had to say:
The Limitations of Marketing Mix Modeling and Other Attribution Models (MMM)
REX: You’ve tried a wide array of measurement approaches. Give us a little perspective on what you found as you search for effective tools to measure marketing impact.
MIKE: Let me start by thanking you for asking me to share my thoughts today. I know many marketing professionals are looking for tools that help them understand their customers better and reach them more efficiently. There are a lot of people out there making some wild promises and many professionals are feeling a little lost in all the noise.
It can be very disheartening. Many advertisers are trying to kluge together an ugly mess of traditional approaches to get the insights they need. Unfortunately, that’s just adding a level of complexity and inefficiency on top of the inefficiencies that may already exist in your marketing mix.
Here’s the challenge: Although Mix Modeling may provide some broad guidance on how to split your marketing pie across media types, including digital, there are some very significant limitations.
The first limitation is timeliness. Results typically lag the actual spend by several months. Marketers find themselves working off yesterday’s news responding to the dynamics of a marketplace that is already weeks or months past. In most companies, marketing is always changing. So, looking backward several months may not be very relevant to the marketing you’re doing today. Think about the impact this has in industries where seasonal changes have a big impact on sales. Retail marketers need up-to-the-minute insights to make sure their programs work for maximum impact on promoting the items their customers need now.
The second limitation is the level of detail. Mix modeling doesn’t provide details on the performance of individual campaigns that could help improve our executional tactics (publishers, placements, titles, etc.) In addition, they do not do well measuring the impact of smaller, experimental media spends like addressable media. This makes it very difficult for marketers to test and learn, especially in new channels with which they have limited experience. We really need to know whether or not these new channels deliver results directly or through impacts they may have on other more established tactics.
The third limitation is that mix modeling doesn’t typically estimate the impact of marketing spend on brand metrics. When it comes to understanding how your advertising builds the long-term value of one of your company’s most valuable assets, these tools are blind. They’re also unable to show how shifts in brand metrics impact sales or the performance of your other marketing efforts.
For many advertisers that use a combination of traditional approaches, it just hasn’t come together. And for those who are thinking of starting to implement new solutions it becomes very confusing.
REX: Well it sounds as if there are a fair number of problems with mix models. What about the attribution solutions you tried?
MIKE: You probably won’t be surprised to hear that we found some real shortcomings with the Attribution models currently available. A lot of those failures have already been covered in the media. In addition, we found that the results for the contribution of Digital Media from Attribution models never matched what the Mix Models estimated the impact of Digital Media to be. Since we couldn’t calibrate between the two models, and without a standard against which to measure, we were left wondering which was telling us the right story?
I think we know, too, or at least believe, that traditional media, like TV, radio, print, and out-of-home advertising, have an impact on our digital marketing. And although recently most Attribution vendors have attempted to adjust/correct for the broader impact that traditional media has on Digital, that isn’t typically as robust as traditional mix modeling. If you look closely at what they do, there is a forced alignment of results. Forcing your That’s not alignment. It’s a hack.
And finally, we could never verify if the Attribution models really worked. For me, personally, I always got the sense that they give a contrived view of what was happening in the marketplace and an illusion of command and control based on very imprecise estimates.
The Solution to Measuring Marketing Impact
REX: So, given the drawbacks of Mix Modeling and some of the shortcomings of Attribution, what are marketers doing? How are they trying to get a more complete picture of their marketing ROI?
MIKE: Well, somewhat ironically, many of the efforts to try and get this more understandable and comprehensive view into marketing ROI are just muddying the water. It seems to have gotten more complicated as start-up firms have raced into the marketplace to fill these gaps. Many of these companies measure individual pieces of the puzzle. But these stand-alone piecemeal solutions have, in my opinion, one fatal flaw: They fail to integrate the effect and impact of what’s happening in one channel on the rest of the marketing mix. They cannot detect the rise in your SEM performance that may be driven by a more effective TV creative. They fail to see the decline in online transactions caused by the overzealous email marketing campaign. They also fall short failing to estimate the impact on sales, i.e., TV attribution typically measures only how TV impacts traffic on owned properties, not how it impacts sales.
The primary problem is that these solutions only look at one piece of the marketing puzzle in isolation, one at a time. For a while TV attribution became hot because advertisers wanted a better understanding of how TV drove traffic to their owned properties. Then they realized they really needed to understand and integrate everything else. These narrow solutions aren’t going to get that job done.
REX: We’ve been talking for a little while now and the situation seems a little grim for marketers looking to improve the ROI from their marketing mix. It seems like marketers find ourselves in the same position as John Wanamaker: We may have made some improvement, but it’s still hard to know what part of our advertising spend is being wasted. What have you found works for you?
MIKE: (LAUGH) Sorry. I didn’t mean to paint such a grim picture. There is hope for marketers, and tools we’ve found that help us find ways to maximize the business impact of marketing and prove that ROI to others in their organization that help determine marketing budgets – Usually the CEO and CFO. Because of the inherent limitations of the usual complicated mix of solutions— Mix Modeling plus Attribution et al – some vendors have chosen to shift direction and develop more integrated approaches. These integrated solutions rely, not surprisingly, on data for individual consumers to model the impact that marketing and advertising have on the business in an integrated and detailed way. That approach provides the benefit of both strategic direction and tactical planning: How much to invest in each type of media coupled with ways to best match copy, creative and messaging for specific online and offline publishers, and the most impactful placements for each ad content. That optimization typically generates an additional 25% improvement in the impact of marketing.
My advice for those seeking a competitive advantage in the marketplace is to find a partner who has taken a different tack. You really want someone who is leveraging this individual-level, person-centric consumer data to simultaneously solve the challenges of Marketing Mix and Attribution short-comings and deliver a comprehensive view into the performance of your marketing mix. They need to have a proven solution.
You’ll want a vendor who can give insight into how much you should spend in each media type, but also provide for a granularity of results and speed that has immediate tactical benefit in optimizing creative, publishers, placements, titles, etc. in near-real time.
You need to make sure, as well, that the two results—MMM and Attribution, are derived from the same data and cover all channels and always agree.
Another benefit of that approach in new solutions is their ability, because they are using individual-specific data, to model the impacts that Sales and Marketing have on Brand metrics, and how those shifts then impact Sales. This was a big deal at Citibank, where we always struggled to find evidence of the impact of brand spending.
REX: So, there is hope. But why not just hand it over to your ad agency to take care of this measurement mess?
MIKE: There are two reasons: First, clients don’t want agencies to measure their own work. According to a recent survey by the ANA, over 90% of marketers indicated that they prefer independent, 3rd party measurement.
Second, independent measurements bring credibility and insight to agency performance. For example, when agencies share performance metrics with a client marketer that show a campaign failed to hit an ROI goal, that data is seen as more believable. The agency gains credibility sharing the bad news. However, when agencies present data that show successful efforts exceeding ROI goals, clients are more often skeptical. The metrics that show their good work are discounted by a skeptical marketer that sees the metrics as self-serving to the agency.
The most important partner to a good agency is independent measurement. Advertisers shouldn’t want, and the smart ones never want, their agencies to measure their own work. That’s why most agencies no longer push their own mix modeling capabilities--advertisers want an independent assessment.
REX: Thanks for taking the time today to share your insights. Is there anything you’d like to add?
MIKE: Thank you, Rex. Today’s marketers need a new approach that leverages more reliable person-level data to do the things that Marketing Mix Models and Attribution attempted. These new tools need to deliver person-centric insights, derived from a single set of data, quickly and reliably to drive higher levels of marketing performance. Simultaneously solving the strategic issue of how to best allocate budget, and the tactical issues of how best to invest within each channel, matching each spot with the best publisher and placement, marketers gain a competitive advantage in today’s crowded and fast-moving marketplace
About the speakers:
Rex Briggs founded Marketing Evolution in 2000 and currently serves as the company’s CEO. Rex is known as one of the world’s leading experts in marketing ROI measurement and optimization technologies. His expertise derives from direct experience measuring and improving the performance of a wide range of marketing programs for more than 100 Fortune 500 marketers. He is also the best-selling author of “SIRFs-Up – How Algorithms and Software are Changing the Face of Marketing.”
Mike Eichorst has over 30 years’ experience applying analytic techniques in banking to maximize the business impact of their investments in marketing and advertising. That has included roles as Strategic Planning Executive and Head of Analytics for the Chase Credit Card Business, and head of Marketing Analytics for Branch Banking, Personal Banking, Home Equity, Mortgage, and Wealth Management. He pioneered the use of Market Mix Modeling at Chase, TIAA-CREF, and at Citibank.
