Freakometrics – The hidden side of business facts Part 2 of 4

Why Predictive Analytics don’t tell you
anything about Success Drivers?


The quest for “what”-questions and the search for hidden reasons for  success has brought early researcher to a sobering insight. It is useless to correlate potential success drivers with measures of success. It is meaningless to compare properties of those who are successful with those who aren’t. This approach simply does not consider the impact of other factors and inevitably must erroneously ignore the influence of other factors.

This was the rationale behind the invention of regression techniques which are foundational of most predictive approaches. Regardless of whether regression techniques come in the disguise of “econometric modeling” or “artificial neural nets”, they do what is required to find factors that drive success in the context of all other factors that have an impact: Great, mission accomplished, correct?

Why marketing-mix models can fool you

One day, I presented Andreas the results of his TV spend on sales using predictive techniques. He nearly fell off his chair. There was almost no effect. “I need to dump the whole TV budget immediately” he screamed at me. The results were correct and wrong at the same time. Why? It was correct because, this is what the data said, based on mathematically accurate processing, and the results that a top-notch marketing-mix agency would have produced. It was wrong because these were just results from predictive techniques. Those just model the direct effect of a particular factor on an outcome.

Predictive techniques do not consider that TV spending does not just leverage sales directly. TV spending makes people google and click on search advertisements, it makes target customers call the call-center, visit the company’s website or online shop. Those resulting actions have positive consequences, too. One could argue that all those action items should also be included in the model as predictive factors to guarantee meaningful results. As a consequence the coefficients of a predictive models would tell you what the impact of TV spend is when viewers do not google, click on ads, call the hotline, and visit the website or the shop. But this is not what Andreas would like to know. It is exactly the indirect effects among predictive factors which tell a different story. Knowing this we were able to help Andreas by separating the indirect from direct effects of his TV spendings on total impact ROI measures.

“If you have a hammer, every problem looks like a nail.” Even though most statisticians know that factors in predictive models are assumed to be independent, they tend to forget it since it is the hammer everyone uses. If you do not believe me, please visit the websites of leading providers of predictive analytic suites. They promise to quantify the impact of success drivers, but they don’t. This is what I call “The Predictive Analytics Fallacy”.

Why can you compete in some price-sensitive markets without price related moves

I was sitting in my favorite chair with a smile on my face. It was Jack on the phone: “Our new market initiative has an overwhelming success. However, I don’t know why”. This was new to me. Typically customers contact us when something went wrong. We took the brand tracker data of this service provider and took a deeper look into this “why” question.

To me it is like sitting as a child in front of a puppet show and the curtain opens: the moment when our algorithms spit out the results. What we found was astonishing but largely useful. The new price scheme that was launched in the service provider’s initiative was not directly driving brand consideration. In other words, conventional driver models would erroneously have told us that price has no impact at all.

It was just feeding the story of being “the challenger” against the market leader, which itself was a major reason why new customers were attracted by the brand. This led to a highly profitable recommendation: To keep momentum –you do not need to continue lowering prices. Instead introduce other initiatives that feed the “the challenger” story.

Predictive analytics is a great discipline. It is good in making predictions, but it largely fails in providing insights regarding “why” questions. You need to dig deeper, much deeper.

If you find this post useful, we appreciate if you “like” it and share it.