Our Years of Magical Thinking (About Numbers)

We all know that research has been failing us lately. Public opinion polling has been missing a trick for a while now (namely: accuracy), and there have been loud proclamations about the death rattle of the market research industry for years. I called the problem with the numbers before the US election - and promised then that I would get into what I think is going on more broadly. I wrote a bit about this with my 5 Ways To Make Sure Your Insights Are Real, but I wanted to go a step further.

So why is research failing us?

To a certain extent, it's been years of magical thinking: imagining that doing the same thing, asking the same sort of questions over and over, will result in a different outcome. Asking about the past simply does not predict the future. And really, both the research industry and their clients suffer from the same thinking.

The fact of the matter is: people have changed, the way we interact with information has changed, the context of the environment has changed, media has changed, even diets have changed. But market research stays the same. Sure, survey research may be gamified or more visual, but the question at the top of screen? Shockingly similar to what someone doing a telephone survey a few decades ago would have asked. Awareness, familiarity, favourability, likelihood to purchase or vote. Remember when NPS seemed newfangled? It's really not. It's the same old questions just with slightly different math. And I can go on at length about the fact that "consideration set" hasn't evolved to match the way people currently think or spend.

So what do we need to do? STOP DOING SO MUCH RESEARCH. Yes, it needed to be in caps. Sorry.

A huge chunk of the market research that companies are doing is completely unnecessary. Someone bought into some tracker a zillion years ago and are afraid to change anything. They don't have the skills internally to approach changing it strategically. But do you really need that tracker when now you have much more nimble access to sales data and your web traffic? Have you looked at all at better proxies for how you are doing in your observed data? 

Stop asking your market research agencies or your comms agencies what you need to do. Start thinking about it yourself. Do you really need to be asking about likelihood to purchase when you're already getting market share data from elsewhere?

We live in a day and age when your company and your observed data as well as your social listening (not just Twitter!) should pretty easily establish your "what's happening now". You do not need a survey to do this. Maybe a couple of questions on an omnibus, but that's it. Not a 10-15 minute long continuously running survey.

Save your research budget for what the observed and social data flags up. Save it for answering the "why" and the "what now". Save it for asking the questions of what to do going forward, not what happened in the past.

And if your agencies (comms or research side!) resist this, let me know. We matchmake for in-house comms teams to find the right agencies for their unique needs. Book in for a coffee to chat about it - I promise you, there's a far better way.

The 36ns Vision - The Story is in the Data

Scheduling bots