Research and the dangers of driving with mirrors

I have no use for side mirrors. You could bang them off the sides of my car with a rusty Ball Pein hammer and I wouldn’t shed one tear.

Side mirrors will look you straight in the eye and tell you it’s clear to make a safe lane change but, maybe one time out of 20, they are hiding a speeding Mustang or transport truck that could kill you. Downright traitors, they are not to be trusted. Driving instructors will tell you as much, urging you to use an accompanying shoulder check when changing lanes. And, ever since a few close calls early in my driving career, I have heeded this driving advice – trusting my own eyesight over any reflected image in glass.

I guess it should come as no surprise then that I have always been so distrustful of the images seen through the mirrored glass of focus group facilities. To me, that large pane of reflective glass has always had a lot in common with my side mirrors – the images we see can be very revealing but it’s dangerous to put too much trust in them. On the road or in the boardroom, blind spots kill.

Want proof? Research said that New Coke was a good idea. In the same vein, research told us that our life would be complete with a cola that wasn’t brown. (Crystal Pepsi anyone?)

And I’m pretty sure research was behind aerosol cheese.

My point is that too much faith is placed in research at the expense of shoulder (and/or gut) checks. David Ogilvy once said that most people use research the way a drunk uses a lamppost – for support rather than illumination. Truer words were never spoken. He’s been gone many years now, so you know this isn’t a new problem. It seems like bad research is like bad tuna – you can’t keep it down.

In my own career, I have witnessed one of the world’s leading packaged goods companies base an entire category segmentation on an in-depth qualitative study with 16 people. I have watched qualitative work sabotage quantitative findings and a few domineering respondents shoot brilliant creative ideas out of the sky.

I’m not saying that all research is bad – it’s still one of the most illuminating tools for clients and their agencies. But I want you to ask yourself the last time the batteries were really fresh, the light was really shining and you got something really valuable out of your focus groups other than a bottomless bowl of M&M’s.

In my own experience, much of the better dialogue from these evening sessions has come from the peanut gallery side of the glass (usually centred on which respondent made the best case for euthanasia or got dressed in the dark).

But, enough about bad research, what about the good stuff? Is ‘creative research’ really the oxymoron everyone believes it to be? In my view, no. There are a number of good research companies out there that are trying out new methodologies and techniques – anthropological studies, observational work, online focus groups, chatroom intercepts, to name just a few.

There are companies selling new approaches but not enough people buying. Why go out on a limb, when the focus group is the familiar old money in the bank that everyone knows? Focus groups are like the weather – everyone complains about it but nobody seems willing to do anything about it.

There are some people, including myself, who feel we might be well served to revisit methodology that pre-dates the focus group mirror. Back in the early days, there were no focus group facilities so researchers and marketers invited regular people into their living rooms for tea and cookies to just talk and share opinions/ideas.

In this ultra-marketing savvy age we find ourselves in, I think you can make a case for demystifying the qualitative process and injecting it with a dose of candour and honesty. I’d be surprised if this didn’t yield better findings.

So, there are better approaches and better methodologies to be had (better recruiting practices too but that’s a whole different article) that will yield better results and better decisions, but you still won’t find me shining up my side mirrors. The reason is that, just as there are bad ads, there is bad research, bad respondents and bad luck.

Too often research is trusted as gospel and too often we flip quickly past that page in the front of every qualitative research deck that says the findings lack statistical significance and are strictly directional in nature.

No matter how good research gets, agency and client stewards of the brand are paid for their instincts and to check over their shoulder when making a major lane change. Those that forget this fact are destined to be blindsided.

Jeff Spriet is the president of chokolat, a branded entertainment company (like branded content, only sexier). While he is wary of shiny glass, he is generally trustful of dogs and small children. He can be reached at (416) 979-4409 or jeff@chokolat.ca.