Opinion Research has its limits
Now don’t get me wrong. Opinion research is extremely important and useful. Hell, we do it here (rather well we like to think.)
But in order to get the best value from it and avoid the pitfalls, it has to be interpreted well, you need to be aware of its limits, what it is good at doing and what it cannot do and lastly, you need to bear in mind that occasionally it can get it wrong.
Four examples demonstrate different aspects of this and I think that this is the only thing that Hillary Clinton, Heineken, obesity and ethical consumerism have in common (but let me know if I’ve missed something there).
American pollsters have been embarrassed recently by their failure to pick Hillary as the winner in New Hampshire and more recently in Texas where they showed Obama leading in the last few days of the campaign.
In the UK, one of the best examples of political pollsters getting it wrong was in 1992 when all of the polls had shown Neil Kinnock leading John Major only to go down to defeat. One reason for that failure was the reluctance of voters to admit to voting Conservative as it was unfashionable at the time.
In the US, the reasons for the failure in New Hampshire in particular are less clear. Did a lot of voters in the Democratic primary only decide on the last day or so of the campaign? Or did a number of voters tell pollsters that they supported Obama as that is how they wanted to be seen, while secretly actually supporting Hillary? Some commentators have said that one possible reason might be that some voters in New Hampshire wanted to be thought of as being able to support a black candidate when in fact they weren’t.
This highlights one of the perennial weaknesses of opinion research. People answer according to how they think they ought to answer or how they would like to think they will behave. That does not always reflect how they behave in practice.
This is seen in the many surveys done over the years about the propensity of consumers to choose products on ethical grounds.
If you are asked the question – will you choose one product over another on ethical grounds, you are likely to answer Yes. Will you do it in practice? Errr, no. In fact, people choose their favourite brands out of habit, or choose the most convenient or the cheapest. They like to think they would seek out the most ethical, but they don’t very often.
As a result of this, polls consistently exaggerate the percentage of the population who are ethical consumers. The number of such consumers is important and it is growing, but it may still be as little as 5 to 15 per cent according to some more reliable polls.
The obesity debate reflects a more deep seated problem with market research for products.
The food industry got itself into a lot of trouble over the quantity of fat, salt and sugar in its products.
The industry was being accused of deliberately causing the obesity epidemic. In fact what it was doing was responding to consumers in taste tests who all said that they preferred the products that tasted better. And of course being human we all love the tastes of salt, fat and sugar.
Sometimes the right answer isn’t always whatever consumers say in research. Sometimes there are other bigger factors at work.
The last example is that of Heineken.
Remember ‘Heineken refreshes the parts that other beers cannot reach’? Probably you will as it was one of the most successful ad campaigns in the last thirty years and was run for well over a decade.
But the campaign bombed in its initial research. A lot of other hugely successful ads also bombed in the research phase. The idea for Loaded, the lads mag did not do well in research either but it went on to be a huge seller for many years.
Brave company executives had a hunch that they would work over time once consumers got used to them. And they were right.
There is a difference in how a consumer thinks of something the first time they hear it and how they feel about it after they’ve seen it many times over many years.
A problem with marketing departments is that sometimes they follow focus groups and research too much and not enough their instincts or knowledge of the market place.
This of course links into politics again. If a politician only follows focus groups, he or she will only follow public opinion and never lead it. Consumers are often willing to follow if someone who has convictions is prepared to take a lead and sets a clear course.
There is an old saying in physics that the act of observation changes what is observed. In other words, the fact that you are observing means that you don’t actually see the full reality as you have changed it by observing. It sometimes applies to opinion research.