Wednesday, 10th September, 2008
Recession busting insight - making your research work harder - 10th September, 2008
For some of you, this might be an old story but bear with me…
Imagine a man with one foot in a bucket of freezing water and one foot in a bucket of boiling water. Taking the average of the water temperatures, the man would have to say that he felt comfortable.
Which is nonsense of course.
If I told you the average of the two buckets was 50 degrees, I’ve told you nothing about the spread of temperatures that give you a fuller picture of the real situation. And if you have commissioned me to do some research into the respective water temperature in buckets and I omit to tell you this vital piece of information, you are not getting your money’s worth.
So what’s this got to do with anything? Well, Vicky’s recent post on recession busting research tips started me thinking along similar lines and how managing your researcher harder with a little knowledge of their black arts can be a viable way of getting more band for your buck.
In the example above, having an understanding of the concept of the standard deviation is would have led a manager to ask a question that that would have identified the spread of results. Better still, the research company should have identified it themselves and explained its significance in this case.
And this isn’t the only area where a little knowledge can be a good thing. While many research companies will offer data such as the standard deviation as a matter of course, I suspect that it is rare that someone commissioning research at a less experienced level will know about some basic concepts and make their researcher work harder.
So, the following is for people commissioning research but unsure of what questions to ask once they receive it. I’m not going to go into mathematical depth with any of these (that’s what you pay me for when you commission the work) but instead you should consider them as more tools for your toolbox with which to prod your research company and your data. The following concepts are, of course, the tip of the iceberg and are chosen here because they are probably the most fundamental but common concepts that we come across which remain something of a mystery to many people.
The purpose of this post is not to befuddle or bore but rather to empower. As a researcher, we know we are going to be asked about sample sizes. We know that many of you remain to be convinced that focus groups are a good thing. But show us you mean business by understanding some of the following and you’ll surely get more bang for your buck.
Mean, Median and Mode
Remember that an average is more that the total score divided by the number of incidences (the mean). The median will tell you the middle point in that data and the mode will tell you the value that occurred the most. Each of these are telling you something different and noteworthy about your data.
Standard Deviation
The standard deviation is a measure of how close most of the results are to the mean. It shows you how far the results are from the mean - which then tells you how representative that mean figure is. 68% of results will be within one standard deviation and 95% (the more usual measure) will be in two.
Using the standard deviation would have told you that, in the example at the beginning of the post, 50 degrees lacked any insight as a figure. It would have revealed that the spread of results was so wide as to be practically meaningless.
Correlation vs Causation
There is a difference, between correlation and causation. The first suggests that there is a link between two events whereas the second suggests that one event caused another. For example, there is probably an historical correlation between the number of pirates on the high seas and the emergence of Europe from a mini-ice age. But one event did not cause the other. However, an upsurge in visitor numbers might cause service ratings to fall. The good news is that it is possible to prove whether a link does exist (whatever caused it) or whether you are just imagingin it. And you can even understand in some circumstances the exact scale of the impact of one event on another.
Statistical Significance
Statistical significance does not mean that something is interesting or noteworthy. Rather boringly, it simply means that something is likely to be true. For example if it were calculated that the statement that 70% of Scottish customers and 60% of English customers preferred continental breakfasts were statistically significant, it would mean that there really was a difference English and Scottish customers - not that the insight was a more exiting one than another insight.
In conclusion: this post isn’t suggesting that you should become experts in statistics. But having a little knowledge might allow you to start asking probing questions of the experts and working their findings harder.




















As Tony Mercer, Head of Quality & Standards at VisitScotland says:
“In Austria, organised by government, everyone that checks in has to give name and nationality. This added to the hotel specific (eg 4 star, location) which all goes into a central database. It means that everyone that spends a night in Austria is recorded. It is so simple, why can’t we do it? Clearly it would require legislation/central co-ordination but its not rocket science. Its so do-able.”
“Our customers don’t match the old regional survey data that was conducted on the street. Groups, for example, get missed. If data such as visitor origin were accurate it would clarify who we are marketing to, so we could target and promote accordingly. We’d ask what are we doing in the key areas where the bulk of visitors are coming from”