Just like any DIY project, botched research has the potential to be a messy, costly disaster.
If you’re making the effort to do customer or market research to support your decisions, you’re committing time and money in the expectation that your business will see a positive benefit from your efforts.
The trouble is how do you know if the evidence you’re using to base your decisions on is flawed? Could your research efforts actually be creating you bigger problems? Problems that may only become apparent after you have committed considerable expenditure.
Here’s an example that one of my fellow forum members at 5000bc.com kindly shared:
Some years ago now a colleague and I surveyed local businesses on potential training courses. We got a clear cut positive response in the training and the preferred courses… But when we actually advertised the courses the response was zero.
Despite their best efforts, this business was left out of pocket when no one purchased the courses they so carefully researched.
This kind of story is really common and it’s one the reasons why you’ll often be told that research is best left to the experts. But it doesn’t work like that in the real world! People always have and always will do there own research, for obvious reasons of cost and practicality.
So here are some practical tips to help you get your DIY research right
These are five common mistakes that can turn your research efforts into a business liability:
1. You load the dice to get the answer you’re looking for
One of the problems with doing your own research is that you are so invested in it, so personally involved, that you can dramatically influence the results.
It is easy to fall into the trap of asking leading questions, meaning that your expected answer is clearly apparent to the respondent. For example “Did you enjoy our lovingly prepared food?”, “Do you think we’re right to be investing in reducing our emissions” or “Don’t you agree that towels should only be washed every two days.”
Your physical presence can also cause you to get the answer you seek. Your research respondents will mostly likely want to please you or avoid offending and given the chance will tell you precisely what you want to hear. You will reinforce this further if you find yourself nodding or verbally agreeing with their “correct” responses, or give them other clues about what you would prefer them to say.
So, when conducting your own research, try to keep your intent hidden and present a neutral, uninvested air.
2. Right questions, wrong people
The most common error I see amongst the teams of school pupils running their own businesses for Young Enterprise is that they design a product for to sell to their parents’ age group, yet they conduct their research amongst their teenage peers.
It is the same mistake as the HIPPO syndrome (making business decisions based on the Highest Paid Person’s Opinion).
You, your family, your friends and your resident HIPPO do not represent your target research population. It is very important that your research sample is representative of the market whose opinions you are seeking.
Put simply, if you want to find out how ferry users think and feel, don’t centre your research around airport users.
3. Right questions, wrong environment
How often do you change your underwear? Do you ever wear the same socks three days in a row?
If I asked you these questions in a focus group, surrounded by other people claiming to always change their socks twice a day, how likely would you be to admit the truth?
What about if I asked you the same question face to face? Personally, it’d need to be an anonymous paper or online survey, before I’d ever confess to performing the occasional sniff test on my socks.
Think carefully when conducting DIY – especially if you’re asking something personal or contentious – in case you are creating a situation where people will be too reluctant to tell you what they really think.
4. Failing to corroborate the evidence
Trusting just one source of evidence leaves you really exposed if that data is flawed. Professional researchers will typically combine several research methods to give a range of perspectives – for example, surveys, one to one interviews and focus groups combined.
Even if you’re using online data, rather than conducting direct research yourself, don’t rely on a single source of information. You’ll find dramatic different results when comparing different sources (especially if the process of repeating the information has obscured the original context and margins of error associated with the data).
And don’t forget, testing is a form of research. Putting a bunch of different offers or products on pre-sale and measuring which ones attract interest can be a useful way of corroborating your other research.
5. Good evidence, bad decisions
As this BBC article explains, the human mind is pretty irrational.
Despite good evidence, poor analysis means that bad decisions are made. There are many complex reasons for this, but here are some common ones you may have experienced:
- Deliberately or otherwise, you pick and choose from the evidence in order to “prove” your preferred theory
- Regardless of the evidence, your selective perception means you see exactly what you want to see
- The evidence we heard most recently or most emphatically take on undeserved importance
- We get blinded by averages.
The average visitor does not exist, instead averages (the mean) can hide meaningful differences in behaviour and give an overly rosy worldview. I know plenty of businesses that think their average website visitor stays for 5 mins, whereas in fact 95% of visits last for under a minute and a handful last for hours.
To conclude
So, there are pitfalls and risks to DIY research and it is harder to do it right than many people realise. But keeping these five common mistakes in mind when you are conducting your own customer, market or business research may help you avert a costly disaster.
I’m collecting DIY research disaster stories right now – I’d love to hear about any business problems you may have encountered based on using dodgy research.
This entry was posted on Friday, February 15th, 2008 at 5:13 pm and is filed under Business research, Tourism market research. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.