I would agree and disagree (sort of) with some of your points!

I completely agree that surveys that are too long will get low response rates (and unless you are going down the line of employing a panel broker, this is an important consideration).

I kind kind of agree about too many open eneded questions but I’m always surprised at how tolerant respondents can be of them – perhaps I’ve just been lucky. I guess, as you say, its a question of balance and too many will undoubtedly lead to a lower response rate.

I would also agree with what you say about the mixed methodology approach and wish that more businesses could take this approach!

Thank you for a very interesting post. Just I would like to bring into readers’ attention two v. important rules: if you run an online survey, keep it as short as possible and avoid too many open questions as you don’t get a great response rate for this (these) particular question(s).
Myself, I like a research approach that combines qual and quant methodologies but it really depends on a research problem. If we would like to define problems, get insight into a specific issue and listen to our clients, I would strongly reccommend qual research and employ a researcher/ research agency.
Thanks
Ela

So I’m going to keep nudging forward on this because I think it will become necessary on some kind of way maybe we haven’t even clearly imagined yet.

I think that one of the big limitations with brief online comments is as you suggest – the lack of complexity. I sometimes wonder whether this is also a lack of context in that a short directed statement on a 2.0 website gives not a clue about the other significant prejudices and influences in that persons life. In other words, you can’t discover the multiple other touchpoints in their lives that make them complete that you can uncover in a focus group.

So how do you get round this? I guess if you’re Google, you should be able to view this persons life in very sharp detail and if you are a company with multiple touchpoints (eg one where you access different products or services depending on mood) then I guess datamining is another way of doing it (I loved this story in Wired way back that touched on this subject) – but I suspect that, for most businesses, the focus group is still the best way forward.

Good post , Stephen.

Tim; you are right about the benefits. I’ve seen clients develop conceptual frameworks with judgmental confidence that allows them to leap ahead and deal directly, now, with the problem. Clients are people, too.

Tim: I’m guessing one of the arts of what you do is to differentiate between the usual and the seemingly usual. For example, I assume that most of the comments you get are variants on already known themes but every now and again, there will be something that is just different and needs pulling out. I’m not sure that technology have reached this human insight level yet in open text fields!

One of the things which convinces clients of the benefits of this is the value of the output.

For example, if about 10% of the comments are suggestions for new products or services then part of the routine becomes producing a specific summary for the product development team.

Customers and visitors can be remarkably open with their views on what they expect that you ’should’ be offering. That information is a goldmine for actions which can help you stand out from your rivals.