How do we know whether our social media, user generated content or online marketing activity is working? Should we spend or save? Those are questions that it is my job to answer - yet the right answer is not always clear cut.
And in just a few weeks I will be speaking about measuring user generated content in travel at the Munich Eye For Travel conference. So rather than present a single slide containing “It depends” - I thought I would thrash out some of my ideas here and hopefully also get some of your thoughts and experiences on measuring user generated content.
It all depends on purpose
Understanding if something is working depends entirely on purpose - after all what is a successful outcome? If I was using my car as a giant garden plant pot, success would be very different in my eyes than if I was trying to drive the vehicle to the South of France.
Intent and purpose determine what success should look like. And what success looks like determines the frame of reference for measuring if you’ve achieved it. First define what matters - only then you can try and measure it.
So, before we can even think about what to measure, we have to address why a travel business might interact with user generated content in the first place. I have defined the following likely purposes - I’d be interested to hear if you think there are more:
- Direct response advertising & marketing - I want people to see this content and take a specific action now;
- Brand awareness advertising & marketing - I want people to see this content, be aware of my existence and respond to my brand/product/destination in a positive way;
- Customer, brand, market and competitor research - I want to gather intelligence I can use for strategic decisions;
- Advocacy, champion and customer get customer marketing - I want to capitalise on the great experiences of previous customers/travellers in order to influence the decisions of future potential customers; and
- Leverage (should this be distinct? - I’m not convinced) - I want to capitalise on the efforts/activities of others to achieve exposure I could not otherwise afford to buy.
The success of activity has to be measured entirely in terms of the context of these goals. There’s a big difference, for example, between trying to use You Tube for direct response marketing and using it for customer research. The same measures simply cannot be applied to both.
Is brand and direct response activity really any different, just because its on a UGC site?
If an advertiser is simply pushing its message out, but that advertising happens to be placed on a UGC site, I don’t see how that is significantly different to any other media placement. So, if the goal of the activity is purely as a direct response campaign, then I would judge success based on marketing ROI, just like any other. I would tag the campaign and specific landing pages, looking ultimately to ensure that the revenue return from the campaign referrals exceeded the expenditure. In a complex, high spend advertising environment, or with pure brand activity, I would certainly consider working with a tool (such as Microsoft’s which is explained well here) that maps a consumer’s ad-clicking history over weeks before making a purchase, rather than assuming the very last thing they clicked before conversion was the only trigger.
However, I don’t see brand and direct response advertising in user generated content environments as offering any particularly unique challenges in terms of measurement philosophy. Is a pure brand video ad on You Tube really so very different in purpose and desired outcomes to one on CNN? While I do believe “its brand advertising” is often trotted out as an excuse not to measure effectiveness properly (and that is another post entirely!) it doesn’t seem to me that the actual measurement issues for ads placed onto UGC sites are ultimately that different from measuring brand issues in other other medium.
My personal interest in better understanding points 3, 4 and 5 of above - customer research, customer advocacy and profile leverage.
Measuring UGC for customer, brand, market and competitor research
Whether you talk about online user generated content or offline customer feedback, from comment cards to text messaging - the real challenge for travel and tourism has never been about simply collecting customer feedback, but rather understanding it to identify the real issues, enable you to prioritize and take actions that contributes both to customer experience and bottom line.
However, when I ask people how they action the feedback they get from UGC and social media, I frequently hear “we have a tool that does that”. That is not actioning the feedback - that is just capturing the feedback. Action requires listening (not just ‘hearing’), understanding and implementing.
For example, social media and buzz monitoring tools offer the opportunity to hear those conversations but like any tool, they require attention, analysis and action on the part of the organisation or they are meaningless.
So I would measure the value of UGC for research purposes, not in terms of how much raw data was accumulated or how many mentions (positive or negative) I had, I would even want to move past understanding if I was tracking generally upwards or downwards in terms of buzz. Instead, if I wanted to understand the value of UGC to the business bottom line, I would measure it in terms of actionable outcomes that resulted from the information.
So, this could be:
- A simple quantitative record of UGC inspired actions
- The estimated value of stopping a conversion leak - eg, we contributed £XXXX to the bottom line by fixing this conversion problem now, not a year down the line
- Money saved on conventional research or data collection activity (though beware, as I would see it as enhancing not entirely replacing existing research activity)
- Uplift in specific satisfaction points, as a result of actioning UGC feedback, and therefore uplift in future intent to recommend/purchase
- Savings in other cost centers, such as PR, training and call centers
I’ve banged on about it before, but the measures have to pass the ’so what?’ test. For example, if you’ve had 5,000 positive UGC mentions - so what? But if UGC has helped increased revenues by £50,000 a year by fixing the number 1 customer pet hate then a big pat on the back and hopefully a nice bonus too.
Measuring advocacy, champion and customer get customer marketing
Jupiter Research recently found that user generated content is used by 40% on online travel researchers - with ratings and reviews/recommendations dominating. And when it comes to accommodation research, user generated content is nearly twice as influential as brand and almost three times more influential than family/friend recommendation. (See our recent post on this).
Perhaps the most significant and powerful aspect of user generated content is its apparent peer to peer authenticity. The business or tourism organisation treads a fine line here - capitalising on the notion of “don’t take our word for it” whilst ensuring authenticity is not corrupted.
However, capitalising on the reviews, comments, images etc of previous visitors is a profound way to influence the decision of future travelers (provided your product is good). How might you measure success here?
I would say there are four distinct factors to consider:
- Participation levels amongst your target group (visitors, previous customers, locals) in terms of creating reviews, uploading images, submitting ratings
- The sentiment and satisfaction expressed in that UGC
- The influence of that UGC over other potential travellers (ie, is the UGC actually being seen)
- The conversions, actions, revenues resulting from that UGC.
Can these things be joined up to give an indication of the value and impact of UGC on a business or destination? I think so, yes. It won’t necessarily be a seamless piece of analytics and I think it requires creativity, but I know it can deliver some valuable insights that pass the ’so what’ test.
I shall use the example of the Hotel Rival in Stockholm, given that I have written about my entirely UGC influenced stay there last year.
So, looking at participation - with extensive reviews on Tripadvisor, images on Flickr and the like, there is plenty of scope for Hotel Rival to aggregate and analyse this data (see this post on montoring online feedback) and maintain an absolute benchmark or a rolling ratio of customer participation in UGC. At the same time, they will almost certainly want to take this one step further and benchmark satisfaction and sentiment, again readily available in the data - provided some analysis time is put in.
The influence of the UGC over other travelers can be determined, at least in part, by the overall importance of the site in terms of traffic, reputation and awareness. Additionally, responses to the specific UGC (such as favouriting, indicating it was found useful, commenting etc) provides guidance to whether the UGC is influential or ignored.
But does the UGC drive revenue? In an ideal world there’d be a nice easy way to track my referral straight from Flickr into Hotel Rival’s web analytics and reservations engine. Unfortunately, in my real world example I skipped bewteen TripAdvisor, Flickr, the hotel site, Expedia, other agents, before returning to TripAdvisor (probably via Google) and booking through Expedia. Track that with confidence (and yes, that is exactly what the Microsoft engagement mapping platform mentioned earlier tries to do). I’m not sure, apart from the fact that I went to great lengths to tell them, that my booking data alone would have revealed to Hotel Rival’s management the original influence of the UGC.
hmm, if in doubt - ask the customer
So I suggest an alternative. In the post stay survey (or for other business types, during feedback research or during the “how did you hear about us” questioning) directly ask the customer if they were influenced by UGC. Get specific if possible and if you can, tie that to reservations data.
Given that when it comes to accommodation research, user generated content is nearly twice as influential as brand and almost three times more influential than family/friend recommendation - you may want to redistribute you marketing efforts when you learn what is really working.
This is my take on measuring UGC in travel and of course I have my own biases. I would love to know if and how you are tracking the value or impact of user generated content within your destination or organisation.
This entry was posted on Wednesday, August 27th, 2008 at 1:53 pm and is filed under Internet usage statistics, Online customer behaviour, Social media measurement, Travel 2.0, Web analytics and web measurement. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
