March 23, 2013 § Leave a comment
We got this client feedback through our partner about Adaptive Questions where respondents can see responses from others.
“I spoke to the client today and they are excited about the prospect of this project. They have only one concern which I could not address… they are worried about the likelihood of negative comments surfacing and being shared within the survey group, and possibly shared outside the group. “
One of our best long-term customers from TurboTax put this in a most elegant way, “You have to expect negative feedback to make any progress. If you filter it out, you are doing a disservice to your company.”
I usually like to put both positive and negative people in the same Adaptive Question because we can see how they agree and disagree. Think about how valuable it is to see agreement and disagreement between Promoters and Detractors, happy and unhappy customers. Personally, I prefer to allow respondents to say anything they want because…
- Bad ideas, negative ideas, complaints don’t usually get seen by very many people because they don’t usually get a lot of agreement because we word questions to get constructive feedback, not complaints
- Only a small portion of respondents see the negative ideas
- Only a small portion of customers participate
- Your mother told you, “If you don’t have anything nice to say, don’t say anything at all.”
Given the sensitivity of some clients such as this one, there are a couple of fairly easy solutions.
- Moderate the comments – only allow constructive comments to be seen. CloudMR has an option to require moderation. Respondents only see the comments you approve. That way we can limit the pool of ideas to positive and constructive improvements. The client will need to give us some guidance which is a part of our process. Downside: If you are not aggressive about approving legitimate ideas during the early part of the fielding process, you could go through all of your sample with just the initial 10 seed ideas.
- Separate positive and negative respondents into two Adaptive Questions(TM) – It is pretty natural to get a rating such as satisfaction or likely to recommend before the Adaptive Question. Simply use the built-in logic to get improvement suggestions from unhappy respondents and positive sound bites from happy customers. This isolates the negatives to an extent.
- Combine 1 and 2. Adapative Questions are ideal for getting WOM. The analysis will include both general buckets of ideas and individual comments. If you want to WOM for your promotions or advertising, we will ask respondents to identify themselves for the quotes.
May 1, 2011 § Leave a comment
Assuming we can get past the user experience with questions, exactly who do the responses represent? The questions are viral and I like that feature a bit because I can always pull a representative sample from the total. The only problem is that Facebook’s interface with website doesn’t work for some of the demographics even if respondents agree to let you see their information. We have had trouble getting anything more than gender from Facebook. Other stuff that would be useful to our clients when drawing a sample…
- Location (at least country)
- Social graphics such as number of friends, activities, etc.
February 20, 2011 § Leave a comment
Lots of engineers around me are talking about sentiment analysis. Most of the market researchers I know are more than skeptical about it. I can see the allure of some sort of magical box that will automatically make sense of all of these verbatim comments, but for me it doesn’t really matter. Just give me the handful of comments that resonate with most of the respondents and I’ll read them myself – sentiment and all. That is what is so exciting about some of CloudMR’s early testing of their proprietary algorithm. It doesn’t include fancy text analyzers or extra complexity. It quickly generates a score for each comment and sorts them. Interestingly the early models stratify the comments and group like ideas all together. Since I can easily see that grouping, the algorithm is clearly doing something right and producing the top ideas. It will be fun to see this implemented over the next few weeks. Jeffrey Henning has an interesting post about this same issue from last summer.
February 5, 2011 § Leave a comment
Since the data gathering on Facebook is so simple and fast, there is really no need to make the analysis overly complex. Expensive and complicated comment categorization software is not needed here. If your Facebook community is small or not very engaged, you may get everything you need by simply reading the responses. If you get more than 25 or so comments they can be tiring to read and more than 50 comments can be downright tedious as you can see from the tally from a question asked by Lowepro on their Facebook page.
CloudMR provides companies such as Lowepro and their agencies a simple and fast alternative that imports comments directly from the company’s Facebook page. The site provides an easy interactive way to look at the comments including keywords used by the community and the top responses based on top-of-mind mentions and “Likes” generated by the comment. You can explore the comments using a word cloud that filters your comments by demographics (among respondents who make them available). CloudMR is simple, fast and free for the first 50 comments analyzed.