While most design researchers and practioners would agree that surveys aren't the best tool for designing experiences, I'm still suprised that we get pushback on the value of other (primarily qualitative) research methods from customer experience professionals and of course their business colleagues. While many of these people will argue to the grave that surveys are "better" than qualitative research methods because they mitigate risk by being both quantifiable and statistically significant, they don't realize that when designing experiences, surveys introduce "risk" well before a survey is analyzed. How? Well, surveys:

  • Limit responses. Most surveys (whether they're open-ended or offer restricted responses) ask users for their reaction or input to a specific question or situation. If you're asking for something that's relatively black and white, that's a perfect technique. But if you're asking people to explain why they did (or didn't do) something or about the nuances of how they did something, or if you want to see how their context influences their behavior, then surveys are difficult to craft because you essentially have to know the answers before you ask the question. And if you don't know all of the right answers, then you're introducing risk by guessing what they may be.
  • Rely on respondants to know the truth. While most consumers (or at least some) don't lie purposely or at least not consistently enough to break a good survey, surveys often rely on consumers to tell you what they will or plan to do and whether they like/dislike or want/don't want something. While those are all important things to know, most people aren't fully aware of, don't remember, or are not able to predict their own behaviors in situations that they haven't been in before. So when trying to design a new interaction, what people tell you they will do may not always be a risk-free way to determine what they will actually do.

The analysis that goes into surveys can be impressive, and their responses can hold interesting trends and insights for customer experience professionals and their colleagues. But don't get so enamored by the analysis that you miss the fact that the technique had its own drawbacks to begin with. Qualitative methods might not come with the fancy charts and statistical analysis (although many social scientists might disagree), but when done right, they offer the most promising way we've seen to overcome the limitations that survey-based techniques impose when trying to design an experience.

If you're interested in learning more about some of these techniques or learning how they're used in other enterprises, join me for the Understand Your Customers track at Forrester's Customer Experience Forum. Speakers from Wells Fargo, Frog Design, and the Design and Usability Center at Bentley University will be sharing their real-world experiences.