BES Playbook

Back to Phase 1: Research & Discovery

3.3.1.2 Surveys

Conducting surveys is a simple, quick tool for gathering a large amount of input. And since this method is generally simpler and easier than others, it can frequently be a default recommendation. However, it is important to consider the advantages and limitations of surveying before selecting this method.

When to use

Use to quickly gather shallow feedback from a large number of respondents, or from a set of respondents who have anonymity concerns.

Requirements

  • Ability to distribute the survey to your desired participants.
  • Ability to collect its results (if anonymous, a third party must be used).

Variations

Identified-response versus anonymous

With identified-response surveys, the responses are tied to the identify of the responder, either through requiring the responder to provide that information, or by issuing uniquely identified surveys to each responder. This level of control can have a suppressing effect on the survey, both in terms of total responses, and in amount of honest criticism offered in answers. On the other hand, slanderous or other malevolent responses will be largely avoided. Be mindful of your subject matter and its prevailing sentiment among your audience.

Identified-response surveys also have the advantage of optionally asking respondents to avail themselves for follow-up surveys or interviews.

Anonymous surveys offer the ability to collect more honest responses when you expect the topic to skew negative, and will generally receive higher participation because of the lower responsibility or risk to the respondent.

Product/output

Categorizing responses to open-ended questions can provide rough quantitative measures around frequency of similar responses. (For example, answers like “Site is slow” and “Pages take too long to load” could both be categorized as “Site performance does not meet expectations.”)

Close-ended questions frequently already lend themselves to this sort aggregate quantifying. This data can then provide support and evidence to personas and user scenarios, as well as help prioritize remediation efforts for major existing deficiencies.

The individual open-ended responses themselves can uncover topics for follow-up user interview sessions around intents, motivations, and frustrations, especially if your survey’s respondents allow their identities to be known and agree to participate further.

Practical considerations

  • Consider distributing the survey to a couple test participants prior to wider distribution, to uncover questions that can be misinterpreted and should be rewritten, and to ensure you receive the types of responses you are expecting.
  • Ask questions that you have reason to believe your respondents are willing and able to answer sincerely.
  • Be aware that requiring respondent identification will reduce your response rate.
  • Avoid using Likert scales (i.e. 1-10 ratings) or similar structures for quantifying opinions. These tend to rely on subjective or arbitrary value assignments by your respondents, which devalues their responses.

References & Resources

References

Resources