Back to Phase 3: Detailed Design
Compared to earlier concept testing, the testing done at this phase should not be centered on exploring solutions so much as refining and optimizing specifics. In much the same fashion as earlier testing, however, participants should be given scenarios or tasks, and asked to complete them through use of the prototyped interfaces, while also thinking aloud to express their thoughts and feelings of the experience as they proceed.
With moderated testing, a moderator sits with the participant to guide the session. This involves giving the participant an introduction to the test and its goals, then giving tasks to the tester as the session progresses, prompting the user to explain their thoughts and actions, and helping prevent the tester from getting completely off track.
In this method, the moderator must be careful to avoid offering any guidance on how to complete the test tasks, even if the tester begs off a difficult task, until the entire test is done. If the tester has veered so far off as to make their continuation unproductive, the moderator may elect to gently direct the tester to start over or move to the next task. The moderator must also be extremely careful not to ask leading or biased questions about the design in an attempt to elicit specific feedback. The moderator should exhibit neutral attitudes regarding the design and the test tasks.
In unmoderated testing, the efficacy of the test relies on the participants to clearly understand the goals of the test and their instructions for properly participating, including a desire to have them continuously thinking aloud for the duration of the session. Because that behavior is fairly unnatural, it is easily forgotten about when attempting a particularly challenging task, which is exactly when that vocalization is most beneficial to the design team. Unmoderated testing is typically done via an online testing tool, where the test can be done at the participant’s convenience on their own computer, which makes participation easier.
In formal testing, the test is conducted in a lab setting with a moderator on hand to lead the sessions. In highly formal lab settings, there is frequently an observation area separated from the test area by a one-way mirror. This keeps the participant and moderator able to focus on the test, while allowing stakeholders and other team members to observe the sessions firsthand. Formal settings often have advanced technologies for recording the test, including video of the participant, screen capture of the participant’s on-screen actions, and even eye tracking. In this way, formal tests can yield highly nuanced findings.
Informal or guerilla-style testing is conducted in any environment where testers can find participants. This can be simply at the participant’s desk, or a conference room. The test need not be conducted in a highly controlled setting in order to elicit meaningful feedback.
Ideally, your setup allows for a recording of the test sessions (most importantly audio recording and screen capture) for later analysis and detailed reporting. If that is not possible, having someone available to take notes during the sessions, such as key quotes from the testers and details around specific difficulties, will suffice. The findings should inform iterations of the detailed designs and prototype, and a repeat of testing until the designs satisfy user needs.