how do i find a good partner for user testing and validation?
good user tests answer a specific development question. they show whether the intended users can complete a task using the product and where obstacles arise. when selecting a partner, therefore, the focus should be on the test objective, participant selection, and evaluation. general positive feedback is not reliable evidence of usability.
scope of services and delimitation
user testing examines how intended users perform specific tasks with a product or prototype. it helps identify difficulties and evaluate design decisions. validation also requires a clear statement regarding which requirements or usage goals are to be confirmed and under what conditions. the term should not be used indiscriminately for every brief feedback session.
a suitable partner aligns the selection of participants with the research question and the product’s stage of development. ask about test tasks, facilitation guidelines, and evaluation. it is particularly important to understand how reliable decisions for change are derived from individual observations. the design consulting by f/p design can help structure such decision-making issues early on.
distinguish between improvement and confirmation
early testing identifies problems while changes are still possible. final validation, on the other hand, requires a sufficiently representative version of the product and predefined criteria. if you confuse the two, you may draw overly broad conclusions from an incomplete model. therefore, first determine whether you want to choose a direction, improve a solution, or verify a requirement.
four criteria for choosing a partner
- select suitable participants: recruit people with relevant responsibilities and experience. colleagues from the project team are not a substitute for independent users.
- present tasks in a neutral manner: test subjects should be able to understand their goal without learning the desired solution from the task description itself.
- make findings transparent: document observations, interpretations, and recommendations separately. this allows for discussion of alternative causes.
- identify limitations: the number of participants, the prototype’s status, and the simulated environment limit the findings. a good partner explains these limitations in the final report.
project process and decisions
a test plan outlines the question, target groups, tasks, and evaluation. a brief trial run verifies whether the structure and wording work as intended. this is followed by the implementation of the test and the prioritization of the observed issues. agree upon how the findings will be incorporated back into development as part of the project scope. after significant changes, the affected tasks should be reviewed again; this prevents an improvement in one area from creating new difficulties elsewhere.
example of a prototype with unclear feedback
suppose users are asked to save a setting on a prototype. several of them select the correct function but are then unsure whether the change was applied. a simple success rate could mask the problem if the task is ultimately completed anyway.
a thorough evaluation therefore also takes into account hesitation, repeated entries, and follow-up questions. the team can then revise the feedback and reexamine that specific point. the second attempt should reveal whether the change addresses the original problem or creates new difficulties. this hypothetical example illustrates why observation and root cause analysis are important in addition to measured values.
common mistakes in selection and contracting
avoid test questions that already reveal the correct procedure. even helpful facilitation can unintentionally obscure problems. check to see if participants with relevant experience and varying backgrounds are involved. another weakness is collecting many quotes without evaluating their significance for the product. have participants explain which observations occurred repeatedly, which arose only under specific conditions, and how these differences influence prioritization.
compare costs and schedules effectively
the costs and duration depend on the recruitment process, the product’s readiness, and the scope of the study’s analysis. have research, development, prototypes, and external expert services listed separately. also, specify which rounds of revisions are included and who approves the feedback. a timeline is only meaningful once these decisions and dependencies are known.
the briefing for an initial meeting
define the decision that should be possible after the test. provide a clearly described prototype status, prioritized tasks, and known uncertainties. list available user contacts and deployment environments. also agree on which data will be documented, who will receive the analysis, and who is responsible for design changes. for safety-critical applications, the study must be coordinated with the relevant technical test plan.
frequently asked questions about user testing and validation
how are test questions formulated?
describe the goal from the user’s perspective without specifying the correct way to use the product. a neutral task helps identify where guidance is lacking. review the wording in advance to ensure that misunderstandings of the task are not confused with problems with the product.
are online surveys the same as user tests?
they can capture expectations and opinions. however, they usually cannot be used on their own to determine whether people are able to perform a specific task. choose the method that best suits the question.
what are the rules for medical devices?
the fda guidance distinguishes between development-related studies and human factors validation. for a specific project, the target market, intended use, and responsibilities must be clarified with the individuals responsible for regulatory affairs.
how many people are needed for a user test?
there is no single number that works for all tasks. it depends on the problem statement, the user groups, and the level of detail required. please explain the rationale behind your choice. an exploratory test and a validation with formal requirements require different planning and evaluation approaches.
are remote tests sufficient?
they can be useful for certain digital tasks. in cases involving physical handling, environmental factors, or interactions with other devices, an on-site evaluation may be necessary. the method should be able to capture the relevant conditions, rather than simply facilitating the most convenient recruitment.
what should a good test report include?
the research question, product status, participant profile, tasks, and observations must be clear and traceable. include the significance of the findings, recommended changes, and open questions. a report is particularly useful when the development team can trace each recommendation back to a specific observation.
the perspective of f/p design
in the ux/ui process, we use early prototypes to evaluate the user experience. for a testing project, we first clarify the key decision point and the current state of the product. this determines which type of analysis is appropriate and which additional validation steps need to be planned separately. you can find more examples of our work in the industrial-design-portfolio.
are you planning a project in this area? talk to f/p design about your project, its current status, and your desired outcome. together, we’ll determine the right design approach and the next steps.
