Validation (German: Validierung) is the objective demonstration that a medical device meets the requirements of its intended purpose (intended use) and the needs of its intended users. In usability engineering, validation refers in particular to the demonstration that the final user interface can be used safely by the intended users under representative conditions of use.
Within IEC 62366-1, this demonstration is called summative evaluation. The FDA uses the term human factors validation testing for it and considers the safe and effective use of the product. Design validation, summative evaluation, and human factors validation are closely related but not entirely synonymous. Design validation according to ISO 13485 considers the medical device as a whole, whereas summative evaluation according to IEC 62366-1 specifically assesses the safe use of the user interface.
- The role of validation in the usability engineering process
- Distinction from verification and formative evaluation
- Representative users and contexts of use
- Critical tasks and risk-based scenarios
- Conducting summative studies
- Typical mistakes in practice
- Regulatory reference
- Frequently asked questions (FAQ)
The role of validation in the usability engineering process
Validation comes at the end of the design and development process, or at the end of the development of the user interface. It builds in particular on the following results:
- Use specification
- User profiles
- Context of use analysis
- Task analysis
- Use-related risk analysis
- Hazard-related use scenario
- User interface specification
- Formative evaluations
- Verification of the user interface
The primary purpose of validation is not to develop new design solutions or to optimize the user interface iteratively. Its goal is to obtain objective evidence of the suitability of the resulting product, that is, of the safe use of the final user interface.
This does not mean, however, that no previously unknown use problems may appear during validation. Observed use errors, close calls, and use difficulties must be investigated systematically, analyzed for their causes, and assessed with regard to the resulting risks.
Validation without sufficient prior analysis and formative evaluation increases the risk that fundamental weaknesses are recognized only at the end of development. If design changes are then required, their effects on verification, validation, and risk management must be assessed.
Distinction from verification and formative evaluation
Validation, verification, and formative evaluation pursue different goals. Verification asks: Do the design outputs meet the specified design inputs? Validation asks: Does the resulting medical device meet the requirements of its intended use and the needs of its intended users? Formative evaluation asks: What strengths, weaknesses, and previously unrecognized use problems does the user interface have, and how can it be improved?
| Activity | Guiding question |
|---|---|
| Verification | Do the design outputs meet the specified design inputs? |
| Validation | Does the resulting medical device meet the requirements of its intended use and the needs of its intended users? |
| Formative evaluation | What strengths, weaknesses, and previously unrecognized use problems does the user interface have, and how can it be improved? |
The distinction does not depend on the method used alone. A usability test can be used both formatively and summatively. What matters are the goal of the investigation, the development status of the product, the selection of tasks and scenarios, the representativeness of users and conditions of use, and the role of the results in the development process.
Formative evaluations can be carried out with early prototypes, partial functions, or alternative design variants. Changes and repetitions are an explicit part of their function. Summative evaluation, by contrast, takes place at the end of the development of the user interface. The test item must be final or sufficiently representative of the intended production product.
Technical verification of individual requirements does not replace validation. Conversely, a summative evaluation does not replace the required verification of technical and functional requirements.
Representative users and contexts of use
The participants of a summative evaluation must adequately represent the intended user groups. Relevant characteristics can include, for example: professional qualification, education and expertise, product experience, experience with comparable products, age, physical abilities, sensory abilities, cognitive prerequisites, language skills and health literacy, medical limitations, and role and responsibilities in the workflow.
Different user groups must not be combined merely because they use the same functions of the product. What matters is whether their abilities, experience, tasks, or contexts of use differ so much that different use problems or risks can result.
The use environment must also be adequately representative. This can include: lighting, noise level, space conditions, protective clothing or gloves, interruptions and distractions, time pressure, workload, emergency situations, mobility, parallel tasks, and collaboration with other people. Use does not necessarily have to take place in the real environment. A simulated use environment can also be suitable if the conditions relevant to safe use are reproduced realistically.
If mandatory training has been specified as part of the intended use, it must also be adequately taken into account in validation. The scope, format, and timing of the training, as well as any necessary interval between training and use, should correspond to real conditions.
Critical tasks and risk-based scenarios
The tasks and scenarios to be investigated are selected on a risk basis. For this purpose, IEC 62366-1 uses in particular hazard-related use scenarios, that is, use scenarios that can lead to a hazardous situation or to harm. The FDA uses the term critical task: a user task whose erroneous or omitted performance can lead to serious harm or could compromise medical care.
The two concepts overlap but are not entirely identical. A global human factors strategy should therefore document in a traceable way how hazard-related use scenarios and critical tasks were identified, reconciled with each other, and taken into account in validation. The current FDA guidance explicitly lists critical task identification as part of the preparatory analysis. The selection can be informed in particular by:
- Possible severity of harm
- Known or foreseeable use errors
- Results of the use-related risk analysis
- Experience with comparable products
- Complaints and post-market surveillance data
- Findings from formative evaluations
- Complexity of the task
- Dependence on perception, memory, or manual skills
- Effectiveness of the intended risk control measures
Not every single operating action has to be investigated to the same extent. The selection and, where applicable, grouping of scenarios must, however, be justified in a traceable way. Interactions with the instructions for use, labeling, quick reference guide, and training materials must be included if this information is necessary for safe use or serves as a risk control measure.
Conducting summative studies
A frequently used method is a usability test under simulated conditions of use. A robust validation plan should describe in particular:
- Goal and regulatory function of the evaluation
- Product to be investigated and software version
- Representativeness of the test item
- Intended user groups
- Inclusion and exclusion criteria
- Number of participants and rationale
- Intended use environments
- Tasks and scenarios investigated
- Training and accompanying materials provided
- Moderation and observation method
- Data collection
- Handling of assistance
- Criteria for classifying observed events
- Root cause analysis procedure
- Link to the use-related risk analysis
The test should be designed so that participants can act independently. Unintended cues from moderation, task wording, room layout, or the order of scenarios must be avoided as far as possible. The following should be recorded in particular: successful and unsuccessful task performance, use errors, close calls, use difficulties, use patterns and deviations, assistance required, understanding of safety-relevant information, and statements from the subsequent root cause analysis.
Observation results should not be assessed solely on the basis of the statement "task passed" or "task failed." Even a task that was formally completed successfully can contain a safety-relevant difficulty, for example if the user acts correctly only by chance or is close to a critical error.
Likewise, an observed use error does not automatically mean that the entire validation has "failed." Every relevant use error, close call, and use difficulty must be analyzed. The assessment covers: why the event occurred, whether the interface or the context of use contributed to it, what possible harm can result, whether the risk is acceptable, and whether additional risk control measures are required. Summative evaluation is thus not a purely statistical pass test. Its outcome is based on the combination of observation data, root cause analysis, and risk assessment.
Typical mistakes in practice
A common mistake is to understand validation as an ordinary usability test with a large number of participants. Other typical weaknesses are:
- Insufficiently defined user groups
- Non-representative participants
- Artificial or overly simplified use situations
- Non-representative product status
- Accessories and accompanying materials not taken into account
- Incomplete selection of critical tasks or hazard-related use scenarios
- Missing traceability to the use-related risk analysis
- Leading task wording
- Impermissible assistance from the moderator
- Unrealistic training conditions
- Insufficient consideration of learning and recall effects
- Missing definition of use error, close call, and use difficulty
- Evaluation based solely on success rates
- Acceptance criteria defined after the fact
- Superficial root cause analysis
- Premature classification of an event as "user error"
- Insufficient risk assessment of observed problems
- Design changes after validation without an appropriate impact analysis
Using a blanket error rate as the pass threshold is particularly problematic. A low error rate may not be sufficient from a regulatory perspective if the observed error can lead to serious harm. Conversely, a single error does not automatically have to lead to a complete repetition of the study. Decisive are the cause, the potential for harm, the risk assessment, and the measures derived from it. Subjective ratings such as satisfaction or perceived ease of use can also be collected as a supplement. They do not, however, replace objective observation of safe task performance.
Regulatory reference
ISO 13485:2016 addresses design and development validation in Section 7.3.7. Validation must be performed according to planned and documented arrangements and must demonstrate that the resulting product meets the requirements of its intended use. Where practicable, validation must be completed before release or delivery of the product. It is performed on representative products and, where applicable, must also take software validation and clinical evaluation or performance evaluation into account.
Design validation according to ISO 13485 is therefore broader than a summative evaluation of the user interface. Human factors validation can form an important part of design validation but does not automatically replace all other validation activities.
IEC 62366-1 describes summative evaluation as an evaluation at the end of the development of the user interface with the goal of obtaining objective evidence that the user interface can be used safely. The focus is on the selected hazard-related use scenarios and the effectiveness of the use-related risk control measures. Summative evaluation thus relates to the validation of the safe use of the user interface.
The FDA uses the term human factors validation testing. It expects an investigation of the final user interface with representative users under actual or appropriately simulated conditions of use. The FDA generally recommends at least 15 participants per distinct user population. This number is, however, not a general requirement of IEC 62366-1 and must not be applied uncritically to every user group in the European context. Methods and sample sizes overall must fit the product, the risk, and the regulatory objective.
As of October 2026, the FDA Quality Management System Regulation (QMSR) applies to the US market (effective February 2, 2026). The QMSR incorporates ISO 13485:2016 into 21 CFR Part 820. The design and development requirements result in particular from 21 CFR 820.10(c) in conjunction with ISO 13485, Section 7.3. Earlier references to 21 CFR 820.30(g) as a stand-alone current FDA requirement for design validation therefore no longer reflect the present regulatory structure.
The fundamental expectations of the FDA guidance "Applying Human Factors and Usability Engineering to Medical Devices" regarding the identification of critical tasks, risk control through design, and human factors validation remain in place.
Validation demonstrates that the resulting medical device meets the requirements of its intended use and the needs of its intended users. In usability engineering, the safe use of the final user interface is investigated through a summative evaluation or human factors validation.
The following applies: representative users + final user interface + representative contexts of use + risk-based scenarios. Validation is not a pure error count. What matters are systematic observation, root cause analysis, and the assessment of whether the remaining use-related risks are acceptable.
Frequently asked questions (FAQ)
Are validation and summative evaluation the same thing?
Not entirely. Design validation according to ISO 13485 considers whether the resulting medical device meets the requirements of its intended use. Summative evaluation according to IEC 62366-1 concentrates on the objective demonstration that the safety-relevant user interface can be used safely. Summative evaluation can therefore be an essential part of design validation.
Is human factors validation the same as summative evaluation?
Both concepts pursue a comparable goal and are often treated as equivalent in practice. The FDA uses the term human factors validation and focuses on critical tasks and the safe and effective use by intended users. IEC 62366-1 uses the term summative evaluation and considers hazard-related use scenarios and the safe use of the user interface. For global approvals, the respective requirements should be analyzed separately and taken into account in a joint study strategy.
Must validation always be performed with users?
Not every design validation necessarily requires a usability study. For the summative evaluation of a user interface, an investigation with representative users under representative conditions is often the suitable method. Manufacturers must select and justify a suitable method that can provide objective evidence of safe use.
How many participants are needed?
IEC 62366-1 does not specify a fixed minimum number of participants. The sample size must fit the method, user groups, tasks, and risks and must be justified in a traceable way. For summative evaluation (human factors validation testing), the FDA generally recommends at least 15 representative participants per distinct user population. This recommendation should not, however, be understood as a universal statistical rule.
May assistance be given during validation?
Unplanned assistance can impair the validity of the validation. If a moderator intervenes, it must be documented why the assistance was necessary, what information was given, how the assistance affected task performance, and how the event was classified and assessed. Intended help offered by the product may be taken into account only if it is part of real intended use.
Does every use error lead to failing validation?
No. There is not necessarily a simple pass or fail scheme. Every observed use error must be analyzed with regard to its cause and its possible effects. It must then be assessed whether the resulting risk is acceptable or whether additional measures are necessary.
What happens if a design change is made after validation?
Every change must be assessed for its effects on the user interface, user tasks, use-related risks, verification, and validation. Depending on the extent and safety-related significance, a documented impact analysis, an update of the use-related risk analysis, additional verification, formative evaluation, a partial repetition of the summative evaluation, or a complete new validation may be required.
Do you want to plan and conduct your summative evaluation or human factors validation in a regulatorily robust way? We support you from study planning and recruitment through execution to root cause analysis and documentation according to IEC 62366-1 and FDA requirements.
More about our usability engineeringSources
- IEC 62366-1:2015+AMD1:2020, Medical devices, Part 1: Application of usability engineering to medical devices
- ISO 13485:2016, Medical devices, Quality management systems, Requirements for regulatory purposes
- FDA Guidance: Applying Human Factors and Usability Engineering to Medical Devices
- 21 CFR Part 820, Quality Management System Regulation (QMSR)