Blog

1 Why data protection in usability studies is part of the study architecture

In many development projects, research questions, user groups, tasks and use scenarios are defined first. Data protection in usability studies often only comes into view when recruitment starts. By then, much has already been decided:

• which method is used,
• which user groups take part,
• whether testing takes place online, in a simulated context of use or in the real environment,
• which cameras, platforms and observation rooms are planned,
• who watches live and who receives recordings later.

Each of these decisions changes which personal data are generated and who sees them. Three examples show how quickly this happens: An additional international team wants to observe live shortly before the study starts. The video conferencing platform activates automatic transcription. Selected video sequences are to be shown later in an internal training session. In all three cases, the group of recipients, the service providers involved or the purpose of processing change.

The General Data Protection Regulation (GDPR) lists in Article 5, among others, lawfulness, transparency, purpose limitation, data minimization, storage limitation, and integrity and confidentiality as principles. No signature fulfills these principles. They must be implemented in the way the study is set up.

Change of perspective: What participants want to know

2 Why “user study” is not a classification

“User study” is an umbrella term. It can refer to an exploratory interview, a contextual observation, a formative usability evaluation, a summative usability evaluation, a post-market activity or a clinical investigation. These activities differ not only in method. They can also trigger different regulatory, ethical and data protection requirements.

A formative evaluation serves to identify strengths and weaknesses of the user interface as well as possible use errors during development. A summative evaluation, or human factors validation, by contrast provides evidence of safe use by representative users on a largely final design. Both are part of the risk-based process under IEC 62366-1 and the FDA Human Factors Guidance.

Whether a study with human participants is a clinical investigation within the meaning of the MDR does not depend on its name. In MDCG 2021-6, the Medical Device Coordination Group explicitly answers the question of whether usability tests are clinical investigations: Depending on the study design, they may or may not fall under the definition. What matters are the purpose, the scope and the question of whether participants are exposed to product-related risks. Where people are involved, the document recommends justifying and documenting why a usability test is not a clinical investigation.

The label “user study” therefore does not yet answer

• whether only use actions are observed or clinical data are collected,
• whether the product is actually applied to patients,
• whether participants are exposed to product-related risks,
• whether an ethics committee or authority must be involved.

3 Study types and their typical data protection questions

3.1 Exploratory studies and context analyses

Interviews, contextual interviews, workplace observations and workflow analyses help to understand the context of use. Personal data are generated even at this stage. Information on specialty, professional experience, role and institution can make a person indirectly identifiable in specialized fields.

Observations in hospitals and practices also reveal an information-rich work context: patient data on monitors, documents, name badges, conversations and people who happen to be present. Before starting, it should therefore be established which areas are observed, whether recordings are necessary and how unintentionally captured information is handled.

3.2 Formative evaluations

Formative studies are iterative. Findings from the first sessions change the design or raise new questions. This flexibility is methodologically intended but must be reflected in the data protection concept. If video recordings, additional observers, transcriptions or new screener questions are added after the first sessions, the data flow changes.

Therefore plan in advance which adjustments are covered by the original study purpose and from which point participant information or consent is reviewed again.

3.3 Summative evaluations and human factors validation

Summative studies require a high level of traceability. It does not follow, however, that everyone involved must know the identity of the participants. A robust data concept can, for example, provide for the following:

• Participants are managed via study IDs.
• Identity data and study data are stored separately.
• Contact data remain with participant management.
• The development team receives anonymized or summarized results and observation logs without names.
• Reports contain aggregated information on the sample.

The central question is: Which characteristics are necessary to make the suitability and representativeness of the sample traceable without unnecessarily extending the link to individuals? Regulatory traceability and data protection are not opposites. They need an appropriate data structure.

3.4 Real use activities

“Real use” is not a regulatory or data protection category of its own. A real use activity can be a contextual observation, a post-market activity, a PMCF activity or, under certain conditions, a clinical investigation. Early on, it needs to be clarified whether the product is actually applied to patients, whether it bears the CE mark and is in use within its intended purpose, and whether treatment data or non-recruited persons are captured.

4 Three settings, three data protection profiles

4.1 Online studies

Online studies often allow a clear separation between participants and the development team: study IDs instead of names, a limited group of observers, controlled provision of recordings. At the same time, they bring their own risks: external platforms, automatic cloud recording and transcription, subcontractors, server locations outside the EU, international observers, screenshots and private information in the participant’s background.

Before starting, you should therefore clarify:

• Which platform is used, and where does it store data?
• Which subcontractors are involved?
• Is automatic recording or transcription activated?
• From which countries do observers access the session?
• Are local recordings and screenshots excluded?
• What information can become visible when the screen is shared?

Online therefore does not automatically mean privacy-friendly. What matters are the platform, the configuration, the group of recipients and the actual data flow.

4.2 Simulated use studies

In a usability lab or a simulated clinical environment, much can be designed deliberately. Camera angles are planned, patient data are replaced by simulated data sets, participants and observers are separated spatially and observers are approved by name. The key planning questions are: Does the face have to be recorded, or are hands, product and user interface sufficient? Is audio needed for the root cause analysis? Which recordings does the client receive? Separating identity from observation is also possible in the lab if the research team conducts the sessions and live observation and recordings follow fixed rules agreed by contract.

4.3 Real use studies in a clinical setting

Hospitals and practices cannot be fully isolated. Additional risks arise from patient data on displays, recognizable wards, confidential conversations and professional dependencies. Proven protective measures are defined observation areas, limited camera angles, covering of patient-related screen content, omitting sound where it is not needed, and clear contact persons at the institution. Here, protection lies less in blinding than in transparency, purpose limitation and control of the data actually collected.

5 The study life cycle as a data flow

Data protection decisions are not made only in the session. They accompany the entire study.

5.1 Contact and recruitment

The first data flow begins long before the session. Where do the contact data come from? Who approaches the person, and what information do they receive in doing so? How can they decline further contact? If data are not collected directly from the person, additional information obligations apply under Article 14 GDPR, including on the origin of the data.

5.2 Screening and selection

Screeners are often the most detailed data set of a study. They capture qualification, experience, work environment, product knowledge and sometimes health-related exclusion criteria. For highly specialized healthcare professionals, even the combination of a few professional characteristics can make a person identifiable. It must therefore be decided which criteria are really necessary, who sees complete responses, whether the client selects individual persons and what happens to the data of persons who are not selected.

5.3 Conduct

Before the first session, the observation and recording concept should be established: Who moderates, who observes, is there a live stream? Are image, voice or screen recorded, and which image sections are visible? Can third parties or patient information be captured?

5.4 Analysis and data transfer

The transfer shows whether the information need was clarified early enough. Does the client receive raw data or results? Do videos stay with the research partner? Who has access to the mapping list, and via which channel are data transmitted? If an identifiable video is to go to further recipients, this must be clarified before recording and communicated to the participants.

5.5 Retention, deletion and further use

After the study, it must be determined which data are retained, deleted or anonymized and whether further use is planned. Inclusion in a test panel for future studies is a purpose of its own. Agreement to a single study does not automatically cover permanent storage for later invitations.

Anyone who knows the life cycle can ask the right questions in the right order. This is exactly what the following decision path is for.

6 Decision path: Seven questions to the right concept

A data protection concept should not emerge from a preferred template but from the actual need for insights.

1. Which insights are actually needed? Aggregated results, observation data, video or individual participant profiles? The more precisely the need is described, the more targeted the data can be determined.
2. Does the client need to identify individual participants? If aggregated information on qualification and experience is sufficient, the identity can remain with the research partner.
3. Which raw data are necessary? For many questions, observation logs, recordings of hands and product, screen recordings or selected sequences are sufficient. Not every analysis needs the same degree of identifiability.
4. Who decides on selection, recording, recipients and retention? These decisions determine the roles under data protection law. Part 2 of this series addresses this in detail.
5. Which separation is possible in the chosen setting? Online, strong separation is often easy. In on-site real use studies, blinding is generally not possible, and transparency and controlled data collection gain importance there.
6. In which countries are data collected, stored and accessed? Platform locations, service providers and the location of the observers are also relevant.
7. What happens after the study is completed? Deletion, retention, anonymization and possible further use should be established before the start.

Conclusion

Usability studies need real insights, and these insights generate personal data, often at the very first contact. The more realistic the setting, the more specialized the user group and the more international the conduct, the more important early planning of the data flow becomes.

Robust planning therefore does not begin with the consent template, but with the classification of the study, the setting and the question of who really needs which data. Participant information and consent then follow almost by themselves. And participants receive what they need for their decision to take part: honest and specific answers.

At USE-Ing., we plan recruitment, screening and study conduct so that data flow and participant information fit together from the start. If you are preparing a study with healthcare professionals, please contact us early.

Frequently asked questions about data protection in usability studies

Further reading

• HCP Recruiting for Usability Tests: Why the Wrong Participants Become a Problem
• Formative Usability Evaluation According to IEC 62366-1
• Human Factors Validation Tests in Line with FDA Requirements
• Healthcare Test Panel from USE-Ing.
• Wiki: Representative users, summative usability evaluation, context analysis

Sources

• Regulation (EU) 2016/679 (General Data Protection Regulation), in particular Article 4(5), Article 5, Article 9, Article 14 and Recital 26: https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng
• Regulation (EU) 2017/745 on medical devices (MDR), Article 2(45) (clinical investigation): https://eur-lex.europa.eu/eli/reg/2017/745/oj/eng
• MDCG 2021-6 Rev. 1: Regulation (EU) 2017/745 (Questions & Answers regarding clinical investigation), December 2023, Question 17: https://health.ec.europa.eu/system/files/2023-12/mdcg_2021-6_en.pdf
• BfArM: Clinical investigations under the MDR / MPDG: https://www.bfarm.de/DE/Medizinprodukte/Aufgaben/Klinische-Pruefungen-und-Leistungsstudien/Klinische-Pruefungen/_node.html
• Medical Devices Law Implementation Act (MPDG): https://www.gesetze-im-internet.de/mpdg/
• FDA: Applying Human Factors and Usability Engineering to Medical Devices (Guidance for Industry and FDA Staff): https://www.fda.gov/regulatory-information/search-fda-guidance-documents/applying-human-factors-and-usability-engineering-medical-devices
• EDPB: Guidelines 05/2020 on consent under Regulation 2016/679: https://www.edpb.europa.eu/documents/guideline/guidelines-052020-on-consent-under-regulation-2016679_en