Europe develops guidelines to improve mHealth app quality

Introduction

In February 2016, the European Commission appointed a working group to draft mHealth assessment guidelines. These mHealth assessment guidelines are intended to establish a soft law reference framework for mHealth apps, providing a framework of safety, quality, and reliability, and effectiveness criteria, to improve the use, development, recommendation and evaluation of mHealth apps on the European market.

The large number of lifestyle and wellbeing apps available on the European market with no clear evidence in relation to their quality and reliability has raised concerns about the ability of consumers to assess their usefulness. In addition, taking into account that the quality of data is essential to link apps to electronic health records and for effective uptake in clinical practice, some further assurances as to the quality of the app were needed.

The working group includes representatives of patients, health professionals and providers, payers, industry, academia and public authorities, representing hereby all interested parties in the consultation process.

The mHealth assessment guidelines are foreseen to be drafted in four iterations, with each followed by further stakeholder engagement. The second iteration, the version which forms the subject of this article (hereafter ‘second draft’) and which was published at the end of May 2016, refined the content of the first draft, which was presented at an open stakeholder meeting at the beginning of May (hereafter ‘first draft’)¹.

The last iteration of the guidelines is set to be produced at the end of December 2016. The feedback is to be included in the final report, which is expected on 25 January 2017.

EU regulatory landscape for mHealth apps

The draft currently under review provides further guidance as to the EU regulatory framework applicable to mHealth apps, focusing on medical devices, data protection and consumer protection legislation, as well as voluntary EU-level activities. Ongoing EU-level voluntary activities in relation to mHealth apps include:

• the privacy code of conduct for mHealth apps which is being developed under the initiative of the European Commission and which was formally submitted for comments to the Article 29 Data Protection Working Party at the beginning of June 2016;
• the EU-wide PAS 277:2015 Health and wellness apps – Quality criteria across the life cycle – Code of practice (‘PAS277’)2 standard, which was developed by the British Standards Institution (‘BSI’); and
• the international Standard IEC 82304-1, which is being prepared by a Joint Working Group of IEC subcommittee 62A (Common aspects of electrical equipment used in medical practice) and ISO technical committee 215: Health Informatics³.

As the framework stands now, there is no EU-wide guidance below the so-called ‘medical device level’ for mHealth apps, other than general consumer protection requirements as provided under the Consumer Rights Directive⁴, Ecommerce Directive⁵ and Unfair Commercial Practices Directive⁶.

While the CE certification process for medical devices provided within the EU medical devices legislation addresses the reliability and validity of health apps, no such guidance is provided for apps that are not covered by the medical devices definition, creating legal uncertainty for such apps.

Currently, for the purpose of delineating between apps that qualify as medical devices and apps that do not, the MEDDEV 2.1/6 Guidelines⁷ provide some guidance for developers. These Guidelines were updated this July 2016 after a process of consultation between the various interested parties, namely, competent authorities, commission services, industry and notified bodies in the medical device sector. The revision had as its aim the clarification of the definitions in relation to standalone software medical devices used in healthcare, and the alignment of the previous guidance with the work carried out in the context of the International Medical Devices Regulatory Forum (‘IMDRF’).

Notably, the criteria specified in the MEDDEV 2.1/6 Guidelines are now expressly said to be applicable to mobile applications and introduced the term ‘software as a medical device,’ which is defined as software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device⁸. Moreover, these updated.

Guidelines now provide in Annex I an overview of illustrative examples of qualification and classification for software used in the healthcare sector. The illustrative examples include (i) hospital information systems, (ii) decision support software, (iii) information systems, (iv) communication systems, (v) web systems for monitoring of data and in vitro diagnostic software laboratory Information systems (‘LIS’) & Work Area Managers (‘WAM’).

These illustrative examples and the updated decision diagrams to assist on (i) the qualification of software as medical device or (ii) the qualification of standalone software as an in vitro diagnostic device should further aid app developers in determining whether their app would qualify as part of a medical device or in vitro diagnostic medical device, accessory to a medical device or an in vitro diagnostic medical device, as standalone software, or not as a medical device.

The second draft recognised that in this regard there is a need to deal with the so-called grey zone as the distinction between what falls within and outside the definition of a medical device is not always clear.

Scope of the second draft guidelines

The second draft further delineates the scope of the guidelines on mHealth apps, proposing a supplementary voluntary guidance for apps in health and social care. The current proposed scope is broad, covering all mHealth apps that are not medical devices, as well as so-called grey zone apps which just fall below the lowest category of medical devices, namely Class 1. The idea would be that for the latter, the mHealth assessment guidelines could propose some additional voluntary assessment criteria in addition to the existing EU legislation on medical devices. 

Off-label uses of apps, i.e. apps being used in a health or social care context for which they were not originally intended, remain out of scope.

Target groups

As to the target groups of the mHealth assessment guidelines, the second draft broadened the initial target group considered to likely benefit from using the guidelines currently under development, to include app aggregators (app stores, app certifiers, etc.), in addition to citizens, mHealth developers, healthcare professionals, and healthcare systems (including hospitals, public authorities and health insurance providers). At this stage the list of those included within the target group is not meant to be exhaustive.

Expectations of the guidelines

The current draft describes in this regard the identified shortcomings of the current situation versus the needs or expectations for the mHealth assessment guidelines in a high-level manner. The identified expectations of the guidelines include among others:

• for citizens to have a simple tool to decide which app to use - citizens cannot be expected to go through all the scrutiny questions but just want to know that the appis ‘safe’ and ‘effective’;
• for mHealth developers to create value from the perspective of app developers and possibly align their apps with international standards;
• for app aggregators to identify the good from the less-good apps;
• for healthcare professionals to assess each app for their own use, positioning the app in the context of evidence of clinical effectiveness; and
• for healthcare systems to assess, validate and/or certify public providers or private third party assessments.

The working group recognised that these expectations need to be further analysed, and that it needs to develop practical recommendations on how these guidelines can best be used in practice by app developers. These more practical insights are expected to be reflected in the next version of the mHealth assessment guidelines.

App evaluation process

For the actual evaluation process of the apps envisaged by the mHealth assessment guidelines, the draft builds further on the three-stages approach provided within the first draft, namely (i) initial validation of the app, (ii) a risk assessment that determines the appropriate level of scrutiny for the app, and (iii) scrutiny of both the technological and the medical aspects of the app.

Within these three stages, nine criteria (also referred to as ‘assessment domains’), which have been identified as relevant to the development process, are to be assessed. These criteria are that the app is: (i) usable and accessible, (ii) desirable to use, (iii) credible, (iv) transparent, (v) reliable, (vi) technically stable, (vii) safe, (viii) effective, and (ix) private and secure. These criteria were identified based on an analysis of existing assessment frameworks relevant for the assessment of mHealth apps.

The intention of the working group is that for each stage, the final guidelines will include a list of questions intended to assist the developer/supplier in their evaluation process.

The questions for the (i) initial validation and (ii) scrutiny stage were already quite developed in the first draft and are set out in Section 8.3 of the second draft:

• The questions on the initial validation of the app cover the initial information gathering and validation; and
• The detailed scrutiny questions set out in Section 8.3 of the second draft cover the nine assessment domains as described above. Notably, this detailed list of questions is just one of the possible models of assessing scrutiny. Indeed, the working group indicated that the assessment domains as well as the methodologies/tools for applying the main assessment criteria, which will involve a combination of a scoring system and of mandatory pass/fail questions⁹, are still under revision and will be further elaborated within the next draft.

The approach to the risk assessment is also still being worked on by the working group. It is intended that the risk assessment procedure will be included within the next draft of the guidelines, which are expected to be released in October 2016.

Nonetheless, the working group has already indicated that globally it already agrees on the risk assessment to be applied, but that nonetheless some examples of the risk assessment approach are still being reviewed and discussed, for example the fact that accessing electronic health records might be high risk if there is a possibility of data modification/data extraction, etc. In any case it is already agreed between parties that the risk assessment should not emulate the complexity of the medical devices risk assessment and should preferably have the form of a (user-friendly) decision tree.

Certification

A notable new addition to the second draft is the inclusion of requirements for so-called app certifiers, either public or private bodies, who could carry out third party certification in relation to compliance with these guidelines. 

The second draft indicates that for private initiatives there should be either a certification process for certifiers or at least random inspections of these certifiers by official bodies to ensure these certifiers themselves adhere to the appropriate standards. More specifically, the second draft sets out a proposal of criteria and methodologies that these third party certifying bodies could use when developing their own certification schemes or to certify the certifiers. These criteria include (i) independence, (ii) goals of the analysis, (iii) depth of the analysis, (iv) methods of the analysis, (v) quality of the analysis methods, (vii) quality management, and (viii) transparency.

What are the next steps?

Currently, the second draft of the mHealth assessment guidelines is open for input and comments from all interested parties, with the deadline for responding being 31 August 2016. 

The aim is to produce the third iteration in mid-October 2016, while the final iteration is set for the end of December, with feedback to be included in the final report, set to be published on 25 January 2017.

The possibility to comment on these iterations of the draft mHealth assessment guidelines is an opportunity not to be missed as it is clear that these guidelines, albeit voluntary, will have a huge impact on the mHealth industry and the future development of mHealth apps in Europe.

This article was first published in eHealth Law & Policy (September 2016 07).

References

1. European Commission, First draft of EU guidelines on assessment of thereliability of mobile health applications.
2. This standard has been suggested as a basis for the standardisation action to be taken by CEN (Technical Committee 251 Health Informatics).
3. This international standard is expected to be published by the end of 2016.
4. Directive 2011/83/EC on consumer rights (repealing 97/7/EC).
5. Directive 2000/31/EC on certain legal aspects of information society service, in particular electronic commerce, in the internal market.
6. Directive 2005/29/EC concerning unfair business-to-consumer commercial practices in the Internal Market.
7. MEDDEV 2.1/6, ‘Guidelines on the qualification and classification of standalone software used in healthcare within the regulatory framework of medical devices.’
8. Definition of the IMDRF/SaMD WG/N10FINAL:2013: ‘Software as a Medical Device (SaMD): key definitions.’
9. The score would involve calculating a risk-related score for each app. Apps failing a mandatory question or not reaching a sufficiently high score would not be recommended/would be rejected.