Biometrics under the EU AI Act

There are significant differences between the approaches of the three European institutions. This article shines a light on how the proposals compare and interact with GDPR and draws attention to areas of uncertainty, particularly in relation to financial services use of AI for fraud prevention. The article also touches upon the high-risk classification of emotion recognition and biometric categorisation tools under the EU AI Act and how this links to special category data under GDPR.

Simply by looking at the number of biometric data related definitions, it is evident that the AI Act puts significant emphasis on systems using biometric or biometric-based data and has a more sophisticated approach to regulating these systems. The GDPR has only one definition relating to biometrics (biometric data under Article 4(14)). On the other hand, the Commission’s proposal for the AI Act contains 6 biometrics related definitions (biometric data, emotion recognition system, biometric categorisation system, remote biometric identification system, real-time remote biometric identification system and post-remote biometric identification system).

The EP position paper adds three further definitions to this (biometric-based data, biometric identification, biometric verification) and expands the definition of “biometric categorisation” to include inferences derived from biometric data. Finally, the Council adds a r definition of “general purpose AI”, which covers image and speech recognition systems that could constitute biometric data in a relevant context.

Some of these definitions are familiar from the GDPR or past opinions of authorities, whereas others are new and still being developed.

The Good And The Bad Biometrics

Neither the AI Act nor the different EU institutions treat all biometric systems the same. For example, the Commission considers biometric categorisation to be only a “high-risk” AI system, whereas the EP considers it to pose an unacceptable risk and bans it (with certain excepted use cases for therapeutic purposes). On the other hand, the Council has taken biometric categorisation systems out of high-risk AI systems and only imposes transparency obligations on them.

The strictest approach has come from the EP, which has expanded the list of banned biometric AI systems and upgraded some others into the high-risk category. It has also distinguished biometric verification systems (1:1 matching) from other biometric and biometric-based identification systems (1 to many matching), considering them to be a lower risk AI system in comparison. Hence, one can think of biometric verification systems as being ‘the good biometrics’ to some degree.

Real-time remote biometric identification has been the star of the show for all three institutions’ debates, which strongly varied in views on the carveouts from bans for such AI systems in publicly accessible spaces. The starting point, from the Commission was targeted at banning Law Enforcement use, with three exceptions: finding victims of crime (including missing children), prevention of imminent threats (such as terrorist attacks) and detection and localisation of criminals facing criminal charges punishable by at least three years’ imprisonment.
The Council expanded these carveouts for law enforcement. The conversations also coincided with the French parliament’s plans to deploy facial recognition technology in public spaces for the Paris 2024 Olympics.

The EP proposal text went in the opposite direction of the Council and banned use of real-time remote biometric identification in public spaces altogether (the ban would affect both private and public entities).

We have provided an Appendix at the end of this article comparing the biometric related provisions in the texts from the three institutions.

What About Biometric AI Systems Used For Fraud Prevention In Financial Services?

Much to the glee of financial services institutions, Annex III – paragraph 1 (5)(b) of the EP text provides a carve out for fraud prevention AI systems from the high-risk systems list: “AI systems intended to be used to evaluate the creditworthiness of natural persons or establish their credit score [are high risk], with the exception of AI systems used for the purpose of detecting financial fraud”.

However, it is unclear whether the Act aims to limit this exemption to fraud systems only used in assessing consumer creditworthiness and credit score, or if it would extend to other fields of financial services such as the payments sector, where fraud prevention is also required for strong customer authentication and transaction monitoring.

Recital 37 of the EP Text states “… AI systems provided for by Union law for the purpose of detecting fraud in the offering of financial services should not be considered as high-risk under this Regulation.” However, currently there is no Union law that expressly provides for use of AI for fraud detection in financial services, even though its use is encouraged by some regulators. The draft Payment Services Regulations does contain a recital (recital 103) which says: “To be able to prevent ever new types of fraud, transaction monitoring should be constantly improved, making full use of technology such as artificial intelligence.”

Separately, given the EP text classifies “biometric and biometric based systems” to be high risk under Annex III, it is also unclear whether “biometric identification systems used for detecting financial fraud” would get the freedom of other AI systems used in detecting financial fraud, or would be batched with biometric identification systems as high-risk. Whilst the recital 33 makes a distinction between one to many and one to one biometric systems, this distinction is not echoed in Annex III.

Special category data under Article 9 GDPR vs the EP Text on banned and high-risk biometric and biometric-based systems

One of the new amendments included in recital 33 (recital 33a) proposed by the EP shows how special category biometric data under GDPR has influenced the high-risk classification under the EU AI Act.

(33a) As biometric data constitute a special category of sensitive personal data in accordance with Regulation 2016/679 [the GDPR], it is appropriate to classify as high-risk several critical use-cases of biometric and biometrics-based systems. AI systems intended to be used for biometric identification of natural persons and AI systems intended to be used to make inferences about personal characteristics of natural persons on the basis of biometric or biometrics-based data, including emotion recognition systems, with the exception of those which are prohibited under this Regulation should therefore be classified as high-risk…

However, biometric data does not always constitute special category data under the GDPR, as the Parliament text seems to assume. Article 9(1) GDPR considers biometric data to be special category data only when it is used for the purpose of uniquely identifying a natural person.

Where the Parliament classifies emotion recognition systems as being either high risk or prohibited, the same processing under the GDPR may not even constitute special category data (i.e. because recognising emotion on a face does not necessarily require the unique identification of the individual).

Whilst emotion recognition systems may suggest mental state of the individual, which would constitute health data and be special category data under GDPR, Art.9 would not prohibit the processing because of the biometric data, but rather because of the processing of derived health data.

Similarly, biometric categorisation systems (banned by the EP text) may allow detection of sensitive data such as an individual’s political orientation in some uses (Facial recognition technology can expose political orientation from naturalistic facial images) . Such processing would not be prohibited under the GDPR on the grounds of processing biometric data, but rather because of processing data revealing political opinions, which is special category data.

We would expect this nuance to be raised during trialogue to explain that biometric categorisation and emotion recognition are not necessarily special category data (prohibited processing) under the GDPR but are classified as banned or high-risk practices under the AI Act. This would show a divergence between the two laws which market participants need to respect.

What is next?

Trialogues have begun; it is expected that the final version of the AI Act will be agreed before the end of 2023.

APPENDIX

Banned Biometric AI Systems under the EU AI Act - Comparing the 3 texts

High Risk Biometric AI Systems under Annex III