The European draft regulation on AI : What about financial services ?

(Translated by DeepL)

On 21 April 2021, the European Commission unveiled its draft regulation on artificial intelligence (P-RIA) aimed at establishing harmonised rules for artificial intelligence (AI) applicable in all European Union Member States. With the P-RIA, the Commission wants to establish a regulatory framework that allows AI to be used in accordance with European values, while encouraging innovation. The P-RIA adopts a risk-based approach and establishes an ex ante and ex post oversight regime. AI providers and other actors designated by the P-RIA will therefore have to comply with the obligations set out therein before and after the AI system is placed on the market or put into service. In the event of a breach, the P-RIA provides for measures such as suspension or withdrawal and fines of up to EUR 30,000,000 for natural persons and up to 6 % of global turnover for companies.

Given the growing use of AI in finance, it is useful to take a look at the P-RIA to understand the legal issues for financial institutions, but also its potential impact in Switzerland.

Firstly, Article 3(1) P-RIA defines an AI system as software developed using one or more techniques or approaches listed in Annex I of the P-RIA and which is capable, based on defined objectives, of generating results such as content, predictions, recommendations or decisions that influence the environments with which the software interacts. The techniques and approaches listed in Annex I include machine learning approaches, logic- and knowledge-based approaches such as inductive logic programming, and statistical approaches. With this broad definition, the Commission aims to take account of technological advances and developments in the AI market.

The P-RIA then classifies AI systems according to their risks, namely unacceptable risks and high risks. The category of unacceptable risks includes AI systems designed to manipulate human behaviour in order to deprive citizens of their free will or those that allow states to socially rate their citizens (see Article 5 P-RIA).

For high risks, Art. 6(1) P-RIA covers any AI system (i) intended to be used as a safety component of a product or is itself a product covered by legislation listed in Annex II of the P-RIA and (ii) the product whose safety component is the AI system, or the AI system itself as a product, must undergo a third-party conformity assessment with a view to its placing on the market or putting into service, in accordance with the legislation referred to in Annex II. Finally, Article 6(2) of the P-RIA specifies that AI systems referred to in Annex III of the P-RIA must be considered as posing a high risk.

Annex III, section 5(b) considers that AI systems intended to assess the creditworthiness of natural persons or to establish their credit risk (credit score) represent a high risk. These AI systems can have significant consequences for individuals, as they can, for example, determine whether or not a mortgage or credit line is granted. These AI systems must therefore be developed and monitored in accordance with the P-RIA to ensure that every individual has fair access to financial services and to prevent discrimination based on personal characteristics. However, it is interesting to note that Annex III only covers the assessment of natural persons and not legal persons.

Any financial institution using or providing an AI system for the purpose of determining credit risk will therefore have to comply with the obligations set out in Chapters 2 and 3 of Title III of the P-RIA, in particular establishing a risk management system, appropriate data management and governance practices, technical documentation, keeping records of each event (logs), and enabling the monitoring of AI during its use. In addition, the AI system must be developed in such a way as to ensure a certain level of accuracy and resilience to cyber attacks or other misuse (see Bacharach, cdbf.ch/1164 on digital resilience).

In order to ensure consistent application of the P-RIA with financial regulations, the authorities responsible for supervising and enforcing these regulations will be designated as competent authorities for the purpose of supervising the application of the P-RIA. In addition, the procedure for assessing the compliance of AI systems and ex post supervision will have to be integrated into the obligations and procedures laid down in Directive 2013/36/EU.

On reading the P-RIA, it appears that AI systems that assess the creditworthiness or credit risk of individuals are considered to pose a high risk, as they may reinforce social inequalities or discriminate against certain categories of the population in economic terms. However, many AI systems used in finance do not fall into the category of high-risk AI systems. It is important to note, however, that if a financial institution uses a chatbot – a technology that does not fall within the category of high-risk AI systems – to interact with its customers, the latter must be aware that they are interacting with such a system due to the minimum transparency requirement set out in Article 52 of the P-RIA. Of course, Annex III may be amended in the future and other AI systems may be added.

Swiss financial institutions may have to comply with the future European regulation, which is unlikely to enter into force for several years. This regulation will have extraterritorial scope, since according to Art. 2(1) P-RIA, it will apply in principle (i) to suppliers placing an AI system on the European market or putting it into service, and (ii) to users of AI systems located in the European Union or in a third country, regardless of whether they are established in the European Union or in a third country.AI system on the European market or put into service for a user or for their own use, regardless of whether they are established in the European Union or in a third country, (ii) users of AI systems located in the European Union, or (iii) suppliers and users of AI systems located in a third country, when the results generated by the AI are used in the European Union.

With the P-RIA, the Commission is therefore planning to establish the first regulatory framework on AI with extraterritorial reach, hoping to emulate the regulatory impact that the GDPR has had in the European Union and other parts of the world.