AI in Business: When a Data Protection Impact Assessment Becomes Mandatory

Artificial intelligence is permeating everyday business life: chatbots in customer service, AI-assisted recruitment, automated credit decisions, predictive maintenance and personalised marketing. All of these applications have one thing in common -- they typically process personal data, often on a large scale and using new technologies. This is precisely where Art. 35 of the General Data Protection Regulation (GDPR) comes in: where processing is likely to result in a high risk to the rights and freedoms of natural persons, a Data Protection Impact Assessment (DPIA) must be carried out before the processing begins. This article explains when the DPIA obligation is triggered by the use of AI and how to design the process in a legally sound manner.

When Does the DPIA Become Mandatory?

Art. 35(1) GDPR requires a DPIA where a type of processing -- in particular using new technologies -- is likely to result in a high risk to the rights and freedoms of natural persons, taking into account the nature, scope, context and purposes of the processing.

The legislator identifies three situations where a DPIA is in particular required:

Systematic and extensive evaluation of personal aspects of natural persons based on automated processing, including profiling, on which decisions producing legal effects or similarly significantly affecting the individual are based
Processing on a large scale of special categories of personal data (Art. 9(1) GDPR) or data relating to criminal convictions
Systematic monitoring on a large scale of publicly accessible areas

All three situations can be relevant when deploying AI systems -- especially the first, which describes precisely what many AI applications do: automated evaluation and decision-making based on personal data.

The German DPA Blacklist: When AI Makes the DPIA Compulsory

What Is the Blacklist?

Art. 35(4) GDPR requires supervisory authorities to publish a list of processing operations that require a DPIA. In Germany, the Conference of Independent Data Protection Authorities (DSK) has published a joint blacklist (Muss-Liste) for the non-public sector. This list is legally binding: if processing falls within its scope, a DPIA must be carried out -- with no discretion.

Entries Relevant to AI

The DSK blacklist includes the following processing activities typically relevant to AI deployment:

Use of AI to process personal data to control interaction with data subjects or to evaluate personal aspects
Profiling to assess work performance, economic situation, health, personal preferences, interests, reliability, behaviour, location or movements
Scoring and evaluation of natural persons, particularly regarding creditworthiness or work performance
Combining data from different sources (data matching), especially where processing goes beyond the original purpose of collection
Processing of biometric data for the unique identification of natural persons

Practical Consequence

In most cases where AI systems process personal data, at least one of the listed entries will be met. The DPIA is therefore not the exception when using AI in business -- it is the rule.

High-Risk Scenarios: AI Applications Under the Microscope

Art. 22 GDPR gives data subjects the right not to be subject to a decision based solely on automated processing -- including profiling -- which produces legal effects or similarly significantly affects them. Typical AI use cases that may fall under Art. 22 include:

Automated credit decisions: AI assesses creditworthiness and decides on loan approvals or conditions
AI-assisted recruitment: Algorithms evaluate applications and make pre-selection or rejection decisions
Automated pricing: AI determines individual prices based on personal profiles (dynamic pricing)
Insurance scoring: AI evaluates risk profiles and determines premiums or rejections

In all these cases, not only Art. 22 GDPR must be observed but a DPIA under Art. 35 is also mandatory.

Processing of Biometric Data

AI systems for facial recognition, voice analysis or emotion detection process biometric data within the meaning of Art. 9(1) GDPR. Such processing is in principle prohibited unless one of the narrowly defined exceptions in Art. 9(2) GDPR applies. A DPIA is always required here.

Large-Scale Data Processing and Surveillance

AI systems that perform video surveillance with facial recognition, workplace behaviour analysis or employee productivity monitoring fall within both the blacklist and the illustrative cases of Art. 35(3) GDPR.

Generative AI and Large Language Models

The use of ChatGPT, Copilot and comparable systems in a company may trigger a DPIA where:

Employee or customer data is entered into the system
The system processes personal data from the company
AI outputs are used for decision-making about individuals
Employee usage data is systematically analysed

The Nine Criteria of the Article 29 Working Party

WP 248: The Assessment Standard

The Guidelines of the Article 29 Working Party (WP 248), endorsed by the European Data Protection Board (EDPB), define nine criteria to be used for the risk assessment:

Evaluation or scoring including profiling and prediction
Automated decision-making with legal effect or comparably significant impact
Systematic monitoring of persons
Processing of confidential or highly sensitive data
Large-scale data processing (large number of data subjects, large volumes of data)
Matching or combining of data sets
Processing data concerning vulnerable persons (employees, children, patients)
Innovative use or application of new technologies
Processing that prevents data subjects from exercising a right or using a service

The Two-Criteria Rule

The guidelines recommend: where at least two of these criteria are met, a DPIA should generally be carried out. AI applications typically satisfy several criteria simultaneously -- in particular criterion 1 (evaluation), criterion 2 (automated decision-making), criterion 6 (data combination) and criterion 8 (new technology).

DPIA Methodology: Step by Step

The DPIA must contain at least:

A systematic description of the envisaged processing operations and the purposes, including any legitimate interests pursued by the controller
An assessment of the necessity and proportionality of the processing in relation to the purposes
An assessment of the risks to the rights and freedoms of data subjects
The measures envisaged to address the risks, including safeguards, security measures and mechanisms to ensure the protection of personal data and to demonstrate compliance with the GDPR

Recommended Approach for AI Systems

Phase 1 -- Processing Description:

Describe the AI system, how it works and the data it uses
Document the data flows: what data is collected, how is it processed, where does it flow?
Record the parties involved: controller, processor, sub-processor
Identify the legal basis for each processing activity

Phase 2 -- Necessity and Proportionality Assessment:

Is the use of AI necessary for the intended purpose?
Are there less intrusive means that would also achieve the purpose?
Is the interference with data subjects' rights proportionate to the purpose pursued?

Phase 3 -- Risk Assessment:

Identify risks to data subjects (discrimination, erroneous decisions, profiling, loss of control over own data)
Assess the likelihood and severity of each risk
Consider the particularities of AI: lack of transparency (black-box problem), bias (discrimination from training data), data minimisation (does the AI process more data than necessary?)

Phase 4 -- Remedial Measures:

Define technical and organisational measures to mitigate risks
Document residual risks and their justification
Assess whether consultation with the supervisory authority under Art. 36 GDPR is necessary (where a high residual risk remains)

Documentation Requirements and the Role of the DPO

Comprehensive Documentation

The DPIA must be documented in writing and be available for presentation to the supervisory authority at all times. The documentation should include:

The complete DPIA including all four phases
The opinion of the Data Protection Officer (Art. 35(2) GDPR)
Where appropriate, the views of data subjects (Art. 35(9) GDPR)
Evidence of regular review and updating (Art. 35(11) GDPR)
All bases for decisions and their reasoning

Involvement of the Data Protection Officer

Art. 35(2) GDPR requires the controller to seek the advice of the DPO when carrying out a DPIA, where one has been designated. The DPO should:

Be involved early in the process -- ideally from the point of decision to introduce an AI system
Review the methodology and completeness of the DPIA
Provide an independent opinion
Monitor the regular review of the DPIA

The DSK Guidance on AI and Data Protection

The DSK guidance "Artificial Intelligence and Data Protection" of May 2024 is aimed at companies, public authorities and other organisations that deploy AI applications. It serves as a checklist and addresses typical data protection requirements in the AI context. In addition, the LfDI Baden-Wuerttemberg published a discussion paper on legal bases for AI deployment that serves as a practical working aid.

Consequences of Non-Compliance

Risk of Fines

Failure to carry out a required DPIA constitutes a violation of Art. 35 GDPR and can be sanctioned with fines of up to EUR 10 million or 2% of global annual turnover under Art. 83(4)(a) GDPR.

Further Consequences

The supervisory authority may prohibit the processing until a DPIA is available
Data subjects may claim damages under Art. 82 GDPR
The absence of a DPIA may be treated as evidence of an inadequate data protection organisation overall, triggering further investigations
Reputational damage through public announcements by the supervisory authority

Practical Recommendations

Use a DPIA Template

Use a structured template covering all requirements of Art. 35(7) GDPR. Some supervisory authorities provide their own templates. The open-source PIA (Privacy Impact Assessment) tool of the French supervisory authority CNIL is also recommended and freely available.

Preliminary Screening

Carry out a threshold assessment for every new AI project: check against the nine criteria of WP 248 and the DSK blacklist whether a DPIA is required. Also document cases where you conclude that no DPIA is necessary -- the accountability principle under Art. 5(2) GDPR requires this traceability.

Regular Review

Art. 35(11) GDPR requires a review of the DPIA, particularly where the risk of the processing changes. For AI systems that are continuously adapted through machine learning, this review is especially important and should be carried out at regular intervals -- at least annually.

Conclusion

The deployment of AI in business will in the vast majority of cases require a Data Protection Impact Assessment under Art. 35 GDPR. The DSK blacklist, the nine criteria of WP 248 and the illustrative cases of Art. 35(3) GDPR speak clearly: AI systems that process personal data are inherently high-risk processing operations. Companies that deploy AI without carrying out a DPIA risk not only fines but also a prohibition of the processing -- with serious consequences for business operations. The good news is that a carefully carried out DPIA is not an obstacle to innovation but an instrument that builds trust and makes risks manageable.

At compleneo, we support you in carrying out Data Protection Impact Assessments for AI systems, assessing your AI compliance and providing data protection guidance for your digitalisation projects. Get in touch with us.

AI in Business : When a Data Protection Impact Assessment Becomes Mandatory