Artificial intelligence models—particularly large language models (LLMs) and other generative systems—are now embedded across sectors, from customer service to cybersecurity to productivity tools. Yet their development and deployment raise foundational questions under the GDPR: When does an AI model contain personal data? Can legitimate interest justify training on personal data? What happens if a model was trained unlawfully?

To address these issues, the Irish Data Protection Commission requested a harmonized interpretation from the European Data Protection Board (EDPB). The result is Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models, adopted on 17 December 2024. The Opinion provides the clearest EU‑level guidance to date on how GDPR applies to AI model training and operation.

The Opinion is explicit about the stakes: AI models “raise a number of wide‑ranging data protection concerns… which impact data subjects across the EU/EEA” (p. 6). It also emphasizes that GDPR is not an obstacle to innovation but a framework that “encourages responsible innovation” (Executive Summary, p. 1).

This article analyzes the Opinion’s answers to the four questions posed by the Irish DPC and explores their implications for developers, deployers, and policymakers.

1. When Can an AI Model Be Considered Anonymous?

1.1 Why the question matters

If an AI model is anonymous, GDPR does not apply. If not, the full GDPR regime—including legal basis, transparency, data subject rights, and accountability—remains in force. The Opinion rejects the idea that training on personal data automatically produces an anonymous model.

1.2 The EDPB’s core conclusion

The Board states that AI models trained on personal data cannot be presumed anonymous. Instead, anonymity must be demonstrated through a rigorous, evidence‑based assessment.

The Opinion is unequivocal:
“AI models trained with personal data cannot, in all cases, be considered anonymous” (Executive Summary, p. 1).

To qualify as anonymous, two conditions must be met:

• The likelihood of extracting personal data from the model—directly or probabilistically—must be insignificant.
• The likelihood of obtaining personal data through queries, intentionally or unintentionally, must also be insignificant.

Both conditions must be evaluated “taking into account all the means reasonably likely to be used” (Executive Summary, p. 1).

1.3 What regulators will examine

Supervisory authorities (SAs) will expect detailed documentation demonstrating:

• Data minimization during training
  Measures to prevent or limit the collection of personal data, including filtering and deduplication.
• Reduction of identifiability
  Techniques to reduce memorization and prevent the model from retaining personal data.
• Resistance to extraction attacks
  Evidence of testing against inversion, membership inference, and prompt‑based extraction.
• Controls on model outputs
  Guardrails, filtering, and monitoring to prevent personal data leakage.

Controls on model outputs are essential because even a well‑designed model with strong data‑minimization practices can still leak personal data through its responses. The EDPB treats output controls as a distinct layer of protection that operates at the point of risk closest to the data subject: the moment the model generates text. Guardrails—whether implemented through fine‑tuning, reinforcement learning, or safety layers—help ensure that the model refuses to answer prompts seeking personal information about identifiable individuals. These guardrails can also prevent the model from generating sensitive inferences, reproducing rare training examples, or responding to prompts that could trigger memorized content. By constraining the model’s behavior at inference time, controllers reduce the likelihood that personal data will surface even if some residual memorization remains.

Filtering mechanisms add another layer by scanning outputs before they reach the user. These filters can detect patterns associated with personal data—names, phone numbers, addresses, or other identifiers—and block or redact them. They can also identify high‑risk categories such as health information, political opinions, or minors’ data. Output filtering is especially important in interactive systems, where users may unknowingly elicit sensitive content or where the model may hallucinate personal details. Effective filtering demonstrates to regulators that the controller is actively preventing the model from disclosing information that could harm individuals or violate GDPR principles.

Monitoring completes the picture by providing ongoing oversight of how the model behaves in real‑world use. Continuous monitoring can detect unusual patterns, such as repeated attempts by users to extract personal data or unexpected model behaviors that suggest memorization. It also allows controllers to identify and correct issues that were not apparent during testing. Monitoring logs, incident reports, and periodic evaluations form part of the accountability record that supervisory authorities expect. Together, guardrails, filtering, and monitoring show that the controller is not relying on static assurances but is actively managing the model’s behavior throughout its lifecycle.

The Opinion provides a “non‑prescriptive and non‑exhaustive list of methods” that controllers may use to demonstrate anonymity (Executive Summary, p. 1–2).

1.4 When anonymity can be lost

Even if a model is initially considered anonymous, certain factors may cause it to fall back under GDPR:

• Fine‑tuning that reintroduces personal data
• New extraction techniques
• Deployment contexts that enable re‑identification
• User prompts that elicit memorized data

The Opinion explicitly asks SAs to consider “factors which would cause the operation of the final AI Model to no longer be considered anonymous” (p. 7).

2. Can Legitimate Interest Justify Using Personal Data to Train AI Models?

2.1 No hierarchy of legal bases

The Opinion reiterates that controllers must choose the correct legal basis; legitimate interest is not a fallback or a default.

“There is no hierarchy between the legal bases… it is for controllers to identify the appropriate legal basis” (Executive Summary, p. 2).

2.2 The three‑step test

The EDPB applies the established three‑step test for legitimate interest:

1. Identify the legitimate interest

The interest must be:

• Lawful
• Clearly and precisely articulated
• Real and present—not hypothetical

Examples the Opinion considers legitimate include:

• Developing a conversational agent
• Improving threat detection in information systems

2. Necessity test

Controllers must show:

• The processing is genuinely needed to pursue the interest
• No less intrusive alternative exists
• The volume of personal data is proportionate to the purpose

SAs should pay “particular attention to the amount of personal data processed and whether it is proportionate” (Executive Summary, p. 2).

3. Balancing test

This is the most complex step. SAs must consider:

• The nature of the data
• The context of collection
• Reasonable expectations of data subjects
• Potential consequences, including downstream harms
• Whether the data was publicly available
• The relationship between data subject and controller
• The source of the data and privacy settings

The Opinion emphasizes that AI technologies are complex and that data subjects may not understand how their data is used. This makes reasonable expectations especially important.

The Opinion notes that SAs may consider “whether data subjects are actually aware that their personal data is online at all” (Executive Summary, p. 3).

2.3 Mitigating measures

Mitigating measures can help rebalance interests but cannot replace a legal basis. They must be tailored to the model and use case.

Examples include:

• Filtering scraped data

Filtering scraped data is about reducing the volume and sensitivity of personal data before it ever enters the training pipeline. When developers remove obvious identifiers, exclude sensitive categories, or discard data from contexts where users would not reasonably expect reuse, they materially reduce the likelihood that the model will memorize or reproduce personal information. This step also helps address the fact that scraped datasets often contain mixed, low‑quality, or unlawfully collected data. By applying filters—whether rule‑based, model‑assisted, or manual—controllers can show regulators that they took proactive steps to minimize the intrusion on data subjects’ rights and to avoid training on data that should never have been collected in the first place.

• Reducing retention of training data

Reducing retention limits the window during which personal data remains accessible within the training environment. Long‑term retention increases the risk of unauthorized access, repurposing, or accidental reuse in future training cycles. By contrast, short retention periods—paired with clear deletion protocols—demonstrate that the controller is not holding personal data longer than necessary and is actively reducing the risk of downstream misuse. This measure also supports the argument that the model’s behavior, rather than the raw data, is the primary output of the training process. Regulators will view disciplined retention practices as evidence of compliance with the storage‑limitation principle and as a meaningful safeguard in the balancing test.

• Limiting model capabilities

Limiting capabilities is a way to constrain the model so that it cannot be used in ways that would create heightened risks for data subjects. This may include restricting the model’s ability to answer questions about identifiable individuals, preventing it from generating sensitive inferences, or disabling features that could be repurposed for profiling or surveillance. These constraints can be implemented through architectural choices, fine‑tuning, safety layers, or output filters. By narrowing what the model can do, controllers reduce the likelihood that personal data will be exposed or that the model will be used in contexts that exceed users’ reasonable expectations. Regulators will see capability limitations as a sign that the controller has considered the risks of deployment and taken steps to prevent harmful or unexpected uses.

• Strengthening human oversight

Human oversight ensures that automated outputs do not operate unchecked, especially in contexts where decisions may affect individuals’ rights or interests. Oversight can take many forms: human review of outputs, escalation paths for high‑risk queries, monitoring for anomalous behavior, or policies requiring human intervention before decisions are made. The EDPB consistently emphasizes that meaningful human involvement can mitigate risks associated with opacity, unpredictability, and potential bias in AI systems. In the legitimate‑interest balancing test, strong oversight mechanisms help demonstrate that the controller is not delegating consequential decisions to an opaque model and is maintaining accountability for how personal data is used.

• Providing enhanced transparency

Enhanced transparency addresses the information asymmetry between controllers and data subjects. Because AI models are complex and often difficult for users to understand, controllers must provide clear, accessible explanations of how the model works, what data it processes, and how outputs may affect individuals. This may include layered notices, in‑product disclosures, or model cards that describe risks and limitations. Transparency strengthens the controller’s position in the balancing test by aligning processing with users’ reasonable expectations. When individuals understand how their data is used—and have the opportunity to exercise their rights—the impact on their privacy and autonomy is reduced, making legitimate interest more defensible.

Mitigating measures “should not be confused with the measures that the controller is legally required to adopt anyway” (Executive Summary, p. 3).

3. Post‑Training: Can Legitimate Interest Justify Processing Inside the Model?

The EDPB’s analysis makes clear that the legal basis used during training does not automatically carry over into deployment. Once a model is operational—whether embedded in a chatbot, integrated into a business workflow, or exposed through an API—it begins a new phase of processing with new risks, new data flows, and new impacts on individuals. Because of this, controllers must reassess their legal basis, including whether legitimate interest remains appropriate. The Opinion signals that deployment is not a mere technical continuation of training but a distinct processing environment with its own compliance obligations.

3.1 Deployment Is a Separate Processing Activity

Training and deployment are not a single, continuous processing operation. The EDPB treats them as separate phases because they involve different purposes, different categories of data, and different risks to data subjects. During training, the controller processes large datasets—often scraped or aggregated—to create or refine the model. Deployment, by contrast, involves real‑time interactions with users, new inputs, and outputs that may affect individuals in concrete ways.

This separation has significant legal consequences. A controller cannot assume that a legal basis that was valid for training (such as legitimate interest) automatically justifies the processing that occurs when the model is used in production. Deployment may involve new purposes—such as customer support, content generation, or fraud detection—that require their own necessity and balancing assessments. It may also involve new categories of data subjects who were not part of the training dataset. The EDPB’s position reinforces that GDPR compliance must be evaluated across the entire AI lifecycle, not only at the point of model creation.

3.2 New Risks in Deployment

Deployment introduces new vectors for personal data processing:

• The model may reveal personal data memorized during training.
  Even if the developer attempted to minimize memorization, the model may still reproduce fragments of personal data when prompted in certain ways. This risk is heightened when users ask for specific individuals, rare events, or unique identifiers. The EDPB expects controllers to evaluate and mitigate this risk before deployment.
• Users may input new personal data during interactions.
  Once deployed, models often receive personal data directly from users—names, account numbers, health information, employment details, or other sensitive content. This creates a new processing operation that must be supported by a valid legal basis and accompanied by appropriate transparency measures. The controller must also ensure that this new data does not inadvertently become part of the model’s internal state or future training data without proper safeguards.
• Outputs may be used for profiling or automated decision‑making, triggering Article 22 GDPR.
  Many organizations deploy AI models to support or automate decisions about individuals—creditworthiness, hiring, fraud detection, or customer segmentation. If the model’s outputs materially affect individuals, the processing may fall under Article 22 GDPR, which imposes strict limitations and requires meaningful human oversight. The EDPB’s Opinion signals that controllers must anticipate these downstream uses and assess whether legitimate interest is appropriate in light of the heightened risks.

These risks illustrate why deployment cannot be treated as a low‑impact or purely technical phase. It is the moment when the model interacts with real people, and the consequences of its behavior become tangible. Regulators will expect controllers to understand these risks and incorporate them into their legal basis assessment.

3.3 Reasonable Expectations Become Even More Important

Users rarely understand how AI models process their data, what information the model may retain, or how their inputs may influence outputs. This asymmetry of knowledge makes the “reasonable expectations” element of the legitimate interest balancing test especially important during deployment. The EDPB emphasizes that transparency obligations increase once the model is operational, because individuals must be able to understand how their data is being used and what risks they may face.

In practice, this means controllers must provide clear, accessible information about how the model works, what data it processes, and how outputs may be used. They must also consider whether users could reasonably expect their data to be processed in this way. For example, a customer interacting with a support chatbot may not expect their conversation to be used for future model training or for profiling. If the processing goes beyond what users can reasonably anticipate, legitimate interest may not be an appropriate legal basis.

The EDPB’s emphasis on reasonable expectations reflects a broader trend in EU data protection: the more opaque or complex the technology, the greater the burden on the controller to justify its use and protect individuals’ rights. Deployment is where this principle becomes most visible, and controllers must be prepared to demonstrate that their use of AI aligns with what users can fairly expect.

4. What Happens if an AI Model Was Trained Unlawfully?

The EDPB’s treatment of unlawful training is one of the most consequential parts of the Opinion because it addresses a question that regulators, developers, and deployers have struggled with for years: If the training was unlawful, is the model itself “tainted”? The Board rejects any automatic, one‑size‑fits‑all answer. Instead, it outlines three scenarios and emphasizes that supervisory authorities must assess each case individually, taking into account the nature of the model, the presence of personal data, and the relationship between the development and deployment phases.

The Opinion underscores that supervisory authorities “enjoy discretionary powers to assess the possible infringement(s) and choose appropriate, necessary, and proportionate measures” (Executive Summary, p. 3). This means organizations cannot assume that a model trained unlawfully must be deleted, nor can they assume that deployment is automatically permissible. The outcome depends on the factual configuration of the model and the processing activities.

4.1 Scenario 1: Same Controller, Personal Data Retained

When the same controller both trained and deploys the model, and the model still contains personal data, regulators will examine whether the development and deployment phases constitute separate purposes. If they do, the unlawfulness of the training may or may not contaminate the deployment phase depending on the degree of separation between the purposes, the nature of the data retained, and the risks posed to data subjects.

Key considerations

• Whether the model still contains personal data that can be extracted or inferred.
• Whether the deployment purpose is distinct from the development purpose.
• Whether the controller can demonstrate a lawful basis for the deployment‑phase processing.
• Whether the controller took steps to mitigate the impact of the initial unlawful processing.

Why this matters

This scenario is especially relevant for organizations that trained models on scraped data without a valid legal basis. If the model memorized personal data, the controller may face enforcement not only for the training but also for any subsequent use. However, if the deployment purpose is sufficiently distinct and supported by a valid legal basis, the controller may still be able to operate the model—subject to corrective measures.

4.2 Scenario 2: Different Controller Deploys the Model

When a second controller deploys a model developed by someone else, the EDPB places a clear due‑diligence obligation on the deploying controller. They must assess whether the model was trained lawfully and whether personal data remains embedded in the model. This is a significant development: it effectively creates a GDPR‑based “supply chain responsibility” for AI models.

The Opinion states that controllers must “ascertain that the AI model was not developed by unlawfully processing personal data” (Executive Summary, p. 3). This assessment must be more or less detailed depending on the risks associated with the deployment.

Practical implications

• Organizations cannot rely blindly on vendor assurances.
• Procurement processes must include questions about training data sources, legal bases, and any prior regulatory findings.
• High‑risk deployments (e.g., profiling, decision‑making, biometric analysis) require deeper scrutiny.

Why this matters

This scenario will reshape commercial AI contracting. Vendors will need to provide detailed documentation about training data provenance, and customers will need to evaluate it. Failure to conduct this assessment could expose the deploying organization to enforcement—even if they did not participate in the unlawful training.

4.3 Scenario 3: Model Is Anonymized After Unlawful Training

The third scenario is the most favorable for controllers. If a model was trained unlawfully but is later truly anonymized, the EDPB concludes that the GDPR does not apply to the model’s operation. The unlawfulness of the initial processing does not “infect” later processing, provided the model no longer contains personal data.

The Opinion states: “If it can be demonstrated that the subsequent operation… does not entail the processing of personal data, the GDPR would not apply” (Executive Summary, p. 4).

Key points

• Anonymization must be robust, documented, and defensible.
• The model must not allow extraction or inference of personal data.
• Any new personal data processed during deployment remains subject to GDPR.

Why this matters

This scenario provides a path forward for organizations that discover defects in their training data after the fact. If they can demonstrate that the model no longer processes personal data—through technical measures, testing, and documentation—they may continue using it. However, the bar for demonstrating anonymity is high, and regulators will expect rigorous evidence.

5. Practical Implications for AI Developers and Deployers

The Opinion’s practical implications extend far beyond the four questions posed. It effectively sets expectations for how organizations must design, document, and govern AI models throughout their lifecycle. These expectations apply not only to large technology companies but also to enterprises integrating third‑party models, startups building domain‑specific systems, and public‑sector bodies deploying AI tools.

The EDPB’s guidance reinforces that GDPR compliance is not a one‑time exercise but an ongoing obligation that spans data collection, model training, evaluation, deployment, and monitoring. It also signals that supervisory authorities will expect a higher level of technical literacy and documentation from controllers than in earlier waves of GDPR enforcement.

5.1 Implications for Model Developers

Developers of foundation models, LLMs, and domain‑specific AI systems must adopt a more structured and defensible approach to data governance. The Opinion makes clear that claims of anonymity, legitimate interest, or low risk must be backed by evidence—not assumptions.

Key expectations

• Document data sources and filtering
  Developers must maintain detailed records of where training data came from, how it was collected, and what filtering or deduplication steps were applied.
• Justify the legal basis with precision
  Legitimate interest requires a clear articulation of the interest, a necessity analysis, and a balancing test—not generic statements about innovation.
• Implement memorization‑reduction techniques
  Regulators will expect evidence of efforts to prevent the model from retaining personal data, including architectural choices. Developers are expected to treat memorization‑reduction as a core design requirement, not an afterthought. The EDPB’s opinion makes clear that regulators will look for concrete, technically grounded steps showing that the model was built and tuned to avoid retaining personal data in the first place. This begins with architectural choices: using training objectives, tokenization strategies, and model configurations that reduce the tendency to memorize rare or unique strings; applying data deduplication and aggressive filtering before training; and incorporating regularization techniques that discourage overfitting. These design‑time decisions demonstrate that the developer understood the risk of memorization and took steps to minimize it before the model ever saw personal data.
• Conduct extraction‑resilience testing
  Testing for inversion, membership inference, and prompt‑based extraction is no longer optional. This testing is becoming a baseline expectation because these attacks reveal whether an AI model still contains or can reveal personal data. The EDPB’s position means organizations can no longer rely on theoretical assurances that a model “does not memorize data.” They must produce evidence—through structured, repeatable testing—that the model does not leak personal information when probed. This marks a shift from passive compliance to active verification. Regulators now expect controllers to understand how their models behave under adversarial conditions and to demonstrate that they have evaluated and mitigated the risk of personal data resurfacing through outputs.

These tests each target a different failure mode. Inversion attacks attempt to reconstruct training data from model parameters; membership inference attacks try to determine whether a specific person’s data was included in the training set; and prompt‑based extraction uses natural language queries to coax the model into revealing memorized personal details. Together, they form a comprehensive assessment of whether personal data is still embedded in the model. The EDPB’s message is that without this testing, a controller cannot credibly claim that a model is anonymous or that it complies with data minimization and integrity obligations. This expectation will likely reshape internal governance, procurement standards, and vendor documentation across the AI ecosystem.

• Maintain accountability records
  Documentation must be sufficient for supervisory authorities to evaluate compliance.

Why this matters

Developers who cannot demonstrate compliance risk enforcement actions, forced retraining, or restrictions on deployment. The Opinion also anticipates future EDPB guidance on anonymization and data scraping, suggesting that expectations will continue to rise.

5.2 Implications for Organizations Deploying Third‑Party Models

Deployers face a different set of challenges. They may not control the training data or the model architecture, but they are still responsible for ensuring that the model they use complies with GDPR. The Opinion effectively requires deployers to treat AI procurement like any other high‑risk vendor relationship.

Key expectations

• Conduct due diligence on training data legality
  Organizations must evaluate whether the model was trained lawfully, especially if it was trained on scraped data.
• Assess whether the model retains personal data
  Deployers must understand whether the model is capable of revealing personal data through outputs.
• Re‑evaluate the legal basis for deployment
  Deployment is a separate processing activity requiring its own legal basis and balancing test.
• Implement transparency and oversight measures
  Users must understand how their data is processed, and organizations must monitor outputs for personal data leakage.

Why this matters

This shifts the compliance burden significantly. Organizations cannot rely solely on vendor claims; they must independently assess the model’s compliance posture. This will likely lead to new contractual requirements, audit rights, and technical disclosures in AI procurement.

5.3 Implications for Policymakers and Compliance Teams

The Opinion signals a more harmonized enforcement landscape across the EU. It also aligns closely with the EU AI Act, which imposes additional obligations on high‑risk AI systems. Compliance teams must therefore integrate GDPR and AI Act requirements into a unified governance framework.

Key expectations

• Prepare for increased scrutiny of web‑scraped data
  Supervisory authorities are likely to focus on data provenance and the legality of scraping practices.
• Align GDPR compliance with AI Act obligations
  Risk management, documentation, and transparency obligations will overlap.
• Anticipate future EDPB guidance
  The Board has already announced forthcoming guidelines on anonymization, pseudonymization, and data scraping in generative AI.

Why this matters

Organizations that treat GDPR and the AI Act as separate compliance regimes will struggle. The Opinion makes clear that data protection principles are foundational to AI governance and will shape enforcement priorities for years to come.

6. Conclusion

The EDPB’s Opinion 28/2024 rejects simplistic assumptions about anonymity, clarifies the limits of legitimate interest, and establishes a structured approach to assessing the consequences of unlawful training. It also signals that supervisory authorities will expect rigorous documentation, technical safeguards, and accountability throughout the AI lifecycle.

As AI systems become more powerful and more deeply integrated into business operations, organizations must treat this Opinion not as abstract guidance but as a practical roadmap for GDPR‑compliant AI development and deployment.