AI in Drug Development: FDA’s Draft Guidance and What Pharma Companies Must Do Now

On January 7, 2025, the FDA published what may be the most consequential regulatory document yet issued for pharmaceutical and biotech companies deploying artificial intelligence: a draft guidance titled Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products. It was the agency’s first formal attempt to define what it expects when AI outputs land in a regulatory submission.

If you are a drug sponsor, contract research organization, or biotech startup using AI to generate data or analysis that will appear in an IND, NDA, BLA, or any post-market submission, this guidance applies directly to your operations. Companies that fail to structure their AI use around FDA’s credibility framework risk rejection of submissions, requests for additional data, and — in the event of an FDA inspection — findings that could delay or derail a product entirely.

This post breaks down the key elements of the draft guidance, explains what the context-of-use framework actually requires in practice, describes how validation obligations scale with risk, and addresses the international regulatory picture — including the EU’s EMA and the ICH — that companies operating in global markets must simultaneously navigate.

Why FDA Issued This Guidance Now

The FDA did not act in a vacuum. By the agency’s own count, more than 500 drug or biologics submissions containing AI or machine learning components had been received since 2016, with particularly heavy concentration in oncology and neurology. AI is being used across virtually every phase of the drug development lifecycle: predicting trial outcomes, modeling pharmacokinetics, analyzing biomarker data, optimizing manufacturing processes, and synthesizing real-world evidence.

But the agency had no unified framework for evaluating the trustworthiness of those AI outputs. Different review divisions were applying different standards. Sponsors were uncertain what documentation to include. And there was a growing recognition — validated by several high-profile incidents in other sectors — that AI models can produce confident-sounding outputs that are deeply wrong in ways that standard software validation does not catch.

The January 2025 draft guidance was intended to fill that gap by establishing a risk-based credibility assessment framework applicable across the full drug product lifecycle.

What the Guidance Covers — and What It Does Not

The scope is important to understand precisely, because misreading it in either direction creates compliance problems.

The guidance covers AI models used to produce information or data that supports regulatory decision-making about drug or biological product safety, efficacy, or quality. This includes AI used in nonclinical studies, clinical trials, post-market surveillance, and manufacturing processes — wherever the AI output feeds into something submitted to FDA.

The guidance explicitly does not cover AI used in early drug discovery (target identification, molecule generation) that does not directly produce regulatory submission data. It also does not cover AI used for purely operational purposes — scheduling, logistics, administrative functions — that have no effect on patient safety, drug quality, or study reliability.

This distinction matters enormously for biotech startups. A company using a large language model to synthesize scientific literature as part of a competitive landscape review is almost certainly outside the guidance’s scope. A company using that same model to generate summaries of clinical adverse event data that will appear in a safety report has a much harder argument.

The practical guidance is: when in doubt about whether a particular AI application is within scope, apply the framework anyway. It is far less costly to over-document than to face a deficiency letter or inspection observation because you assumed an AI application was out of scope.

The Context-of-Use Framework

The centerpiece of FDA’s approach is the context-of-use (COU) framework, which requires sponsors to define precisely what question an AI model is being used to address and in what setting it is being applied.

This sounds straightforward but is more demanding than it appears. The FDA is not asking for a general description of what the model does. It is asking for a specific statement that ties the model to a particular regulatory question and population — the kind of statement that would allow a reviewer to evaluate whether the model is actually appropriate for the use being claimed.

For example, a submission that describes an AI model as “a machine learning classifier for adverse event detection” is not an adequate context-of-use definition. An adequate definition would specify that the model is a gradient-boosted classifier trained on Phase III clinical trial safety data for Drug X in adult patients with Indication Y, used to predict serious adverse event risk within 30 days of initiation, intended to support the benefit-risk narrative in Section 5 of the NDA.

This level of specificity is not bureaucratic formalism. It is what allows the FDA reviewer — and the sponsor — to determine what evidence is actually needed to trust the model for this specific application.

Risk Assessment: Scaling Validation to What Is at Stake

Once the context of use is defined, the guidance requires sponsors to assess the risk associated with relying on the AI model’s output. The risk assessment considers two primary factors: the potential impact of model error and the degree to which the AI output is subject to human oversight before it affects a regulatory decision.

High-impact, low-oversight applications carry the highest risk and require the most extensive credibility evidence. An AI model used to generate the primary efficacy analysis in a pivotal trial — where model error could lead to an ineffective or harmful drug reaching patients — sits at the top of the risk scale. An AI model used to flag potential protocol deviations for human review, where every flag is independently verified before action is taken, carries substantially lower risk.

This scaling principle has immediate practical implications. Companies cannot use the same validation approach for every AI application. A lightweight AI tool that automates a routine data formatting step does not need the same documentation depth as a model that drives a key regulatory determination. Mapping each AI application to the appropriate risk tier — and documenting that mapping — is a necessary first step before any credibility assessment plan is written.

The Seven-Step Credibility Assessment Framework

FDA’s draft guidance articulates a seven-step framework for building and documenting model credibility:

Step 1: Define the question of interest. What specific regulatory determination is the AI model intended to support? This must be stated with the precision described above.

Step 2: Define the context of use. What are the specific conditions — patient population, data type, intended use setting — under which the model will be applied?

Step 3: Assess AI model risk. Given the question and context, what is the potential impact of model error? How much human oversight will exist between the model output and the regulatory decision?

Step 4: Develop a credibility assessment plan. What evidence will be collected to establish that the model is trustworthy for this specific COU? This plan must be proportionate to the risk level determined in Step 3.

Step 5: Evaluate the AI model. Execute the credibility assessment plan. This typically includes training data documentation, model performance evaluation, and testing on independent datasets.

Step 6: Address model limitations. Identify and document the model’s known failure modes, edge cases, and performance gaps. FDA expects sponsors to be forthcoming about limitations, not to paper over them.

Step 7: Communicate credibility evidence. Prepare the documentation that will be included in the regulatory submission — the package that lets the FDA reviewer evaluate and trust the model’s contribution.

The FDA is clear that this is a framework, not a checklist. How robustly each step is executed depends on the risk level. But all seven steps apply to all AI applications within scope.

Validation Requirements in Practice

For pharmaceutical and biotech companies, the most operationally demanding requirement is the validation obligation embedded in Steps 4 and 5. FDA’s guidance makes clear that standard software validation approaches borrowed from 21 CFR Part 11 or GMP quality systems are not sufficient for AI models.

AI models can fail in ways that traditional software does not. They can overfit to training data and perform poorly on new populations. They can encode biases present in training datasets. They can be sensitive to seemingly minor changes in input distribution. And they can degrade over time as the real-world environment shifts away from the conditions under which they were trained.

External validation is particularly important for AI. FDA explicitly notes that holding out an entire clinical trial dataset for testing — rather than using a portion of the same data used in training — provides more meaningful validation evidence. Independent laboratory data, data from a different geographic region, or data from a different time period can all serve this function.

For generative AI components in particular, reproducibility is a serious challenge. Unlike deterministic algorithms, many generative models produce different outputs for identical inputs depending on initialization conditions. If your AI generates a clinical summary, an adverse event narrative, or a statistical analysis plan, and that output cannot be exactly reproduced from the underlying data, you have an audit trail problem. FDA inspectors expect to be able to trace any regulatory submission data to its source — including AI-generated source data.

Companies should implement version control for AI models (including logging which model version produced which output), document training data provenance, and maintain logs of all model outputs that feed into regulatory submissions.

EU EMA and ICH: The International Dimension

Companies operating in global markets must simultaneously satisfy the FDA’s framework and the European Medicines Agency’s evolving requirements — and work toward compliance with ICH guidance that is designed to harmonize the two.

In January 2026, the FDA and EMA jointly published ten guiding principles for the use of AI and machine learning in drug development, signaling deepening regulatory cooperation. The core principles — well-defined contexts of use, explainability, validation evidence proportionate to risk, transparency about limitations, and human oversight — closely parallel the FDA’s draft guidance, which was intentional.

The EMA has its own evolving framework for AI in medicines development. EMA’s guidance emphasizes explainability requirements that are in some respects more demanding than FDA’s current draft. European regulators have been influenced by the EU AI Act, which classifies certain high-risk AI applications in healthcare and requires conformity assessments and transparency obligations that go beyond US requirements. Companies pursuing simultaneous US and EU submissions need to map their credibility documentation against both frameworks.

ICH’s ongoing work — particularly through the IGSN (International Good Statistical Note) working group addressing statistical methodology for AI — is aimed at producing harmonized guidance that both agencies will adopt. Until that guidance is finalized, sponsors should build their AI validation approaches around the most demanding requirements of any jurisdiction in which they intend to submit.

The practical approach for global companies: structure your credibility assessment plan around FDA’s seven-step framework, ensure your explainability documentation meets EMA’s higher threshold, and track ICH developments to identify any additional harmonization requirements before they become mandatory.

What Companies Must Do Now

FDA’s comment period on the January 2025 draft closed April 7, 2025. Final guidance is expected in 2026. But waiting for finalization is a mistake. FDA reviewers are already applying the credibility framework concepts in their review of submissions, even before the guidance is finalized. Companies that show up to a pre-NDA meeting without a coherent story about how their AI outputs were validated are already experiencing the consequences.

Here is what should be happening in your organization right now:

Conduct an AI inventory. Map every AI application in your development program and categorize it as within-scope or out-of-scope under the guidance. For every within-scope application, assess the risk tier. This inventory is the foundation of everything else.

Build your context-of-use statements. For each within-scope AI application, draft a context-of-use statement that meets FDA’s specificity standard. This document is not a regulatory filing — it is an internal working document that drives everything downstream. Get it right before you need to.

Develop proportionate credibility assessment plans. For high-risk applications, engage your biostatistics, data science, and regulatory affairs teams jointly. The credibility assessment plan is a technical document with significant legal implications — it is the document an FDA inspector will ask for first.

Review your vendor contracts. If you are using third-party AI tools in your development program, your contracts almost certainly do not contain the provisions FDA’s framework implicitly requires: the ability to audit validation documentation, access to training data provenance, notification rights for model updates, and cooperation obligations in the event of an inspection. Address these gaps now.

Engage FDA early. The agency’s Type B meeting process exists precisely for situations where novel methodological questions need to be addressed before a major submission. If you are planning to use AI in a pivotal trial — particularly as the source of primary efficacy data — a pre-NDA or pre-BLA discussion specifically addressing your AI credibility approach is advisable.

The Bottom Line

FDA’s January 2025 draft guidance on AI in drug development is not a bureaucratic inconvenience. It is the agency’s clearest statement yet of what trustworthy AI-supported regulatory science looks like. Companies that internalize the credibility assessment framework — defining specific contexts of use, conducting risk-proportionate validation, documenting limitations honestly, and maintaining audit-ready records — will be better positioned not only for regulatory approval but for the post-market world, where AI-related deficiencies discovered after approval carry far higher stakes.

The companies that treat this guidance as a compliance checkbox rather than a scientific and operational standard will find that the difference shows up at exactly the moment it matters most: in the review division, or worse, in an inspection.

This post is for general informational purposes only and does not constitute legal advice. Reading this post does not create an attorney-client relationship. If you have questions about your specific situation, consult a qualified attorney.