Evaluating AI Vendor SOC 2 Reports for AI-Specific Control Gaps

SOC 2 Type II has become table stakes for selling AI software into the enterprise. It's the price of entry now. Roughly two out of three B2B buyers won't even take a meeting without one, and compliance posture has jumped into the top three vendor-selection criteria for most enterprise AI buyers, a sharp climb from where it sat back in 2024. Show up to a deal without a Type II report and the technical evaluation never starts, you're filtered out before anyone looks at what the product does.

But what if the report itself wasn't built to answer the questions AI buyers actually have? The standard hasn't been rewritten to close that gap. It's still the 2017 framework with points of focus refreshed in October 2022, and nothing newer has replaced it. Baker Tilly reports that ISO 42001, the NIST AI RMF, and the EU AI Act have not been incorporated into the SOC 2 framework, and that the AICPA has not issued AI-specific Trust Services Criteria. Which means a vendor can walk away with a clean opinion and never have had a single tested control around model accuracy, drift, or the integrity of its training data. The report exists. The confidence it seems to promise about AI-specific risk is not.

What the Trust Services Criteria test in AI systems

To understand why that gap exists, look at what the criteria were designed to measure. Five categories make up the Trust Services Criteria: Security, which is mandatory in every engagement, and then Availability, Processing Integrity, Confidentiality, and Privacy, which get scoped in depending on what the business actually does. That structure works fine for a system where behavior is fixed by code someone wrote and reviewed. It starts to strain the moment the system's behavior is generated on the fly by a model.

Three assumptions underlie the whole framework, and AI systems break all three. Controls can be documented and tested at one point in time. Systems process information "as intended," based on logic someone wrote down. And humans make the decisions about who gets access to what. None of that holds when an AI agent writes its own code and executes it without a person approving each run. The access controls that stop an unauthorized employee from touching production don't do anything to stop an agent that generates and runs its own instructions. Auditors have started flagging this explicitly: when an autonomous agent takes a privileged action and no human requested it, an accountability gap results, because SOC 2 expects privileged actions to trace back to an accountable person, not a service account with nobody behind it.

Processing Integrity, on paper, is the one criterion built to catch exactly this kind of thing. It's supposed to test whether processing is complete, valid, accurate, timely, and authorized, and for an AI system that should mean testing for drift, data poisoning, hallucination, biased output. In practice, it appears in scope less often than any other category. Most vendors only put Security in scope and leave model accuracy, data privacy, and AI governance completely untested. So the criterion that's most relevant to AI behavior is also the one buyers are least likely to see tested at all. None of this is a failure of diligence on anyone's part. It's a framework built for a different kind of system, being asked to do a job it was never scoped to do. When Processing Integrity is in scope, Baker Tilly (December 2025) treats model development, validation, versioning, and monitoring as the extra depth the criterion requires, but when it is out of scope, none of those controls are examined at all.

Auditor behavior shifting to compensate for what the criteria do not require

So auditors are adapting, without waiting for the AICPA to hand them a new rulebook. Two years ago, an auditor sitting down with an AI company asked pretty much the same questions they'd ask any SaaS vendor: firewall configuration, access logs, encryption at rest, backup procedures, incident response plans. That's changed. Baker Tilly (December 2025), Schellman, and Linford & Co. (May 2026) have each put the same four procedures at the start of AI company fieldwork: model lineage on demand, prompt and inference logging with redaction, evidence on LLM subprocessors, and drift and change management. Not one of those four is written into the Trust Services Criteria. Auditors added them because the old checklist stopped being enough.

The evidence bar is rising too. A static screenshot used to be fine. Now auditors want continuous-monitoring exports with timestamps, proof that a control was actually operating for the entire audit window, the day someone took a picture of the dashboard. As SureCloud's chief product officer Matthew Davies put it, auditors haven't been handed a new AI checklist, so they're improvising with what they already have. That's arguably harder to prepare for than a new rule would be. A fixed requirement, you can plan around. An auditor's judgment call, you can't always predict.

The depth of AI-specific testing varies widely by firm, and a clean Type II opinion from an auditor with no AI methodology is not comparable to one from a firm that tests drift monitoring, prompt injection, and agent accountability. One might have tested drift monitoring, prompt injection exposure, and agent accountability. Another might not have gone anywhere near any of it. Few CPA firms have published a methodology for AI SOC 2 as of 2026, so buyers can't assume methodological consistency across reports. What's driving the expansion isn't a rule change, it's buyer pressure. Confidentiality now appears in a substantially higher share of SOC 2 reports than it did in 2023, the CBIZ 2024 SOC Benchmark Study cited in SureCloud's guide found, and that shift traces to buyer pressure, not rule changes. Which raises the real question for anyone evaluating a vendor: if the cover page can't tell you what was actually tested, what do you look at instead? Auditors are asking materially harder questions about AI systems in 2026 without a formal rule change, and that inconsistency creates a new reader burden.

Model integrity and drift: what to look for when Processing Integrity is in scope

Start with the description of controls section. If Processing Integrity is listed in scope, the question worth asking is whether the auditor actually tested model versioning, drift monitoring, and output reliability, or whether they just checked data-processing completeness in the plain old SaaS sense: was the batch job complete, was the record valid, nothing more. Those are two very different audits wearing the same label.

Concept drift, where the relationship between inputs and outputs quietly shifts over time, and hallucination, where a model states something false with total confidence, aren't covered by conventional SOC 2 controls. Auditors are increasingly aware of this. Testing for it consistently is a separate matter. A rigorous auditor can be asked directly for model lineage on demand. A rigorous auditor names a specific model version running in production and asks the vendor to reproduce its full training history on the spot: the dataset snapshot, the code commit, the approval that pushed it live. If the vendor can't do that, it's a gap under PI1.2, and it appears before the audit even gets to firewalls. Ask for it. If a vendor's compliance team can't answer what model lineage evidence looked like, that tells you something.

Look also for drift-monitoring dashboards with real alerting, and change-management tickets showing every model deployment got approved, tested, and could be rolled back if it went wrong. That combination maps to CC8.1 under the Common Criteria. Why does any of this matter so much? Because hallucination isn't a rare edge case anymore. Stanford's 2026 AI Index looked at hallucination rates across 26 leading frontier models and found rates climbing as high as 94% under some evaluation conditions. Governance has to start from the assumption that any given output might be confidently wrong, and build verification, logging, and escalation around that assumption rather than around the hope that it won't happen.

So what does a buyer actually ask for? If Processing Integrity is in scope, request the specific tests run for model accuracy and drift. If it's out of scope, ask what compensating controls exist for output integrity instead. Baker Tilly treats model development, validation, versioning, and monitoring as the depth Processing Integrity ought to require for an AI vendor. A report that lists the criterion in scope but shows no trace of those tests should trigger a follow-up call. And for any vendor running autonomous agents: traditional change management assumes every change gets authorized, designed, documented, tested, and approved before it ships. Agent-generated code skips every one of those stages, because it executes at runtime with no human in the loop. A report that doesn't address that directly is incomplete for any vendor running agents, no matter how clean the opinion looks.

Training data governance: why traditional data controls leave content integrity unexamined

Move from what the model outputs to what it was trained on, and the gap gets wider, not narrower. Standard SOC 2 data controls answer one question well: who can access the training data. They say nothing about whether the content of that data is trustworthy. Access control and content integrity are not the same problem, and a report can nail the first while never touching the second.

That gap isn't hypothetical. Research out of CyLab in 2025 found that an attacker needs to touch only a small sliver of a pre-training dataset, not much of it at all, to run an effective data poisoning attack. Think about what that means for the threat model: someone with nothing more than read access, someone who can submit a handful of documents into a dataset a model trains on, can compromise that model without ever needing write access to a production system. Every access-control test in a standard SOC 2 report is built to catch unauthorized writes. This kind of attack doesn't need one. Training data is a genuinely new attack surface, and the traditional SOC 2 data controls simply weren't built to examine content integrity at all.

So what should a report show instead? Look for evidence that training data lineage, provenance, and content integrity are explicitly controlled and tested, beyond access logs that show who logged in and when. Vendor risk teams are already building this into their own questionnaires in 2026, writing training-data lineage requirements straight into vendor assessments and expecting the auditor to have tested the same thing. Model lineage on demand, the same procedure mentioned above, doubles as a useful signal here too: if an auditor can trace a deployed model back to its training data, that traceability serves as a proxy for content integrity having been considered. If they can't, that absence should be probed. Baker Tilly's approach applies the COSO framework and places data governance under Risk Assessment and Control Activities specifically. For any vendor processing training data at real scale, those COSO components should appear explicitly in the controls description, named, not implied.

Prompt injection exposure and inference logging: where the audit trail most often breaks down

Prompt injection sits in an odd spot. Adversarial input manipulates model behavior at inference time, and no existing Trust Services Criterion addresses it directly. Its consequences land squarely inside categories auditors already test: Confidentiality and Security. That's the practical hook for anyone evaluating a report.

The incidents are not abstract. CVE-2025-34291, found in the Langflow AI platform, let attackers steal authentication tokens through CORS and CSRF flaws, take over accounts, and then run arbitrary Python code. CVE-2025-62615, a critical server-side request forgery bug in AutoGPT, and CVE-2025-23317, an NVIDIA Triton flaw exploitable through a crafted HTTP request, tell a similar story: AI-specific vulnerabilities open attack surfaces that fall outside what traditional change-management testing was built to anticipate. These are illustrative cases from the past two years, not an exhaustive list, but they show the shape of the problem clearly enough.

This is why prompt and inference logging with redaction sits among the four procedures Baker Tilly, Schellman, and Linford & Co. all now run at the start of AI fieldwork. And it's where things go wrong more often than anywhere else in the audit trail. The common failure: teams log raw prompts and completions, PII and PHI included, with zero redaction applied before the log gets written. What was supposed to be an audit trail becomes a Confidentiality finding instead. Auditors now expect a specific logging structure for every inference: model version, a prompt hash or a redacted prompt, any tool calls made, the outcome. Missing that structure counts as both a Security gap and a Processing Integrity gap.

So when a report crosses your desk, ask two things. Does it describe prompt logging as an actual tested control, and does the evidence show redaction happening at the moment the log gets written, rather than as some cleanup step applied later. The underlying model-management layer should be checked too. And for any vendor calling a hosted model like OpenAI or Anthropic: if full prompt text gets logged before it ever leaves the system boundary, that vendor is carrying an unresolved tension between wanting a complete audit trail and protecting confidentiality. Ask how they resolved it. If they can't answer clearly, that's the answer.

LLM subprocessor chains: why the vendor's system boundary is rarely where buyers assume it ends

One might argue this is the blind spot buyers discover last, and often too late. When an AI vendor calls out to a hosted model, OpenAI, Anthropic, or any inference API, that provider is a vendor or subservice organization, not part of the vendor's system boundary, and the SOC 2 report's scope may not cover it. That provider is a subservice organization sitting outside the vendor's own system boundary, and the SOC 2 report in front of you may never actually cover it.

Schellman maps this to CC9.2, the supply-chain control criterion, for AI vendors specifically. The expectation is that the vendor has done its own risk assessment of the hosted model provider and can produce evidence of it. That's a reasonable ask. It's also one buyers rarely think to make, because the instinct is to assume the vendor's SOC 2 report covers everything the vendor's product touches. It doesn't, not automatically, and the boundary question is exactly the kind of thing that separates a report that looks thorough from one that actually is.