The SAS 70 Problem, Again

How AI Assurance Standards Will Actually Emerge

Standards rarely show up because a committee sits down and says, “Hey, there’s a new category here—let’s write a standard for it.”

More often, the market starts using an existing standard to answer a question it was never designed to answer. If that question matters enough, the formal framework eventually catches up to what people are already doing in practice.

SAS 70 is the clearest precedent we have for this pattern.

‍

The SAS 70 pattern

SAS 70 was introduced in 1992 to report on controls at service organizations, with a specific focus on controls relevant to financial reporting. That was its entire mandate.

But as more technology functions moved to outsourced providers through the 1990s and 2000s, buyers started asking a different question. They didn’t just want to know whether a vendor’s controls supported accurate financial statements—they wanted to know whether they could trust that vendor with their systems and their data more broadly.

There was no purpose-built report for that question. So buyers reached for the one that existed. They asked for SAS 70 reports, even when the underlying concern had little to do with financial reporting.

Over time, SAS 70 became a proxy for something much broader than its original design. The gap between what the report was built to prove and what it was actually being asked to prove eventually became too wide to ignore—and the standards world responded.

SSAE 16 took over the financial-reporting lineage. SOC 2 was built around broader trust criteria: security, availability, processing integrity, confidentiality, privacy—the things buyers had been informally asking SAS 70 to cover all along.aicpa-cima+1

The standard didn’t get replaced because it failed. It got replaced because the market had already outgrown it, and the formal framework simply caught up to where practice had drifted.

‍

Where AI agents sit today

We may be at the early stage of the same process with AI agents.

Right now, when a customer evaluates an AI vendor, they ask for a SOC 2 report. That’s a reasonable starting point—it establishes that the vendor has controls around security and availability.

But the follow-up questions are where things get interesting. And they’re questions SOC 2 was never built to answer:

Which model version actually handled this specific interaction?
What tools did the agent call, and in what order?
What changed in the agent’s behavior when the underlying model was swapped?
What exactly did the agent do, step by step, in this particular run?
Which policies applied to this action, and were they current at the time?
If something goes wrong six months from now, can we reconstruct exactly what happened?

These are runtime questions about a specific interaction, not point-in-time questions about whether a control existed and operated correctly across an audit period. SOC 2 tells you the organization had a control environment. It doesn’t tell you what a particular agent did at 2:47pm on a Tuesday, using which model, under which policy, with which tools available to it.

‍

Why agents make this harder than traditional software

An AI agent is increasingly a runtime system rather than a fixed pipeline. Its path through a workflow isn’t necessarily known in advance. The model behind it can change—sometimes automatically, sometimes at the provider’s discretion. Tools can be selected dynamically based on context. Two interactions that look identical on the surface can produce genuinely different sequences of actions underneath.

That variability changes what “evidence” needs to mean. For traditional systems, evidence of a working control is largely retrospective and aggregate: the control was in place, it operated as designed, across the period in question. For an agent, that’s necessary but not sufficient. You also need to know, for this interaction: what context the system had, which rules applied at that moment, what the agent actually did, and what evidence survives afterward to reconstruct it.

Observability gets you part of the way there. A trace tells you what the agent did—which calls it made, in what order, with what inputs and outputs. That’s valuable, but it’s descriptive, not evaluative. A trace can show you the sequence of actions. It can’t tell you, on its own, whether the agent should have taken those actions given the policy and regulatory context in force at that time.

‍

The gap forming around SOC 2

Once agents start taking consequential actions—approving something, denying something, representing a position, executing a transaction—assurance starts to require a connection between four things:

‍

Element	What it means	Why it matters
Action	What actually happened	What was allowed
Policy	What was allowed	What was required
Regulatory context	What was required	What can be proven later
Evidence	What can be proven later	What actually happened

‍

This mirrors emerging requirements in the EU AI Act, NIST AI RMF, and ISO/IEC 42001, all of which call for traceable records of AI system behavior, the data and models in use, and the ability to reconstruct decisions on demand.

That connection is the gap. I think the next layer of AI assurance will form around exactly this space—not as a replacement for SOC 2, but as a layer that sits around it, the way SOC 2 eventually sat around what SAS 70 used to be asked to do. The pieces will likely include:

Model and tool lineage — which model and which tools were actually in play for a given interaction, not just which were configured as available.
Runtime evidence — a record of what happened in a specific interaction, not just aggregate control operation over a period.
Regulatory context at the moment of action — the specific rule set in force when the agent acted, not the rule set in force when the audit was performed.
Continuous evaluation — checking behavior against policy on an ongoing basis rather than sampling at a point in time.
Reconstructability — the ability to go back, months later, and rebuild exactly what an agent did and why it was allowed to do it.

If the SAS 70 history is a reliable guide, this won’t announce itself as a standards body declaring the old framework insufficient. It will look like buyers quietly asking SOC 2 reports to prove things they were never built to prove—and the standard either stretching to cover the gap or a new layer forming alongside it once the gap gets too wide to paper over.

I think that’s already starting to happen with AI agents.

What should companies do while this layer is still forming?

Start capturing the pieces this layer will eventually require, even informally:

Which model and tools handled each interaction
What policy applied at the time
Evidence that can be pulled up on demand later

Waiting for a formal standard means having no answer when a customer, regulator, or auditor asks the question six months from now.

This is the problem Carver is building infrastructure around: a regulatory trust center, continuous evals, and observability tagging that connect an agent’s actions to the policy and regulatory context governing them, with evidence that holds up on reconstruction. Learn more at carveragents.ai

‍

FAQ

Is this arguing that SOC 2 is inadequate for AI companies?

No. SOC 2 remains the right answer to the question it was designed for—whether an organization’s security, availability, and confidentiality controls are appropriately designed and operating. The argument is that a second, adjacent question is emerging: what did a specific agent do, under what policy, and can it be reconstructed—and that question needs its own evidence layer.

Why can’t SOC 2 just be extended to cover this?

It might be, over time, the way SSAE 16 and SOC 2 eventually absorbed what SAS 70 had informally been asked to do. But that historical process took years and required the underlying practice to mature first. In the meantime, the runtime, per-interaction nature of agent behavior is a different shape of evidence than a point-in-time control audit is built to produce.

What’s the difference between observability and assurance here?

Observability—traces, logs, spans—tells you what an agent did. Assurance requires also knowing whether it should have done it, based on the policy and regulatory context that applied at that specific moment. A trace without that connective layer is a record without a verdict.

Why use the SAS 70 to SOC 2 transition as the analogy specifically?

Because it’s a documented case of a standard being stretched past its original scope by market demand, rather than being designed top-down for a new category. It suggests that new AI assurance frameworks are more likely to emerge from vendors and buyers pushing on the edges of SOC 2 than from a standards body pre-emptively defining “AI agent assurance” from scratch.

How does Carver fit into this?

Carver’s regulatory trust center, evals, and observability tagging are built to capture exactly this connective evidence—linking an agent’s actions to the policy and regulatory context in force at the time, with a record that can be reconstructed later. It’s built as a layer around existing assurance frameworks like SOC 2, not a replacement for them.

‍

Sources: AICPA / SSAE 16 materials; ISACA Journal; CliftonLarsonAllen; Kaufman Rossin; Roosa CPA; SCL; Atlantic.net; KirkpatrickPrice; Drata; Linford & Co.; NIST AI RMF; EU AI Act; ISO/IEC 42001.aicpa-cima+1

https://www.aicpa-cima.com/resources/download/2017-trust-services-criteria-with-revised-points-of-focus-2022

‍