AI Agent Liability: What Changed in September and What to Do About It

TL;DR
In September, AI liability showed up in bank risk assessments, insurance policies, state regulation, and court cases. The question behind all of them is the same: can you show an AI agent was operating within the boundaries you intended? If agents are on your 2027 roadmap, you need to know which obligations apply, keep an audit trail, and watch for drift.

Something shifted in September

For most of this year, AI liability lived in think pieces and conference panels. It was a real concern, but a distant one. Over the last few weeks, the conversation has moved into places that can change how companies operate: bank risk assessments, insurance policies, state regulation, and litigation involving alleged real-world harm from AI systems.

This isn't only a conversation for model providers. For companies deploying AI agents into real workflows, the questions are getting practical:

Who is responsible when an agent acts?
Can you prove what it was supposed to do?
Can you show the system was governed appropriately when something went wrong?

What moved in September

Several developments point in the same direction:

Banks flagged risks from AI shopping agents. They cited fraud, privacy, and consumer-protection concerns, areas where organizations have limited tools for managing autonomous systems.
Regulators are asking whether agentic AI needs new oversight. Many existing frameworks assume a human reviews an AI recommendation before taking action.. That assumption is harder to apply when the agent can act on its own.
New York is putting the RAISE Act into operation. On September 21, Governor Hochul announced that starting in November, large frontier AI developers will be directed to register with the state. From January 2027, they must comply with transparency, safety-framework, and incident-reporting requirements, including reporting critical safety incidents within 72 hours to a new office within the Department of Financial Services. The law targets large frontier developers, not the companies that deploy their models. Its wider significance is that state-level rules can shape how compliance programs are designed across jurisdictions. (Governor's announcement)
Courts are seeing AI harm cases. Whatever the outcomes, questions of AI-related responsibility are now in litigation.
Companies are rethinking governance for "rogue" agents, particularly where an agent takes an action nobody explicitly anticipated or approved.
Insurers are addressing AI in cyber coverage, including exclusions and the scope of coverage for AI-related incidents.

None of these creates a new liability rule on its own. The useful signal is that regulators, banks, insurers, and companies are all starting to ask the same thing:

Can you demonstrate that an AI system was operating within the boundaries you intended?

For organizations deploying agents, a model-provider agreement alone makes that hard to answer.

What this means for your next roadmap cycle

If AI agents are on your roadmap, these questions are worth answering before an incident puts them on the agenda:

Obligations: Can you show what obligations an agent was operating under when it took a given action, and when those obligations were last checked?
Audit trail: If a regulator, customer, or insurer asked for one tomorrow, could you produce it without reconstructing events from application logs?
Drift: Have you defined what behavioral drift looks like for your agents? How would you know if behavior gradually moved away from the controls and compliance posture you approved?
Insurance: Does your cyber policy address autonomous AI activity? Are there exclusions or conditions that could affect coverage if an agent makes or executes a decision?

These are becoming part of the practical work of putting AI systems into production

What we're building at Carver

At Carver Agents, we're building Executable Legal Infrastructure (ELI) for AI agents, a layer that connects legal and regulatory requirements to what agents actually do. That means:

Discovering obligations that apply to an agent and its workflow
Mapping agent actions against those obligations
Running evaluations before an agent is deployed
Monitoring for behavioral and compliance drift after deployment

The goal is to make compliance something you can demonstrate continuously, rather than something you reconstruct after an incident.

There's a useful parallel with SOC 2. As cloud software became central to business operations, companies needed a practical way to show that controls were in place and working. AI agents create a similar need, with one important difference. An agent doesn't just store or process information. Depending on how it's deployed, it can make decisions, interact with customers, trigger workflows, and take actions across systems.

So the question isn't only whether you have an AI governance policy. It's whether you can produce evidence that the policy was reflected in the system's behavior.

‍

FAQ

Are we liable for what an AI agent does, even if we didn't build the model?
Potentially, yes. Using a third-party model doesn't automatically transfer responsibility for how an agent is deployed or what it does. Liability depends on jurisdiction, contracts, applicable regulations, system design, and the circumstances of the incident. For companies deploying agents into real workflows, governance, monitoring, and auditability matter more as a result.

Does it help that we're using a reputable provider like OpenAI or Anthropic?
It can be one part of showing that reasonable steps were taken, but it doesn't eliminate deployment risk. How the agent is configured, what permissions it has, what controls surround it, and how its behavior is monitored all matter. The model provider is one part of the overall risk picture.

Why does agentic AI need different rules than regular software?
Not necessarily completely different rules, but agents create governance challenges that existing frameworks don't always address cleanly. Traditional deployments often assume a human reviews an output before a consequential action. An agent may decide, call another system, or trigger a workflow on its own. Organizations then need to ask not only whether an output is accurate, but whether the action was authorized, compliant, and appropriate in that context.

What does New York's RAISE Act mean for us?
The RAISE Act is law. It requires large frontier AI developers to register with the state, publish safety and transparency frameworks, file risk assessments, and report critical safety incidents within 72 hours. Registration begins in November 2026, with compliance required from January 2027. It applies to developers, not deployers, so most companies deploying agents aren't directly covered. But it shows where state-level AI governance is heading, and it may affect your vendors and contracts. Check the law's scope against your own situation.

Will our cyber insurance cover something an agent does on its own?
Don't assume it will. Coverage depends on specific policy language, exclusions, conditions, and the circumstances of the incident. Ask your carrier or broker directly how the policy treats AI-driven actions and whether any exclusions apply.

What can we do right now?
Three things are a good start:

Know the obligations: Understand which legal and regulatory requirements apply to each agent and workflow.
Maintain an audit trail: Capture enough evidence to reconstruct what the agent was allowed to do and what it actually did.
Watch for drift: Monitor whether behavior keeps matching the controls and compliance posture under which the agent was approved.

The goal isn't to predict every failure. It's to make the system's behavior explainable and auditable when something does happen.

Where does Carver Agents fit in?
Carver Agents continuously discovers applicable obligations, connect them to agent behavior, evaluate agents before deployment, and monitor for drift once they're live. So when someone asks whether an agent was operating within its obligations when it acted, the answer doesn't have to depend on a manual investigation weeks later. It can be backed by evidence.

This article is for general information and is not legal advice. Consult qualified counsel about your specific obligations and insurance coverage.

Talk to us

If you're finalizing plans for the coming cycle, we can walk you through how Carver ELI maps obligations to agent behavior, supports audit trails, and monitors drift in production.

Book a 30-minute AI Liability Review

References

Governor Kathy Hochul, AI Safety: Governor Hochul Announces Next Steps to Regulate Major AI Developers and Protect New Yorkers (Sept 21, 2026)
Skadden, New York Enacts AI Transparency Law on Heels of White House Actions (Jan 2026)
Davis Wright Tremaine, New York RAISE Act: AI Safety Rules for Developers (Dec 2025)
RiskAI, Who Pays When AI Fails? Emerging AI Liability Trends (Sept 2025, background)
Carver Agents, Legal Infrastructure, Built Into Your Agent Lifecycle
Carver Agents, RegTeaming: The Missing Layer in AI Agent Evaluation

‍

United States
447 Broadway,
2nd Floor Suite #563,
New York 10013

Listen to Carver Conversations on Moltbook

AI Regulatory OS
Regulatory Intelligence
Regulatory Sources
Regulatory Platforms
Horizon Scanning
Regulatory Monitoring
Intelligence vs Compliance

Podcasts
Knowledge Base
Resources
Glossary
Use cases
Developers
Blog

Home
Legal Infrastructure
Regulatory Risk Intelligence
Technology
About Us

hello@carveragents.ai
General enquiries
support@carveragents.ai
Product support
legal@carveragents.ai
Privacy and legal requests
github.com/carveragents