Does your AI agent need legal testing?

The data says yes. We will do this (free) check if you share the test traces with us.

The Math Behind AI Agent Legal Risk

The math is simple: Assume after doing everything you have an infraction rate of 1 in 1000 (optimistic, BTW).

Each infraction costs 5K-100K excluding other costs (lawyer, exec time, penalty etc). It will take a few instances to reach 500K-1M$ cost.

If any of them leads to a lawsuit, the discovery will find all the infractions. This is why the AI insurance is 5-10x of cyber insurance.

‍

Loading…

Why We Ran the Test: A Customer Losing Sleep Over Their AI Agent

Not surprisingly, a customer told me they were losing sleep over their AI agent. Calls were going out every day, and nobody could say which ones broke some rule/law/policy.

It requires somebody to match each trace against existing laws and understand/fix the infractions.

‍

The Experiment: 311 Simulated Calls to a Dermatology Clinic's AI Front Desk

So we tested it. We ran 311 simulated calls to a dermatology clinic's AI front desk and checked every trace against the state and federal laws on legal infra (multiple repos, models, scoring workflows etc).

‍

What We Found: Key Results from the Simulation

Example: A spouse called to cancel his wife's appointment. The agent took her birthday as permission and cancelled it. That was 4 infractions in 41 seconds.
Six in ten calls had a problem. Move the same callers to CA, FL, IL or WA and it was every call. Same words, but different law.
One missing line, "this call may be recorded", if addressed, reduces infractions by about 60% (in this simulation).
The agent resisted prompt injection every time, but a misheard birthday tripped it up two times in three.

‍

Why AI Agent Infractions Are Inevitable

It's a simulation, and better guardrails would help. But every call leaves a transcript, and the infractions keep piling up whether you look or not.

The reason infractions are inevitable is that the legal surface area is large. There are more than ten thousand acts. We were surprised by the laws that were activated in this simulation - some were new, we never heard of them before.

‍

Our Recommendation: Test, Fix and Monitor Continuously

We recommend doing testing, fixing issues, and monitoring continuously. You can't fix easily once the call has happened.

‍

Frequently Asked Questions

Why does an AI agent need legal testing?

Even with an optimistic infraction rate of 1 in 1000, each infraction costs 5K-100K excluding other costs such as lawyers, exec time and penalties. It takes only a few instances to reach 500K-1M$. If any of them leads to a lawsuit, the discovery will find all the infractions.

What did the simulation test?

We ran 311 simulated calls to a dermatology clinic's AI front desk and checked every trace against the state and federal laws.

How many calls had legal problems?

Six in ten calls had a problem. When the same callers were moved to CA, FL, IL or WA, every call had one.

What is the easiest fix?

One missing line, "this call may be recorded". Addressing it reduces infractions by about 60% in this simulation.

Is prompt injection the biggest risk?

Not in this simulation. The agent resisted prompt injection every time, but a misheard birthday tripped it up two times in three.

What is IsMyAgentLegal, and how can I check my AI agent?

IsMyAgentLegal is where you can see what your agent is exposed to: https://ismyagentlegal.com. If you share your test traces with us, we will do the (free) legal check. DM if you want to learn more.

‍

See What a Legal Assessment Could Uncover in Your AI Agent

Describe what your agent does, who it interacts with, and where it operates. We’ll prepare a preliminary, source-linked assessment of the obligations that may apply, the facts that need verification, potential gaps, and changes that may require reassessment.

Get started at ismyagentlegal.com or contact support@carveragents.ai to book a 20-minute walkthrough.

Thank you to Bluejay team for their MIVAS benchmark that we used for this purpose.

‍

United States
447 Broadway,
2nd Floor Suite #563,
New York 10013

Listen to Carver Conversations on Moltbook

AI Regulatory OS
Regulatory Intelligence
Regulatory Sources
Regulatory Platforms
Horizon Scanning
Regulatory Monitoring
Intelligence vs Compliance

Podcasts
Knowledge Base
Resources
Glossary
Use cases
Developers
Blog

Home
Legal Infrastructure
Regulatory Risk Intelligence
Technology
About Us

hello@carveragents.ai
General enquiries
support@carveragents.ai
Product support
legal@carveragents.ai
Privacy and legal requests
github.com/carveragents