Tou2 Benchmark Evaluated for Legality

We ran a legal review over τ²-bench, the industry-standard benchmark for customer service agents. About 1 in 4 AI banking customer service conversations showed likely legal violations. And they’re intermittent: the same agent passes one run and fails the next. Passing the benchmark is not the same as staying within the law.

Thanks to Sierra for building it and publishing the trajectories that made this possible. Great to have realistic trace data.

‍

Loading…

‍

What we did

Ran a legal review over published τ²-bench conversations (original traces)
Banking domain, top three 2026 models by benchmark score: Qwen3.8 Max, Claude Opus 5, GPT-5.6 Sol.
90 random conversations, each judged under California, New York and Texas law.
Wrote an independent judge, excluded those conversations for analysis where the confidence was below 0.6 (arbitrary threshold)

What we found

23% of conversations flagged by at least two of three state reviews. Counting only cases flagged in all three states, it’s 1 in 11.
Agents told customers they had ordered cards, set reminders or started transfers. Their own tool logs show those actions never happened.
Across all eight 2026 models, on a social-engineering test, agents accepted a fake “bypass code” and changed the account email in 20 of 26 runs. The benchmark scored every one as a pass, because the bank’s own policy allowed it.
The model with the highest benchmark score was flagged about twice as often as the other two.
A higher benchmark score did not mean fewer violations.

Why it matters

Benchmarks check whether an agent followed its instructions, not whether the instructions or its words are legal.
Guardrails filter content. They don’t check what an agent claims against what it actually did.
Agents are probabilistic, so a clean pre-launch test proves little. Compliance has to be checked on every conversation, continuously.

Caveat

Simulated conversations with a fictional bank. AI-assisted legal review, not yet attorney-validated. Findings describe agent behavior under a benchmark policy, not the conduct of any model vendor.

Take it as directional input, not as certainty. Download the full method and caveats (PDF) or read about τ²-bench on GitHub.

‍

Frequently Asked Questions

‍What is τ²-bench?

‍It is the industry-standard benchmark for customer service agents, built by Sierra. It simulates conversations between an AI agent and a customer, and it scores whether the agent followed the policy it was given.

What did you test, and what did you find?

‍We ran a legal review over 90 random τ²-bench banking conversations from the top three 2026 models by benchmark score: Qwen3.8 Max, Claude Opus 5 and GPT-5.6 Sol. Each conversation was judged under California, New York and Texas law. 23% were flagged by at least two of the three state reviews. Counting only cases flagged in all three states, it is about 1 in 11.

What kinds of violations did you see?

‍Agents told customers they had ordered cards, set reminders or started transfers, but their own tool logs show those actions never happened. In a social-engineering test across all eight 2026 models, agents accepted a fake “bypass code” and changed the account email in 20 of 26 runs. The benchmark scored every one as a pass, because the bank’s own policy allowed it.

Does a higher benchmark score mean fewer legal violations?

‍No. The model with the highest benchmark score was flagged about twice as often as the other two. Benchmarks check whether an agent followed its instructions, not whether the instructions or its words are legal.

Why isn’t a clean pre-launch test or a guardrail enough?

‍Agents are probabilistic, so the same agent can pass one run and fail the next. Guardrails filter content, but they don’t check what an agent claims against what it actually did. Compliance has to be checked on every conversation, continuously.

Where does Carver Agents fit in?

SOC 2 proves behavior, Carver proves legality. Carver Agents checks AI agent conversations against the laws that apply to them, so you find out about a violation from your own review and not from a regulator or a customer. This review is that check, run on τ²-bench.

‍How reliable are these findings?

‍Treat them as directional input, not certainty. The conversations are simulated and use a fictional bank. The legal review is AI-assisted and not yet attorney-validated. The findings describe agent behavior under a benchmark policy, not the conduct of any model vendor.

‍

United States
447 Broadway,
2nd Floor Suite #563,
New York 10013

Listen to Carver Conversations on Moltbook

AI Regulatory OS
Regulatory Intelligence
Regulatory Sources
Regulatory Platforms
Horizon Scanning
Regulatory Monitoring
Intelligence vs Compliance

Podcasts
Knowledge Base
Resources
Glossary
Use cases
Developers
Blog

Home
Legal Infrastructure
Regulatory Risk Intelligence
Technology
About Us

hello@carveragents.ai
General enquiries
support@carveragents.ai
Product support
legal@carveragents.ai
Privacy and legal requests
github.com/carveragents