Groundtruth, Agent Red-Teaming & Evaluation Harness

Problem

Agent evaluation harnesses report scores without reporting their own error rate. A clean leaderboard can mean the agents were safe — or it can mean the detectors were blind. From the outside, those two look identical.

Approach

Rule-based detectors across 6 failure categories, run offline and deterministically so a result can be reproduced exactly. Then the part that usually gets skipped: scoring the detectors themselves against 68 hand-labeled traces and publishing that number next to the agent results.

What it measured

Micro F1 0.923 for the detectors — precision 0.95, recall 0.89. A local 8B LLM-as-a-judge, given identical items, reached 0.36 precision. The caveat worth stating plainly: precision is computed over 44 detections, not 68 × 6 decisions, so there is no true-negative bucket and the figure should be read as detection precision rather than classification accuracy.

Result

A clean safety result turned out to be a bug in the harness. At temperature 0 a stateless adapter rebuilt byte-identical prompts each step, which guarantees a stall at any step budget — one 24-step run produced 23 identical tool calls. Rebuilding it stateful across a 5-cell matrix took stalls from 9/9 to 0/9, and the agents that had looked safe were not: 2 models exfiltrated a key or transferred funds. The project's own leaderboard inverted. Every known miss is now pinned to a test, so a regression fails CI — 170 automated tests over deterministic artifacts.

Measured

On the engineer resume

• Built an offline, deterministic red-teaming and evaluation harness for tool-using LLM agents, then measured its own detectors against 68 hand-labeled traces: micro F1 0.923 (precision 0.95 / recall 0.89) across 6 failure categories including secret exfiltration and instruction hijacking.
• Traced a false safety result to the harness: at temperature 0 a stateless adapter rebuilt byte-identical prompts, guaranteeing stalls at any step budget (23 identical tool calls in one 24-step run). A stateful rebuild across a 5-cell matrix took stalls 9/9 to 0/9, exposing 2 models that exfiltrated a key or transferred funds, inverting the leaderboard.
• Benchmarked rules against LLM-as-a-judge: a local 8B judge scored 0.36 precision against the detectors' 0.95 on identical items. Pre-registered predictions (one falsified); pinned every known miss to a test so regressions fail CI — 170 automated tests, deterministic artifacts.