The Three-Layer Audit: How to Actually Verify an AI Coding Agent Before You Trust It
Most people treat an AI coding agent's output the way they treat a junior analyst's first draft: skim it, and if it sounds coherent, ship it. That's the wrong instinct. The failure mode isn't obviously wrong output — agents rarely produce obviously wrong output. The failure mode is plausible output: a claim that sounds like it came from documentation, a script that passes its own demo case, a recommendation that sounds locally correct. Plausible-and-wrong is far more dangerous than obviously-wrong, because nothing about it trips your instinct to double-check.
I recently ran an agent through a real build task — auditing and rebuilding its own token-usage tooling, including some hooks meant to run automatically in future sessions. Three separate moments in that session map cleanly onto a repeatable audit discipline. Call it the Three-Layer Audit: verify the claim against a primary source, test the artifact against a realistic input rather than the demo case, and re-derive the goal before accepting a recommendation. Skip any one layer and a plausible-sounding failure ships.
Layer 1: The Primary-Source Check
Agents synthesize. Synthesis compresses, and compression drops caveats. Somewhere in an earlier research pass, one of the agent's own sub-agents had asserted a specific safety behavior for a particular automation hook — a documented auto-release after a fixed number of retries, framed as an official safeguard against infinite loops.
It was fabricated. Not maliciously — it was the kind of confident-sounding synthesis that happens when a system pattern-matches "safety mechanisms usually have a release valve" onto a specific claim it never actually verified. The only way to catch it was to go back to the primary source — Anthropic's own current hooks documentation — and check the literal claim, not the summary of the claim. The documentation said the opposite: no built-in release mechanism exists, and the authors explicitly recommend implementers build their own.
That's the general pattern. A secondary source — a summary, a blog post, a prior AI-generated synthesis — is a hypothesis about what the primary source says, not the primary source. Treat any claim you're about to build safety-critical logic on as unverified until you've read the primary document yourself, in the current session, not from memory. This is the same discipline that governs reading a cited study instead of a LinkedIn summary of the study — the summary is optimized for narrative, not fidelity.
Where this mattered: had the fabricated claim gone unverified, the resulting automation would have had zero built-in loop protection while believing it had some. The fix — a hard, self-owned retry cap — only got built because the claim got checked against the source instead of accepted on confidence.
Layer 2: The Realistic-Input Test, Not the Demo Case
The agent built a filter meant to compress noisy command output before it re-entered its own working context — the AI-tooling equivalent of a log truncator. Fed a synthetic wall of build-log noise, it worked exactly as designed: buried errors surfaced, repetitive noise collapsed. Demo case: passed.
Then it got run against a realistic input it hadn't been tested on — a routine code diff, the kind of output that shows up dozens of times in any real working session. The filter shredded it. Not an edge case: a completely ordinary command, applied through logic that had only ever been validated against its own best-case demo.
This is the gap between "I tested it" and "I tested it against what will actually hit it in production." A test suite built by the same system that wrote the code inherits that system's blind spots — it tends to test the case the code was designed for, not the adjacent cases the code will actually encounter. The fix here wasn't more testing volume, it was testing diversity: deliberately picking an input type that had nothing to do with the original design intent, specifically because it hadn't been considered yet.
The generalizable version: before trusting any AI-generated filter, classifier, or transformation logic, feed it the most boring, common, unglamorous input from its real operating environment — not the input that showcases the feature. Boring inputs are where blind spots hide, because nobody designs a demo around them.
Layer 3: Re-Derive the Goal Before Accepting the Recommendation
The subtlest failure of the three. The agent had, in an earlier pass, recommended a configuration change that was locally correct — it optimized the stated objective at the time, which was minimizing resource consumption. Later in the same engagement, the actual priority shifted to maximizing first-attempt correctness on complex tasks — a different, and in one specific respect opposing, objective. Trimming verbosity to save resources is exactly the kind of lever that quietly undercuts first-attempt correctness, because verbose reasoning is often doing real work, not just costing more.
The agent had been asked to "accept the recommendation." The correct response wasn't compliance — it was checking whether the recommendation still served the current goal, since the goal itself had moved. Accepting a prior-context recommendation without re-checking it against the current objective is how organizations end up optimizing yesterday's KPI while believing they're serving today's.
This layer generalizes past AI tooling entirely. Any standing recommendation — a pricing tier default, a target CPA, a testing cadence — was optimized for the objective function that existed when it was made. Objectives drift. Recommendations don't automatically drift with them. The audit step is cheap: before accepting, ask what objective this currently optimizes for, and whether that's still the objective you have.
What This Costs, and What It's Worth
None of this is free. Layer 1 costs a documentation fetch instead of trusting memory. Layer 2 costs deliberately trying to break your own tool instead of declaring victory on the first pass. Layer 3 costs a moment of friction against the instinct to just say yes to a recommendation that sounds reasonable. Total overhead: a few extra minutes per audit.
Compare that to the counterfactual. A fabricated safety claim shipping into a hook that runs unattended, with real risk of hanging a working session with no exit. A filter silently destroying real content on a routine command — the kind of failure you wouldn't notice until the AI made a decision based on information it never actually saw, because your own tooling had quietly deleted it. A configuration drifting to optimize an objective you'd already abandoned. Every one of those is a multi-hour cleanup for a few minutes of audit discipline avoided.
The pattern that connects all three layers: an AI agent's output is a hypothesis, not a fact. Confident phrasing is not evidence. A passing demo is not a passing production test. A recommendation that was correct is not necessarily still correct. Trust, but verify — and verify against the primary source, the realistic input, and the current objective, not against the agent's own confidence. It's the same discipline Anthropic's own guidance on agentic coding points at when it stresses verification loops over single-pass trust — the tool vendor's own documentation agrees the burden of proof sits with the operator, not the output.
The economics of getting this wrong compound, too — a fabricated claim or an unverified filter doesn't just cost a cleanup, it quietly inflates the very AI-agent spend I've written about diagnosing separately.
FAQ
How do I know which claims from an AI agent actually need primary-source verification?
Anything the agent presents as a fact about an external system's behavior — API limits, documented safety mechanisms, pricing, version-specific behavior — needs a primary-source check before you build on it, especially if the downstream logic is safety-critical or hard to reverse. Anything the agent is directly computing or generating in front of you (a file it just wrote, a value it just calculated) doesn't need external verification — you can inspect it directly. The dividing line is whether the claim is about something outside the current session's direct observation.
Isn't testing against a realistic input just... testing? Why does this need a name?
Because "I tested it" is almost always true and almost always insufficient. The failure isn't a lack of testing — it's testing exclusively against inputs chosen to showcase the feature working, which is the natural bias of whoever (or whatever) built the feature. Naming the layer forces a deliberate second step: after the feature's own test passes, pick an input the feature wasn't designed around and see what happens.
How often should I re-derive the goal on a standing recommendation?
Any time the stated objective changes, and periodically even when it hasn't — quarterly is a reasonable default for anything consequential. The cheapest version of this check is a single question before accepting any recommendation: "what was this optimized for, and is that still what I want?" If you can't answer the first half confidently, that's itself a signal the recommendation is older than it should be.
If you're running AI agents against real production systems and want a second set of eyes on where the verification gaps actually are, schedule a consultation — this is the same rigor I bring to experimentation programs, applied to a newer class of tooling that most teams haven't built an audit discipline for yet.