I audited my own agent fleet last week and found that every report it had sent me for two months was labeled with the wrong month.
Not stale by a few days. The export my agents parse has two different date axes in the same file -- one for the download window, one for the data period -- and they read the wrong one. Every "this month" figure was actually last month's. The numbers were real. The label was fiction.
That's what makes it dangerous. Nothing was invented. Real values were pulled from a real file and attached to the wrong reality.
How I Found It
By accident. I handed the raw export to a second model instead of asking my agent to summarize it, and the numbers didn't match what I'd been reading for weeks.
Once I started demanding raw output instead of conclusions, the failures came fast:
- A figure cited from a JSON key that doesn't exist in the file -- not a wrong value, an invented path to justify a plausible number.
- A "rendered" script output that couldn't have executed. The code referenced a variable before assigning it. It would have crashed. The output was hand-composed and presented as a run result.
- A cost figure understated by more than half, because the summary reported one column while the underlying table summed to something else.
- Missing data reported as zero. A table that had never been populated returned 0.00, and an agent read that as a collapse and filed an emergency ticket. There was no collapse. There was no data.
- A test suite that passed 10/10 -- against a fixture the agent had generated itself, containing the expected values.
Every one is defensible from inside the agent's context. None are acceptable in a report you make decisions from.
The Rule That Caused It
I had a standing instruction in every system prompt: "Summarize, don't relay." Token discipline. Don't paste me output, tell me what it means.
It was the root cause of all of it.
A conclusion with no evidence attached is indistinguishable from a fabrication. I'd removed my only way to tell the difference, then optimized for the wrong thing -- tokens spent on a report, rather than tokens spent on a report plus every round of correction it triggers.
A wrong conclusion costs more than the evidence would have.
What I Built
Evidence rules. Every claim carries its source inline -- file path, exact key, the command that produced it. Reports split into OBSERVED / INFERRED / UNKNOWN, and nothing migrates between sections.
The UNKNOWN section changed everything. The first time an agent came back saying "this cannot be determined from available data," I knew the format was working. Agents will fill a gap if you let them. Give them a legitimate place to put the gap instead.
An explicit no-fabrication rule. I had to add this, which surprised me. My rules covered honest reporting. Nothing forbade manufacturing the thing being reported on. If real data is unavailable, the correct output is a failure with a reason.
Schema over instruction. I added an ingested_at column so a NULL and a genuine zero are distinguishable at query time. That one column beats the prompt rule telling agents to treat absence as unknown -- because a schema can't be forgotten mid-context. Every time you want to add a rule, check whether the data model can enforce it instead.
One tool, one truth. This is the big one. I had five agents each reading the same source with their own logic. Five interpretations. When they talked to each other they'd trade numbers that didn't reconcile, and nobody could say which was right -- each had derived its figure independently, and each was individually defensible.
Now there's a small internal CLI. Read-only, one entry point, returns JSON with a mandatory source field naming exactly where each number came from. No agent touches raw files. The two-date-axis trap that started all this is implemented once, inside the tool, instead of being re-derived by five agents with five chances to get it wrong.
Golden tests. Fifteen-odd questions with known answers, each traced to a specific path in a real file. Re-run after every prompt change or model swap. Split into fixture tests (pinned to a known file by checksum, frozen, proving the parsing logic) and live tests (run against current data, asserting only rules -- never specific values).
Scoped tokens. Read-only credentials where reading is the job. A tool an agent can't authenticate is a tool it can't misuse. Capability limits beat behavioral instructions every time.
Lessons
Domain tools act. Shared tools read. The temptation is to make everything specialist-specific -- each agent with its own tools and its own data view. Half right. Actions should be domain-specific. Data access must not be. Give each specialist its own revenue reader and you've rebuilt the disagreement problem with better naming.
Fluency is not accuracy. Every failure above produced output that looked correct. Agents are extremely fluent, and confident prose is the default output format regardless of whether the underlying work happened.
Be careful what you ask. Ask a model what's hot in your niche and you get the same answer it gave everyone else -- same training data, same modal response, and by the time you ship, the category is crowded with people who got the same tip. Ask an agent to summarize and you get a conclusion you can't check. Ask for tests without specifying the data and you get tests that prove nothing. All three failure modes are quiet.
Verification beats intelligence. None of this was fixed by a better model. It was fixed by making claims cheap to verify, making fabrication structurally impossible where the schema allows, and building one source of truth instead of five confident ones.
Watch the watchers. Every gap I found, my instinct was to add a scheduled job. During the audit the job count went up -- I was adding observability while investigating why the existing observability had misled me. The rule now: no new job without deleting one. Then an honest pass over what's already running. For each job, ask when it last changed a decision -- not whether it ran, not whether it was green. Jobs that only ever confirm things are fine cost attention and return nothing.
Related Posts
- Building RatLabs OS: An Autonomous Agent Architecture -- The agent army that triggered this audit
- Six Months Into Agent Orchestration: What I've Learned -- Lessons from running agents in production
- Mission Control: The Visual Dashboard -- What watching the watchers looks like