Debug the Agent’s Assumption, Not Only Its Output

When generated code fails, the visible error is often several steps removed from the first wrong turn. Fixing only the output can leave the reasoning defect ready to reappear in the next task.

The same is true of human debugging, but agents make the pattern easier to miss because they can produce a large, internally consistent patch from one unsupported premise. By the time a build fails, many files may agree with each other and disagree with the repository.

Start with observable actions rather than a story about what the model “thought.” A compact trace can record the files inspected, commands run, assumptions declared, and checks performed.

const trace = [];

export function record(step, evidence) {
  trace.push({ step, evidence, observedAt: new Date().toISOString() });
}

record("inspect-config", { file: "eleventy.config.js" });
record("assume-output", { directory: "dist" }); // Unsupported assumption

The output directory error is easy to correct. The more valuable finding is that the agent guessed instead of reading the configuration.

Keep traces factual. Record commands, file paths, outputs, and explicit assumptions. Avoid inventing an inner monologue for the model. The goal is to find the first action that was not justified by available evidence.

A useful trace also records negative evidence: a file was searched for and not found, a command returned no matches, or a test covered only one code path. Absence can explain why a later guess seemed plausible without making it correct.

Prompt reminders help, but executable checks travel farther.

import assert from "node:assert/strict";
import { access } from "node:fs/promises";

await access("_site/index.html");
assert.ok(true, "Eleventy produced the documented output directory");

The example is intentionally plain. A durable guardrail should fail near the incorrect assumption and explain what to inspect next. A large end-to-end test may catch the same problem much later with a much less useful message.

Choose the cheapest durable control that fits:

• Repository guidance for a convention that requires judgment.
• Schema validation for structured content.
• A focused unit test for deterministic behavior.
• Output validation for generated artifacts.
• Browser automation for interaction and rendering contracts.

Not every mistake deserves a new test. If the task was genuinely one-off, a corrected implementation and a clear review note may be enough. Guardrails earn their maintenance cost when they prevent a class of plausible future failures.

Read a failure trace from the start and ask where evidence first stopped supporting the next step. That point is usually cheaper to repair than every consequence below it.

Suppose an agent edits a generated file, then adds an exception to formatting, then changes the build because its edit disappears. The last change may look like the problem, but the earliest divergence was failing to identify which files are sources and which are output. Repair that boundary and the downstream “fixes” become unnecessary.

After correcting the cause, repeat the relevant workflow from a clean build. Cached output can make a broken assumption appear resolved. The clean run proves the repository can reproduce the result without the accidental state that existed during debugging.

Finally, review the diff for compensating changes. Agents, like people, may have altered configuration or loosened validation while searching for a path forward. The right ending is not merely a passing command; it is the smallest coherent change whose evidence now matches the system.