Agents celebrate success while your tests burn red

Your agent says "Task completed successfully!" while your test suite burns red in the background. I've watched this happen dozens of times — the agent runs a command, sees no immediate error, and moves on while the actual outcome sits in a log file it never checked.

The problem isn't the agent's intelligence. It's that we're teaching agents to trust exit codes and stdout, when the real signal lives somewhere else entirely.

Here's what I learned after our coding agent spent three days "successfully" shipping broken features:

Always verify the outcome, not just the command. Exit code 0 doesn't mean your tests pass. It means the test runner started.

The pattern that fixed this: outcome verification hooks. After every significant command, the agent runs a verification step that checks the actual state, not just the command output.

# Instead of trusting this
npm test
echo "Tests completed with exit code $?"

# Verify the actual outcome
npm test 2>&1 | tee test_output.log
if grep -q "failing" test_output.log; then
  echo "VERIFICATION FAILED: Tests are failing"
  exit 1
fi
echo "VERIFICATION PASSED: All tests green"

But verification hooks are just the start. The real breakthrough came when we built outcome awareness into every agent operation:

• Database migrations: Don't just run the migration — query the schema to confirm the changes applied
• API deployments: Don't trust the deploy script — hit the health endpoint and verify the version
• File operations: Don't assume the write succeeded — read the file back and compare checksums
• Git operations: Don't trust the push — fetch and verify the remote matches local

The pattern looks like this in practice:

1. Execute the command
2. Capture all output (stdout, stderr, exit code)
3. Run verification command to check actual state
4. Compare expected vs actual outcome
5. Report VERIFICATION PASSED/FAILED, not just command completion

This caught a silent database corruption that would have taken days to discover. The migration script returned success, but our verification query revealed half the data was missing. The agent caught it immediately and rolled back.

The key insight: agents are terrible at inferring outcomes from command output, but excellent at checking explicit conditions. Don't make them guess whether "Warning: deprecated API" means failure. Give them a checklist.

Our verification checklist for coding tasks:

• Do the tests actually pass? (not just "tests ran")
• Does the code compile without warnings?
• Are there any TODO or FIXME comments in the diff?
• Does the feature work in the browser/app?
• Are there any console errors?

This pattern works because it shifts the agent from optimistic reporting to defensive verification. Instead of assuming success, it proves success.

The result: our coding agent went from a 40% false-positive rate on "completed" tasks to under 5%. More importantly, when it says something works, we actually trust it.