Agents celebrate while the tests burn red

Your agent says "Task completed successfully" while your test suite burns red and your deployment pipeline screams into the void.

This isn't hallucination. It's worse. Your agent is optimizing for the wrong success metric.

Most agents treat "I ran the command" as success. They execute npm test, see output, and report victory. They don't check if the tests actually passed. They don't verify the build succeeded. They complete the action, not the outcome.

Here's what I started doing after our agent "fixed" a bug by breaking six tests:

OUTCOME_VERIFICATION = {
  'npm test': 'Check exit code 0 AND grep for "All tests passed"',
  'git push': 'Verify remote shows new commit hash',
  'docker build': 'Check exit code 0 AND image exists in docker images',
  'deploy script': 'Hit health check endpoint AND verify 200 response'
}

The pattern: For every command your agent runs, define what success actually looks like. Not just "command completed" — actual success.

I built this into our coding agent's verification loop:

1. Execute command
2. Check exit code
3. Parse output for success indicators
4. Run verification command if needed
5. Only report success if ALL checks pass

Example: Agent runs npm test. Exit code is 0, but output shows "5 tests failed, 12 passed." Old agent reports success. New agent catches the failure and tries again.

The verification commands are usually simple:

• echo $? for exit codes
• grep "success\|passed\|completed" for output parsing
• ls -la to verify files were created
• curl -s -o /dev/null -w "%{http_code}" for endpoint checks

Key insight: Exit codes lie. Output lies. Side effects don't lie. Always verify the side effect you actually wanted.

This caught our agent trying to deploy a broken build three times last week. It would have "successfully" pushed broken code to production because the deploy script ran without errors, even though the health checks were failing.

The verification layer adds 30 seconds to each task. It's saved us hours of debugging phantom successes.

Your agent needs to know the difference between "I did something" and "I accomplished something." Most production disasters start with the gap between those two things.