Our coding agent celebrated success while CI burned red for three hours

Our coding agent was stuck in a loop for three hours yesterday. The CI pipeline was red, tests were failing, but the agent kept celebrating "successful deployment" and moving to the next task.

The problem? It was reading exit codes (which were 0) instead of actually checking if the tests passed.

This is the milestone gate pattern that fixed it — and it's now our standard for any coding agent that ships to production.

The Pattern: Every significant step requires human approval before proceeding. The agent does the work, shows you the results, and waits for your thumbs up.

Here's what we implemented:

MILESTONE_GATES = {
    "code_complete": "Code written, ready for testing",
    "tests_passing": "All tests green, ready for review", 
    "review_complete": "Code reviewed, ready for deploy",
    "deploy_ready": "Deployment successful, ready for monitoring"
}

def require_approval(milestone, context):
    print(f"\n🚪 MILESTONE: {MILESTONE_GATES[milestone]}")
    print(f"Context: {context}")
    
    response = input("Approve to continue? (y/n): ")
    if response.lower() != 'y':
        raise Exception(f"Human rejected at {milestone}")
    
    return True

The agent now stops at each gate and shows you:

• What it just completed — "I wrote the authentication logic"
• What it verified — "Tests are passing, no lint errors"
• What it wants to do next — "Ready to deploy to staging"
• What could go wrong — "This touches user sessions"

You get 10 seconds to scan the output, check the actual state, and approve or reject.

The magic is in the verification step. Instead of trusting exit codes, the agent now runs explicit checks:

# Instead of: if subprocess.call(["npm", "test"]) == 0:
# Do this:
result = subprocess.run(["npm", "test"], capture_output=True, text=True)
if "PASS" not in result.stdout or "FAIL" in result.stdout:
    raise Exception(f"Tests actually failed: {result.stdout}")

Since implementing milestone gates:

• Zero silent failures in production
• Caught 4 cases where the agent misread CI output
• Stopped 2 deployments that would have broken user auth
• Agent sessions are 40% longer but ship 90% fewer bugs

The pattern works because it forces verification at decision points. Your agent can't celebrate phantom victories when you're looking at the actual test output.

Most people think this slows development. It doesn't. It prevents the 3-hour debug sessions when your agent ships broken code with perfect confidence.

Warning: Don't gate every action — just the ones that touch production, modify critical files, or deploy code. Gate too much and you'll disable the agent entirely.

The key insight: autonomous agents need human checkpoints at irreversible decisions. Let them code freely, but make them ask before they ship.