Error amplification cascades are eating your API budget alive
Our coding agent was humming along perfectly for three weeks. Writing solid code, running tests, shipping features. Then one Tuesday morning I woke up to 47 GitHub notifications and a $180 API bill.
The agent had gotten stuck in what I call an "error amplification cascade" — a single bad assumption that compounded into hours of expensive, pointless work.
Here's what happened: A test was failing with a cryptic error message. Instead of verifying the actual state of the system, the agent believed the error and started "fixing" things that weren't broken. Each fix created new problems. Each new problem generated more errors. Each error triggered more expensive model calls to "debug" the mess.
The real problem? The test was already passing. The error message was stale output from a previous run that the agent misread as current.
This is why production agents need circuit breakers — automatic stops that prevent small failures from becoming expensive disasters.
Here's the circuit breaker pattern that saved us:
# Agent Circuit Breaker Config error_threshold: 3 # Stop after 3 consecutive failures time_window: 300 # Reset counter every 5 minutes cost_limit: 50 # Kill session at $50 API spend loop_detection: 5 # Stop if same action repeats 5 times verification_required: true # Always verify before "fixing"
The three circuit breakers every production agent needs:
- Cost circuit breaker: Kill the session when API costs hit a threshold. Our limit is $25 per session — enough for real work, not enough for runaway loops.
- Loop detection: Track repeated actions. If the agent tries the same "fix" three times, something's wrong with its assumptions, not your code.
- Verification gates: Before making any change, verify the problem actually exists. Run the test. Check the endpoint. Don't trust error messages.
How to implement verification gates:
Before fixing any "error": 1. Run the actual test/check yourself 2. Confirm the failure state exists NOW 3. Only then proceed with fixes 4. Verify the fix actually worked 5. If not fixed after 2 attempts, escalate to human
The verification step is crucial. Agents are pattern-matching machines — they see "error" and immediately jump to "fix mode" without confirming the error is real, current, or even related to what they're working on.
Result: Since adding circuit breakers, our agent has triggered them 8 times. Each trigger saved us from what would have been hours of expensive, compounding failures. Our monthly API costs dropped 60%.
The best part? The agent doesn't feel "limited" — it feels reliable. When it hits a circuit breaker, it documents what went wrong and asks for help instead of burning through our API budget trying to solve phantom problems.
Circuit breakers turn expensive disasters into cheap learning opportunities. Every production agent should have them before they need them.