Agent model switching burned $47 on a $3 conversation

Your agent switches between Claude and GPT mid-conversation, and you have no idea why or what it's costing you.

Most people think model routing is about price optimization. It's not. It's about preventing your agent from burning through your API budget on invisible failures.

Here's what actually happens: Your agent hits Claude's rate limit, automatically fails over to GPT-4, burns through 3x the tokens on a retry loop, then switches back to Claude when the rate limit resets. You see "task completed" in your logs. Your bill shows $47 for what should have been a $3 conversation.

Without routing visibility, model switching becomes silent budget drain. Every failover is a cost multiplier you can't see.

The fix isn't smarter routing logic. It's routing observability.

Track three metrics in real-time:

• Route reason — rate limit, context overflow, task complexity, or cost optimization
• Completion cost — tokens × time × model price, not just token count
• Retry cascade depth — how many model switches happened for one user request

Here's the logging pattern that catches expensive routing before it kills your budget:

{
  "request_id": "req_123",
  "route_chain": [
    {"model": "claude-3-5-sonnet", "reason": "default", "tokens": 1200, "cost": 0.018},
    {"model": "gpt-4", "reason": "rate_limit_failover", "tokens": 3400, "cost": 0.102},
    {"model": "claude-3-5-sonnet", "reason": "retry_after_reset", "tokens": 1800, "cost": 0.027}
  ],
  "total_cost": 0.147,
  "expected_cost": 0.018,
  "cost_multiplier": 8.2
}

Set alerts when cost multiplier exceeds 3x. Most routing failures cascade within 5 minutes — catch them before they compound.

The pattern that saves money: Route by task completion cost, not token cost. A 10K token Claude conversation that succeeds costs less than a 3K token GPT conversation that fails and retries.

Smart routing isn't about picking the cheapest model. It's about picking the model most likely to complete the task on the first try.

Build routing rules around completion probability, not token price. Your agent will cost less and work better.

Most people discover their routing is broken when they get a $400 API bill for a Tuesday afternoon. Build the observability now, before your agent finds every expensive edge case.