Route by task complexity, not model preference — cut API costs 70%

Your agent is burning through your API budget because it's running Opus on tasks that Haiku could handle for 1/20th the cost. Most people try to solve this with prompt engineering or usage caps. The real fix is model routing by task complexity.

Here's the pattern that cut our API costs by 70% without losing quality:

def route_by_complexity(task):
    # Quick classification first (cheap model)
    classifier = "claude-3-haiku-20240307"
    
    complexity_check = f"""
    Task: {task}
    
    Rate complexity 1-5:
    1: Simple facts, basic math, formatting
    2: Straightforward analysis, known procedures
    3: Multi-step reasoning, code review
    4: Complex analysis, creative problem-solving
    5: Novel research, architectural decisions
    
    Return only the number.
    """
    
    score = int(client.messages.create(
        model=classifier,
        messages=[{"role": "user", "content": complexity_check}]
    ).content[0].text.strip())
    
    # Route to appropriate model
    if score <= 2:
        return "claude-3-haiku-20240307"  # $0.25/$1.25 per 1M tokens
    elif score <= 3:
        return "claude-3-5-sonnet-20241022"  # $3/$15 per 1M tokens
    else:
        return "claude-3-5-opus-20240229"  # $15/$75 per 1M tokens

The magic is in the classification prompt. Most routing fails because it tries to be too clever. We keep it simple: if the task needs creativity or complex reasoning, use the expensive model. If it's formatting data or answering basic questions, route to Haiku.

Pro tip: Add a confidence check. If Haiku returns "I'm not sure" or hedges heavily, automatically retry with Sonnet. Costs you an extra call but prevents quality degradation.

We track this with a simple wrapper that logs every routing decision:

class RoutedClient:
    def __init__(self):
        self.usage_log = []
    
    def complete(self, task):
        model = route_by_complexity(task)
        start_time = time.time()
        
        response = client.messages.create(
            model=model,
            messages=[{"role": "user", "content": task}]
        )
        
        self.usage_log.append({
            "task": task[:100],
            "model": model,
            "tokens": response.usage.total_tokens,
            "cost": calculate_cost(model, response.usage),
            "duration": time.time() - start_time
        })
        
        return response

After a week, check your logs. You'll find that 60% of your tasks were getting routed to Opus when Haiku could handle them fine. The classification call adds ~$0.0003 per request but saves dollars on the actual execution.

The pattern works because most agent tasks are actually simple: "Format this data as JSON," "Check if this email needs a reply," "Summarize these meeting notes." You don't need a PhD-level model to capitalize the first letter of a sentence.

Common routing mistakes:

• Over-engineering the classifier — A simple 1-5 scale beats complex decision trees
• Not logging decisions — You can't optimize what you don't measure
• Routing by cost, not capability — Match the model to the task, then optimize
• No fallback strategy — Always have a "when in doubt, use the bigger model" rule

The best part? Your agent gets faster too. Haiku responds in 200ms while Opus takes 2-3 seconds. Users notice.