Stop paying GPT-4 prices for GPT-3.5 work — here's the routing pattern that cuts costs 60%

Your agent is burning through your API budget because it's using GPT-4 to answer "What time is it?" and Claude Sonnet to write "Task completed successfully."

Most agent frameworks route every single request to your primary model. That's like hiring a surgeon to take your temperature.

Here's the routing pattern that cut my monthly OpenAI bill from $340 to $140:

// Simple request classifier
const classifyRequest = (message) => {
  const simplePatterns = [
    /^(what time|current time|date)/i,
    /^(yes|no|ok|thanks|got it)$/i,
    /^(status|progress|update)\?*$/i,
    /^(list|show|display)\s\w+$/i
  ];
  
  const complexPatterns = [
    /write.*code/i,
    /analyze.*data/i,
    /create.*plan/i,
    /solve.*problem/i
  ];
  
  if (simplePatterns.some(p => p.test(message))) return 'simple';
  if (complexPatterns.some(p => p.test(message))) return 'complex';
  if (message.length < 50) return 'simple';
  return 'complex';
};

Route simple requests to GPT-3.5-turbo ($0.001/1K tokens) and complex ones to GPT-4 ($0.03/1K tokens). That's a 30x price difference.

But here's what most people get wrong: they try to build the perfect classifier. Don't. Start with obvious patterns and add edge cases as you find them.

Pro tip: Log every request with its classification and cost. Review weekly. You'll spot patterns you missed.

The real win comes from context-aware routing. Your agent knows what it's doing:

const routeByContext = (message, context) => {
  // Writing code? Use the smart model
  if (context.mode === 'coding') return 'gpt-4';
  
  // Status updates? Use the cheap model
  if (context.lastAction === 'completed') return 'gpt-3.5-turbo';
  
  // First message in conversation? Use smart model
  if (context.messageCount === 1) return 'gpt-4';
  
  // Following up on previous work? Match the model
  return context.previousModel;
};

I track three metrics:

• Cost per conversation: Are complex conversations staying on the expensive model too long?
• Escalation rate: How often does a "simple" request need the smart model?
• Quality degradation: Are users complaining about dumb responses?

The sweet spot is 70% simple requests, 30% complex. If you're above 50% complex, your classifier is too conservative.

Here's the routing config I actually use:

const MODEL_ROUTING = {
  simple: {
    model: 'gpt-3.5-turbo',
    max_tokens: 150,
    temperature: 0.3
  },
  complex: {
    model: 'gpt-4-turbo-preview',
    max_tokens: 2000,
    temperature: 0.7
  },
  escalation: {
    model: 'gpt-4-turbo-preview',
    max_tokens: 1000,
    temperature: 0.5,
    system: 'Previous response may have been insufficient. Please provide a more detailed answer.'
  }
};

Notice the escalation tier. When a simple response gets a follow-up question, automatically route to the smart model. Saves face and saves money.

Warning: Don't route based on user identity. Route based on task complexity. Your VIP users don't want to pay for waste either.

The pattern works for any provider. Anthropic has Claude Haiku ($0.25/1M) and Claude Sonnet ($3/1M). Same 12x price difference.

Start with this: log every request for a week without routing. Then analyze what percentage could have used the cheap model. You'll be shocked.