Coding agents waste money on model overkill. We built a tournament to fix it.

I watched our coding agent pick Claude Sonnet 3.5 for everything. Database migrations? Sonnet. Simple bug fixes? Sonnet. Writing tests? Still Sonnet.

The bill was $400/month. The work quality was inconsistent. Some tasks took forever because we threw a Ferrari engine at bicycle problems. Others failed because we needed surgical precision but got creative interpretation.

Most teams pick their coding model the same way they pick their favorite coffee shop — gut feeling, brand loyalty, whatever worked last time. But coding tasks have wildly different complexity profiles, and the model benchmarks don't tell you which one actually works for your codebase.

Here's what we built instead: a 48-hour model tournament.

The Benchmark That Actually Matters

We took 20 real tasks from our backlog — bug fixes, feature additions, refactoring jobs, test writing. Mixed complexity. Then we ran each task through four different models:

• Claude Sonnet 3.5 (our default)
• GPT-4 (the safe choice)
• Claude Haiku (the budget option)
• Gemini Pro (the wild card)

Each model got the same context files, same prompts, same 30-minute time limit. We measured three things: completion rate, code quality (did it pass our review checklist), and cost per task.

The Results Broke Our Assumptions

Haiku dominated simple bug fixes. Faster, cheaper, cleaner code. Sonnet was overkill — it spent 15 minutes architecting solutions for 2-line fixes.

Sonnet crushed complex refactoring. Haiku got lost in the dependency graph and suggested changes that would break everything.

GPT-4 was the most consistent but never the best. Solid B+ across everything.

Gemini surprised us on test writing. It understood our testing patterns better than the others, probably because our codebase has a lot of public examples it trained on.

The Routing Rules That Cut Our Bill 60%

Now our coding agent routes by task complexity, not model preference:

# Simple tasks (< 5 files touched)
if task.complexity == "simple":
    model = "claude-haiku"
    
# Complex architecture (> 10 files, new patterns)
elif task.complexity == "complex":
    model = "claude-sonnet-3.5"
    
# Test writing specifically
elif "test" in task.type:
    model = "gemini-pro"
    
# Default fallback
else:
    model = "gpt-4"

We classify task complexity upfront using file count, dependency analysis, and keyword matching. Takes 30 seconds, saves 60% on costs.

Pro tip: Run your own tournament. Your codebase is different. Your tasks are different. The model that works for everyone else might be expensive overkill for you.

The best part? Our agent now finishes simple tasks in 3 minutes instead of 15. Turns out the right tool for the job isn't always the most powerful one.

Stop picking models based on marketing benchmarks. Build a leaderboard based on your actual work.