Big context windows are expensive compute, not infinite compute

Anthropic's new 1M token context window changes the game, but not how you think.

Most people see big context and think "finally, I can dump my entire codebase into the prompt." That's expensive thinking. At $15 per million input tokens, loading your whole project costs $0.015 per conversation. Multiply by 50 conversations per day and you're spending $22.50 daily just on context loading.

Here's what actually works: use big context for decisions, not storage.

Our coding agent now loads full context only when it needs to understand system architecture or trace dependencies across files. For routine tasks — bug fixes, feature additions, documentation — it uses our three-tier memory system with targeted file loading.

The pattern looks like this:

// Context routing logic
if (task.requires_architecture_understanding) {
  context = await loadFullCodebase()
  model = "claude-3-5-sonnet" // Big context, smart model
} else if (task.files.length > 5) {
  context = await loadRelevantFiles(task.files)
  model = "claude-3-5-sonnet" // Medium context
} else {
  context = await loadTargetFiles(task.files) + memory.getRelevant()
  model = "claude-3-5-haiku" // Small context, fast model
}

We route by context necessity, not context availability. The agent gets the full codebase when debugging a race condition across microservices. It gets targeted files when fixing a CSS bug.

This cut our context costs by 73% while making the agent faster on routine tasks. Haiku with 10K tokens beats Sonnet with 200K tokens for most day-to-day coding work.

The context window isn't infinite compute. It's expensive compute. Route accordingly.

Big context windows are powerful when you need them. They're wasteful when you don't. The trick is teaching your agent the difference.

Memory tiers handle the 80% case. Big context handles the 20% case where you actually need to see everything at once. Build systems that use both intelligently, and your agent gets smarter while your bills get smaller.