Our coding agent was burning 40K tokens re-reading the same docs every session

I thought our coding agent was finally stable. It was shipping clean PRs, following our style guides, and hadn't broken CI in weeks. Then I checked the logs.

The agent was burning through 40,000+ tokens per session just re-reading the same documentation files. Every time it needed to check our API patterns, it would re-ingest the entire 15,000-word architecture guide. When it needed a database query example, it would re-parse all 200 examples in our cookbook.

The fix wasn't smarter caching or a bigger context window. It was documentation that thinks.

Here's what I mean. Instead of dumping everything into one massive ARCHITECTURE.md file, we split our docs into callable chunks:

docs/
├── quick-reference/
│   ├── api-patterns.md       # 500 words, covers 80% of cases
│   ├── db-queries.md         # 300 words, common patterns only
│   └── error-handling.md     # 400 words, our standard approach
├── deep-dives/
│   ├── authentication.md     # Full OAuth flow details
│   ├── rate-limiting.md      # Complete implementation guide
│   └── monitoring.md         # Full observability setup
└── examples/
    ├── user-crud.md          # One complete example
    ├── payment-flow.md       # One complete example
    └── webhook-handling.md   # One complete example

The agent now starts with quick-reference docs (1,200 tokens total) and only pulls deep-dives when it hits something complex. Token usage dropped 70%.

But the real breakthrough was making docs contextually aware. Each quick-reference file starts with a decision tree:

# API Patterns Quick Reference

## When to use this doc:
- Building CRUD endpoints
- Need standard error responses
- Following our REST conventions

## When to check deep-dives instead:
- Custom authentication flows → see deep-dives/authentication.md
- Rate limiting implementation → see deep-dives/rate-limiting.md
- Complex validation logic → see examples/user-crud.md

## Standard Pattern:
[actual content here]

Now the agent knows what it's looking for before it starts reading. It's not just consuming documentation—it's navigating it.

Pro tip: Add a "Last updated" timestamp and "Confidence level" to each doc. The agent learns which patterns are stable (high confidence, use freely) vs experimental (low confidence, ask before implementing).

The pattern works beyond code docs. We restructured our business process docs the same way. Instead of a 50-page employee handbook, we have:

• Daily operations (what you need 80% of the time)
• Edge case procedures (what to do when things go wrong)
• Policy deep-dives (full context for complex decisions)

Our support agent went from re-reading the entire handbook for every escalation to pulling exactly the procedure it needed. Response time dropped from 8 minutes to 90 seconds.

The key insight: agents don't need comprehensive documentation. They need navigable documentation. Give them a map, not an encyclopedia.