model routing — four signals, three patterns
signals
1. task classification
2. prompt length
3. embedding to known-hard
4. self-confidence (logprobs)
combine all four; each alone misses
pattern: CASCADE (cheap-first)
cheap model (Haiku-class)
$0.25/M input, $1/M output
confidence check
frontier only if low-confidence
~10% escalated in mixed traffic
~2x latency on escalated
pattern: PRE_ROUTE
classifier up front (5-10ms)
cheap for simple
frontier for hard
fastest overall, quality floor depends on classifier accuracy
the 2026 price curve
GPT-4-level quality: $20/M (late 2022) → $0.40/M (2026) ≈ 50x cheaper
aggregate LLM price drop: ~10x per year, mostly serving efficiency (Phase 17 · 04-09)
routing captures those gains at the app layer
drift is the real risk
monitor: thumbs up/down per route · LLM-judge sample per route · escalation rate · refusal rate
cheap-model creep is silent until a competitor benchmark surfaces it