Day 26 — Route Small Models Deliberately

Today Model Lab makes a small model useful by narrowing its job and surrounding it with evidence and verification.

Start from the eval set

Partition frozen cases by observable difficulty: short extraction, long synthesis, clean structured input, conflicting evidence, tool requirement, and adversarial content. Run every candidate route on the same cases.

Build a cascade

request ─▶ deterministic rule/tool?
             ├─ yes: execute and verify
             └─ no: small model ─▶ verifier
                                  ├─ pass: accept
                                  └─ fail/uncertain: stronger route

The router consumes declared capabilities, context size, modality, policy, measured calibration, and budget. Do not route only from the model’s self-reported confidence.

Decompose before escalating

Replace “reconstruct the incident” with bounded subtasks: extract timeline candidates, link citations, classify contradictions, then synthesize. A smaller model may perform each narrow step reliably when the whole task fails.

Measure the frontier

Report quality, safety, p50/p95 latency, cost, and escalation rate. A cascade that saves 50% cost but silently loses rare safety cases is a regression.

Run:

cd rust/rust-ai-engineering
cargo run -q -p mosaic-evals --example capability_calibration
cargo run -q -p mosaic-harness --example capability_adapters

Completion proof

Define one route rule, its calibration evidence, fallback, and rollback. Add cases near the threshold and prove monotonic behavior where appropriate.

Deep references: Decomposition and Verification and routing.

Next: Day 27 — Test-Time Compute →.