Day 27 — Spend Test-Time Compute

Today Model Lab spends extra inference deliberately.

Choose a pattern

• generate → verify → targeted repair for structured violations;
• multiple candidates → deterministic scorer when a reliable scorer exists;
• planner → executor → verifier for decomposable work;
• self-consistency only when agreement correlates with correctness on your eval.

Every pattern needs a hard cap and explicit stop condition.

Keep feedback independent

A generator grading itself with the same prompt and evidence may repeat the same mistake. Prefer deterministic checks, tools, a separately calibrated grader, or human review for high-risk ambiguity.

Account before spawning

Candidate search can multiply model calls and memory. Reserve total calls, tokens, deadline, and cost before concurrent dispatch. Cancellation stops unnecessary candidates when a verified winner appears, but usage already incurred remains accounted.

Compare paired cases

Run baseline and candidate policy on identical cases. Report how many failures were repaired, how many valid answers were damaged, added latency, and added cost. Keep the policy only if the trade is explicit.

Break it deliberately

• verifier accepts the same flaw it was meant to catch;
• repair prompt loses the original evidence;
• candidates differ only cosmetically;
• loop expands budget after failure;
• canceled calls disappear from accounting.

Completion proof

Ship one bounded repair policy with ordinary, repairable, irreparable, provider-failure, and budget- exhaustion cases.

Deep reference: Test-time compute.

Next: Day 28 — Tune or Redesign? →.