Context Manifests and Budget Allocation

A model context window is a finite execution resource. It is not a document folder and it is not a reason to concatenate everything the product knows. Instructions, demonstrations, conversation, retrieved evidence, memory, tools, media, protocol framing, reasoning, and output all compete for capacity. Which items enter the window is a product decision that changes model behavior.

If a harness assembles context with scattered push_str calls, provider-side truncation, or an unrecorded “top-k,” two runs cannot be compared honestly. You know the prompt release and model, but not the actual evidence and state available to the model. A fluent output may hide the fact that the one decisive log line was dropped.

The solution is a context manifest:

context policy
  window - output reserve - protocol reserve - safety reserve
                    │
candidate set       │
  exact identities  │
  token receipts    ▼
  lanes        deterministic allocator
  priority          │
  utility           ├── selected render order
  required          ├── typed drop reasons
  placement         ├── per-lane usage
                    ├── stable-prefix identity
                    └── manifest identity

The manifest is the receipt for one selection decision. It records what entered, what did not, why each optional item was dropped, which required item made a request impossible, how many tokens were reserved, and which allocator release made the choice.

This chapter builds mosaic-harness::context_budget and packages MP-21 · Context Budgeter. The reference is provider-free and fully deterministic. It uses recorded token-count receipts so the required lab runs without a network, model weights, paid API, or accelerator.

Learning objectives

By the end you will be able to:

distinguish model context window, input capacity, output reserve, protocol reserve, and safety reserve;
construct a versioned policy rather than relying on provider auto-truncation;
divide context into instruction, demonstration, conversation, evidence, memory, and tool lanes;
mark behavior-critical candidates required and treat overflow as a run error;
bind every token count to exact content and a named counter release;
explain why counting items independently does not prove the compiled provider request fits;
assign explicit priority and utility instead of using retrieval rank as a universal score;
implement deterministic required-first optional selection with stable ties;
preserve stable prefix placement for caching without changing semantic order;
emit selected and dropped decisions with typed reasons;
detect total-budget, lane-budget, and optional-count pressure separately;
hash policy, candidate set, stable prefix, and final manifest;
reproduce a stored manifest and detect mutation;
measure the quality regret of a greedy allocator against an exact solver;
evaluate context selection as part of product behavior.

Prerequisites and dependency order

Complete the context-cost chapter, behavior/run contracts, prompt versioning, and provider-neutral request construction. The context policy needs the model route’s real window and output bounds. Candidate identities come from prompt releases, conversation events, evidence envelopes, memory records, and tool schemas. The selected order becomes input to the provider compiler.

This chapter comes before retrieval because a retriever produces candidates; it does not own the entire window. It comes before memory because memory competes with evidence and conversation. It comes before tool design because tool declarations and tool results consume context too.

Completion artifact

Run:

cd rust/rust-ai-engineering
cargo test -p mosaic-harness context_budget
cargo run -q -p mosaic-harness --example context_budgeter_lab

Pinned result:

context window                 1,024 tokens
output reserve                   256
protocol reserve                  64
safety reserve                    64
input capacity                   640
selected input                   630
remaining input                   10
stable prefix                    260
selected candidates              instruction, demo, latest-user, crash-log, memory
mutation cases                     9
policy SHA-256                  95079edc3cc835f5496e4b386ce02f3f7a319cae2914004931fa70752a0a88c5
candidate-set SHA-256           f8a9903857d9402e9cb756c30e7eae247319c904414fd5c648f1eec00d7184f1
manifest SHA-256                0cb9127ebd297b75d93ee2da217cf6f9b014d4708389d4919fb3e3bff6c300b8
artifact bytes                  4,821
artifact SHA-256                7a00e2210a1803c753ea7e552b51d4b6acf6aa0f85152ed83d368a7f57c4a4d2

Fifteen module tests plus one example test cover exact reserve arithmetic, deterministic selection, required total/lane overflow, weak count rejection, total/lane/optional-count drop reasons, priority-density ties, stable-prefix identity, count/content drift, duplicate IDs/positions, placement hoisting, lane completeness, strict decoding, and stored-manifest mutation.

Derive input capacity before selecting content

Let:

(C_w) be the route’s total context window;
(R_o) be reserved output;
(R_p) be provider protocol/framing overhead;
(R_s) be uncertainty and safety headroom.

Then:

C_input = C_w - R_o - R_p - R_s

The reference computes every subtraction with checked arithmetic and requires a positive result. For the lab:

1,024 - 256 - 64 - 64 = 640

These are independent reserves.

Output reserve

The output reserve protects the completion budget promised by the behavior contract. Filling the window with input and hoping the provider finds room for an answer is not valid allocation. When the product requires a 2,000-token structured report, reserve that capacity before retrieval.

Some APIs describe input and output maxima separately; others document a combined context. The route adapter must translate provider/model limits into the policy. Do not assume a marketing “context length” means the same thing across APIs.

Protocol reserve

Candidate content is not the whole provider input. Tokenization may add:

message-role markers and separators;
system/developer framing;
content-block metadata;
tool names, descriptions, and schemas;
structured-output schema;
image/audio/video accounting;
special start/end tokens;
provider-managed conversation state.

The protocol reserve covers known framing that candidate-level counts omit. It is versioned with the adapter and model. After compiling the provider request, count the complete request when the provider offers that operation.

Safety reserve

The safety reserve is headroom for count uncertainty and small representation changes. It is not permission to use an inaccurate counter indefinitely. Track:

observed provider input - predicted provider input

by adapter/model release and choose headroom from the error distribution. If the margin repeatedly hides a systematic error, fix the counter.

Google’s current token-counting API can count a complete GenerateContentRequest, including system instructions, tools, and multimodal input. Its token guide also states that the context window limits combined input and output. Anthropic exposes a token-counting endpoint, and OpenAI documents a counting-tokens guide. Use current route-specific operations; never treat a character heuristic as permanent truth.

A token count is a receipt, not an integer

The reference stores:

pub struct TokenCount {
    pub tokens: u32,
    pub method: TokenCountMethod,
    pub counter_release: String,
    pub counted_content_sha256: String,
}

pub enum TokenCountMethod {
    ProviderCount,
    LocalTokenizer,
    ConservativeEstimate,
}

The content digest prevents reusing a count after text, media detail, tool schema, or normalization changes. The counter release explains which tokenizer/provider route produced the number. The method distinguishes an exact count from an estimate.

Hugging Face Tokenizers documents a pipeline of normalization, pre-tokenization, model, and post-processing. Its Encoding retains IDs and offsets. Changing any component, special-token policy, or tokenizer file can change counts. Pin the tokenizer artifact and configuration rather than writing “BPE count” in a trace.

Exact does not mean universal

A local tokenizer can be exact for a pinned local model input representation. It may not be exact for a remote provider that adds hidden framing. A provider count can be exact for the submitted request at one API/model release. It may not reproduce after the provider changes preprocessing.

“Exact” means exact under named evidence. Record the scope.

Required items need stronger counts

The lab policy rejects a required candidate whose method is only ConservativeEstimate. A required instruction cannot be silently dropped if the estimate was wrong. Either obtain a strong count, increase explicitly justified headroom, or fail before inference.

Optional candidates can use conservative estimates if the policy permits. Overestimation may reduce recall; underestimation may exceed the final request. Measure both effects.

Candidates are typed potential context

Each candidate records:

pub struct ContextCandidate {
    pub id: String,
    pub lane: ContextLane,
    pub trust: ContextTrust,
    pub placement: ContextPlacement,
    pub required: bool,
    pub priority: u16,
    pub utility_microunits: u64,
    pub original_position: u32,
    pub content_sha256: String,
    pub provenance_sha256: String,
    pub token_count: TokenCount,
}

The allocator does not store raw content. It selects immutable identities. Rendering later resolves those identities under authorization and verifies their digests.

Field	Purpose
ID	Stable join key within the candidate set
Lane	Independent capacity class
Trust	Instruction/user/evidence handling and security policy
Placement	Stable cache prefix or run-variable body
Required	Must be selected or allocation fails
Priority	Product/business ordering before efficiency
Utility	Comparable benefit signal within a policy
Original position	Semantic order and deterministic ties
Content digest	Exact bytes/representation counted
Provenance digest	Source and derivation identity
Token receipt	Cost and counter evidence

Do not derive required from a high retrieval score. A user request, safety rule, output schema, or current tool result may be required even when it has no semantic similarity score. Conversely, retrieved evidence with a high score can remain optional.

Lanes stop one source from consuming the whole window

The policy defines exactly one maximum for each lane:

instructions;
demonstrations;
conversation;
evidence;
memory;
tools.

For lane (l):

sum(tokens(selected item i where lane(i) = l)) <= lane_max(l)

The total input constraint still applies across all lanes.

Why a total budget is insufficient

Without lane limits:

retrieval can fill the window and remove conversation;
hundreds of tools can crowd out task evidence;
long history can remove the current user request;
demonstrations can consume output-critical space;
stale memory can overwhelm fresh observations.

Lane limits express product architecture. They do not guarantee good selection, but they make crowding observable and reviewable.

Hard maximum versus soft target

The reference implements hard maximums only. A production allocator may add:

minimum allocations;
soft target plus shared overflow pool;
per-modality sub-lanes;
tenant/product-specific policies;
adaptive output reserve;
task-type lane profiles.

Minimums can waste capacity when a lane has no useful candidate. Shared pools improve utilization but make behavior less obvious. Begin with a strict policy, measure drops and quality, then introduce complexity behind a new allocator release.

Required-first is a correctness rule

The allocator admits required candidates in stable original order before considering optional items. If any required item violates total or lane capacity, allocation returns an error:

RequiredTotalBudgetExceeded
RequiredLaneBudgetExceeded(ContextLane)

It never records a required item as “dropped.”

That distinction matters. Optional evidence can be absent while the task remains valid. A missing current user turn, behavior instruction, output schema, policy rule, or unresolved tool observation means the planned inference no longer represents the task.

When required context does not fit, possible higher-level actions include:

choose a larger compatible route;
split the task;
use a verified summary with new provenance;
reduce the output requirement through a new behavior contract;
ask the user to narrow scope;
fail explicitly.

The allocator must not choose among these silently.

The optional selector is deterministic and deliberately simple

After required admission, optional candidates are ordered lexicographically by:

higher priority;
higher utility per token;
higher absolute utility;
fewer tokens;
earlier original position;
stable candidate ID.

Each candidate is then either selected or assigned one reason:

TotalBudget
LaneBudget
OptionalItemLimit

Density comparison uses integer cross multiplication:

utility(a) / tokens(a) > utility(b) / tokens(b)

is compared as

utility(a) * tokens(b) > utility(b) * tokens(a)

This avoids floating-point drift. Products use checked widened arithmetic.

Priority comes before density

Priority represents a policy tier. A safety-relevant or task-critical optional item should not lose to a cheap but marginal item merely because its utility-per-token ratio is lower. Within one priority, density spends capacity efficiently.

Priority and utility must be calibrated. If every team assigns its candidates priority 1,000, the policy has no meaning. Own the scale centrally and inspect slice-level drop rates.

This greedy algorithm is not globally optimal

The reference does not claim to solve multidimensional knapsack optimally. A combination of two medium-density items can produce more total utility than the first high-density item. Lane limits make the optimization harder.

That limitation is a teaching feature:

the algorithm is easy to trace;
its worst-case work is bounded by candidate sorting;
tie behavior is explicit;
quality regret can be measured against an exact solver on small fixtures;
a new solver gets a new release identity.

Do not replace an understandable heuristic with an unbounded optimizer in a request path without latency and state-space limits.

Render order and cache prefix are different from selection order

Selection order decides admission. Render order preserves declared placement and original order:

selected stable-prefix items by original position
then
selected variable-body items by original position

The validator rejects a candidate set where a stable-prefix item originally appears after variable content. Hoisting it would change semantics.

The lab renders:

stable prefix:
  instruction (120)
  demo        (140)

variable body:
  latest-user ( 90)
  crash-log   (180)
  memory      (100)

OpenAI’s current prompt-caching guide states that cache hits require exact prefix matches and recommends placing static content before variable content. Anthropic’s prompt-caching guide also exposes explicit cache boundaries. These mechanisms differ; the manifest records product-level prefix intent, while adapters implement current provider rules.

The stable-prefix digest binds candidate ID, content digest, counted tokens, and counter release. It can group runs that intended the same prefix. It does not prove a provider cache hit. Record provider-reported cached input separately.

The manifest records negative decisions

The completed manifest includes:

policy and candidate-set digests;
input capacity, selected tokens, and remainder;
stable-prefix tokens and digest;
total selected utility;
selected IDs in render order;
one decision per candidate;
one usage record per lane.

Keeping dropped items is essential. When a model misses a fact, debugging should answer:

not retrieved?
retrieved but rejected by policy?
dropped by evidence lane?
dropped by total capacity?
dropped by optional-count limit?
selected but ignored by model?

A list containing only selected IDs collapses those different failure classes.

verify_context_manifest recomputes allocation from the exact policy and candidate set and compares the complete manifest. Changing selected utility, a drop reason, render position, or usage breaks reproduction.

Walk through MP-21

The lab starts with 640 input tokens and seven candidates:

Candidate	Lane	Required	Priority	Tokens	Outcome
instruction	Instructions	Yes	1,000	120	Selected
demo	Demonstrations	No	700	140	Selected
latest-user	Conversation	Yes	1,000	90	Selected
crash-log	Evidence	No	900	180	Selected
screenshot	Evidence	No	850	220	Dropped: lane
old-turn	Conversation	No	400	160	Dropped: total
memory	Memory	No	300	100	Selected

Required admission uses 210. crash-log raises total to 390 and evidence usage to 180. screenshot would raise evidence usage to 400, above its 320 lane maximum, so it is dropped even though total capacity could still fit it at that moment. demo raises total to 530. old-turn would reach 690, so it drops for total capacity. memory reaches 630, leaving 10.

This example proves why drop reason depends on state at the time of deterministic consideration. Changing priority changes both selection and later failure reasons; therefore priority is part of the candidate-set identity.

Mutation suite

The formal lab executes nine cases:

Mutation	Expected result
None	Allocated
Inflate a required item past total input	Required-total error
Inflate required instruction past lane	Required-lane error
Mark required count conservative	Weak-count error
Change counted digest only	Count/content mismatch
Duplicate candidate ID	Candidate-set error
Move stable item after variable content	Placement-order error
Baseline lane pressure	Allocated with lane drop
Change stored manifest utility	Manifest mismatch

Mutation tests are stronger than happy-path snapshots. Each one proves that a named invariant is actually enforced.

Re-count the fully compiled request

Candidate allocation is necessary but not the final admission check. The full pipeline is:

select candidate identities
  → resolve exact content
  → render prompt/messages
  → compile provider wire
  → count complete provider request
  → compare with route input capacity
  → submit

If the final count exceeds capacity:

do not let the provider truncate;
record predicted and actual counts plus counter/adapter releases;
return a typed preflight failure;
update reserve/counter policy or run a new declared allocation pass;
preserve the failed manifest for diagnosis.

Re-running allocation after observing full count needs a bounded strategy. Otherwise adapters can oscillate: drop item, framing changes, add item, exceed again. A simple policy allows one conservative repair pass that only removes optional items and never re-adds them.

Security analysis

Selection is an authorization boundary

A candidate being retrievable does not make it authorized for this run. Before allocation, enforce tenant, principal, purpose, data residency, retention, and sensitivity scope. The provenance digest does not grant access.

Utility is attacker-influenced

Retrieved documents can manipulate lexical/semantic scores. Conversation and memory can be poisoned. Never allow untrusted content to assign its own priority, required flag, trust class, or lane. Those are computed by trusted harness stages.

More context increases attack surface

Every extra document, screenshot, transcript, and memory record can contain indirect instructions. Lane budgets reduce exposure but are not prompt-injection defenses. Preserve trust labels through provider compilation, restrict tool authority independently, and evaluate adversarial candidates.

Dropped secrets still require safe traces

The manifest should store IDs, hashes, costs, and decisions—not raw secret content. A dropped item can still be sensitive. Debug UIs must re-authorize any content resolution.

Performance and operational analysis

The reference allocator sorts at most 128 candidates and performs one pass. Its memory is bounded by candidate and decision vectors. Important production metrics include:

candidate count by lane;
selected/drop count by reason and lane;
required-overflow rate;
predicted versus provider-counted tokens;
reserve utilization;
remaining input tokens;
stable-prefix length and provider cache hit;
selected utility and exact-solver regret on sampled cases;
allocation latency and allocations;
quality metrics sliced by policy/allocator release.

Avoid high-cardinality metric labels such as candidate ID or content digest. Those belong in traces. Metrics use bounded lane, reason, provider, and policy-release labels.

Failure investigations

Provider rejects a manifest that fits

Likely cause: candidate counts omit protocol, media, tools, or special tokens.

Compare the full provider count with:

selected candidate tokens + protocol reserve

Record the delta. Update the adapter/counter release; do not silently increase a global margin without a measured bound.

Critical evidence is dropped for lane pressure

Do not immediately enlarge the evidence lane. Check whether:

it should have been marked required;
duplicates consumed capacity;
chunk size is excessive;
retrieval utility is poorly calibrated;
a lower-priority item entered first;
another route fits the task.

Then add a regression case and release a new policy.

Cache hit rate falls after personalization

Inspect stable-prefix candidates and order. User-specific memory may have been marked stable or inserted before shared demonstrations. Move it to variable body only if semantic order remains valid. Provider cache keys and model releases must also match.

Greedy selection has high quality regret

Construct small cases, solve them exactly offline, and compare selected utility plus downstream behavior. If regret matters, implement a bounded dynamic program or beam search with explicit state/time ceilings and a new allocator identity.

Exercises

Recall

Why is output reserve subtracted before selecting input?
What does a token-count content digest prevent?
Why is required overflow an error rather than a drop?
How can a candidate fit total capacity but fail allocation?
What does the stable-prefix digest prove?
Why does a candidate-level exact count not prove the request fits?

Implementation

Add per-item maximum tokens and a focused required/optional failure pair.
Add a one-pass final-count repair that may remove optional candidates only.
Add an exact 0/1 knapsack oracle for at most 20 optional candidates and report greedy regret.
Add duplicate-content detection that distinguishes exact duplicate from overlapping source ranges.
Add a soft lane target plus shared overflow pool while keeping hard maxima.
Add a provider-count receipt for the fully compiled Chapter 4 request.
Benchmark 8, 32, 64, and 128 candidates.

Design

Design allocation for video where frame count and text-equivalent tokens are route-specific.
Design a policy that preserves the current user turn and most recent unresolved tool result.
Decide whether verified memory belongs in stable prefix. Analyze cache reuse versus freshness.
Specify an adaptive output reserve without allowing output starvation.
Design a dashboard that reveals systematic lane starvation without exposing content.

Solution guidance

Input and output share finite capacity or independent provider limits; behavior requires output room.
It prevents applying a stale count to changed content.
Dropping it changes the task contract; orchestration must choose a declared alternative.
Its lane can be full.
It proves the manifest selected the same versioned prefix descriptors, not that provider bytes or cache state were identical.
Combined framing and boundary tokenization can add or change units.
Check required items with errors and optional items with typed drops.
Make repair monotonic: only remove, cap passes, record both manifests.
Use the oracle offline or under a strict small-state limit; never let request latency grow exponentially.
Compare artifact digest plus locators; overlapping chunks require interval logic.
Consume lane target first, then a bounded shared pool, while retaining per-lane hard caps.
Bind count to provider route and exact wire digest.
Record median/tail time and allocations, not one warm timing.
Treat sampling policy and media detail as part of candidate identity and count receipt.
Mark them required, reserve their lanes, and fail if they cannot fit.
Only immutable, broadly reusable verified memory belongs in stable prefix; user/run state is variable.
Base it on output contract/task class with a hard minimum and bounded adjustment.
Plot bounded lane/reason/policy labels and inspect traces under authorization.

Check your understanding

Can a manifest be valid with zero remaining tokens?
Can an optional item be dropped before a lower-priority item is selected?
Does a high retrieval similarity make evidence required?
Which identity changes when only max_optional_items changes?
Why does the allocator retain old-turn after dropping it?
What operation should happen after provider wire compilation?
When does cache-prefix optimization become a semantic bug?

Completion checklist

Authoritative references

Definition of done

You are done when the 1,024-token lab policy deterministically produces a 640-token input capacity and 630-token selection; all required items survive or allocation errors; screenshot and old-turn drop for different, correct reasons; the 260-token stable prefix is content-bound; nine mutations prove the named invariants; the stored 4,821-byte artifact reproduces exactly; and you can explain both the reference greedy allocator’s value and its non-optimality.