Behavior Specifications and Completion Conditions

A model can produce polished JSON, a confident conclusion, and the phrase “all checks passed” without completing the user’s task. Fluency is an output property. Completion is a relationship between a prior specification and independently observable facts after execution.

This distinction is the foundation of harness engineering:

                         written before execution
                  ┌────────────────────────────────┐
                  │ behavior specification         │
                  │ predicates · output · effects  │
                  │ artifacts · budgets · versions │
                  └───────────────┬────────────────┘
                                  │ evaluates
                                  ▼
run ─► trace ─► receipts ─► completion observation ─► deterministic evaluator
                                                          │
                                ┌─────────────────────────┼──────────────────────┐
                                ▼                         ▼                      ▼
                             complete                 incomplete        invalid observation

The evaluator does not prove that every semantic claim in the world is true. It proves a narrower, useful statement: this exact observation satisfies this exact contract under this exact evaluator release. Whether an individual predicate is trustworthy depends on the evidence producer behind it. A compiler receipt can establish compilation. A sandbox diff can establish that its observed filesystem snapshot did not change. A calibrated reviewer may assess tone. A human may authorize a financial effect. One undifferentiated passed: true cannot honestly represent all four.

This chapter builds MP-20, a provider-free behavior-contract compiler in the shared mosaic-harness crate. It uses strict Serde types, bounded collections, stable identities, typed evidence and effects, deterministic joins, and mutation tests. The result becomes the definition of done for later context, memory, tool, replay, evaluation, routing, and project milestones.

Learning objectives

By the end you will be able to:

• distinguish an objective, requirement, predicate, oracle, observation, receipt, and verdict;
• write behavior as observable conditions rather than implementation wishes;
• separate preconditions, invariants, completion conditions, and non-goals;
• define output shape without confusing structural validity with semantic correctness;
• bind a completion observation to the exact specification and evaluator release;
• ground predicate results in typed evidence rather than model assertions;
• distinguish required, allowed, forbidden, authorized, and observed effects;
• treat budgets as part of behavior instead of optional telemetry;
• explain complete, incomplete, invalid observation, invalid specification, failed, cancelled, budget-exhausted, and needs-review states;
• choose deterministic, model-based, human, and environment oracles honestly;
• implement strict Rust contracts without admitting unknown or contradictory input;
• detect duplicate IDs, dangling references, receipt mismatches, and version drift;
• build mutation cases that fail for one diagnosed reason;
• identify what a content digest proves and what it cannot prove;
• evolve a contract without making historical reports ambiguous.

Prerequisites and dependency order

Complete the Rust chapters on enums, errors, Serde contracts, collections, and public API design. Complete provider-neutral capabilities so “the model” is already an explicit dependency, not a global. Complete the multimodal evidence and adversarial chapters so typed evidence, locators, lineage, quarantine, and fail-closed mutations are familiar.

This chapter comes before context engineering and tools. Context selection is useful only relative to a behavior contract. Tool authorization is meaningful only when required and forbidden effects are explicit. A retry is justified only when the same completion condition might become true. Replay is trustworthy only when the specification, observations, and evaluator identity are preserved.

The sequence is therefore:

behavior contract
   ├── context required to attempt it
   ├── tools authorized to observe or act
   ├── budget available to the loop
   ├── receipts produced during execution
   └── evaluator deciding whether to stop

If you start with a prompt, you tend to define success as “the model returned something.” Starting with the contract makes the prompt one replaceable strategy for satisfying observable conditions.

Completion artifact

Run:

cd rust/rust-ai-engineering
cargo test -p mosaic-harness behavior
cargo run -q -p mosaic-harness --example behavior_contract_lab

The module has 15 focused tests. The example applies 11 mutations and writes a content-addressed report:

target/labs/mp-20/
  1c9e4ffceffad318b940fe531e173295d9e0cc8dc7198ad3fb089d1f1667f89e.json

Pinned output:

artifact bytes              3,079
artifact SHA-256             0c503414b0466fa354d321486b851447e3501bcbb33e00bc7b462cfd0a09a88e
specification SHA-256        1c9e4ffceffad318b940fe531e173295d9e0cc8dc7198ad3fb089d1f1667f89e
mutation cases               11
passed expectations          11
complete                     1
incomplete                   5
invalid observations         4
invalid specifications       1

The implementation is crates/mosaic-harness/src/behavior.rs; the executable is crates/mosaic-harness/examples/behavior_contract_lab.rs; the lab contract and fixtures are in labs/harness/mp-20-behavior-contract.

Start with an observable behavior, not a persona

Compare two task descriptions:

You are a world-class incident analyst. Review this repository carefully and write an insightful risk report.

and:

From snapshot S, return one JSON risk matching schema risk-report-v1. Cite an exact source span and a passing independent verifier. Perform at least one authorized read. Do not write to the repository or communicate externally. Complete within two model calls, two tool calls, five seconds, 20,000 cost microunits, and 4,096 output bytes.

The first may influence style. It does not define completion. The second exposes questions that can be answered by code:

• Was the specified snapshot available?
• Does the output parse?
• Does it conform to the required structure?
• Did semantic and evidence validators pass?
• Is the cited source exact and content-bound?
• Is there a verifier receipt?
• Was the repository unchanged?
• Was every effect authorized?
• Did any forbidden effect occur?
• Did resource usage remain inside the declared envelope?

A behavior specification should describe externally meaningful outcomes and constraints. It should not prescribe hidden chain-of-thought, demand a particular prose feeling, or assert an implementation detail that no observer can verify.

Normative language is useful only with scope

Specifications often use MUST, SHOULD, and MAY. BCP 14 defines these keywords, and RFC 8174 clarifies that their special meanings apply to uppercase forms. The keywords do not make vague prose executable. “The answer MUST be good” is still vague. “The output MUST validate against schema identity risk-report-v1” names a checkable condition.

For every normative statement, record:

1. the subject it constrains;
2. the phase where it applies;
3. the observation or receipt that can establish it;
4. who owns the oracle;
5. the result when it is false or cannot be measured.

Six different objects that teams often collapse

The word “requirement” hides several roles.

Object	Question	Example
Objective	Why is the run being attempted?	Identify one supported incident risk
Predicate specification	What fact must hold?	The risk has source and test evidence
Oracle	How can that fact be judged?	Citation join plus pinned verifier
Observation	What did this run report?	Predicate passed with evidence IDs A and B
Receipt	What stable evidence supports it?	Test result digest and source locator
Verdict	Does the whole observation satisfy the contract?	Incomplete: verifier receipt absent

An oracle is not necessarily a function inside the completion evaluator. The evaluator can verify that a required TestReceipt exists and is well formed, while a separate pinned verifier actually runs the test. In a higher-assurance system, the evaluator must also verify the receipt’s signature, producer release, subject identity, freshness, and scope. MP-20 deliberately starts with the join and type boundaries; later tool and policy chapters strengthen receipt authenticity.

The separation prevents circular reasoning:

unsafe
model writes answer ─► model says answer is correct ─► harness marks complete

stronger
model proposes answer
  ├── schema validator checks shape
  ├── source resolver checks citations
  ├── test runner checks executable claim
  ├── policy engine checks authority/effects
  └── completion evaluator joins independent receipts

“Independent” is relative to the failure being tested. Two graders using the same model, prompt template, and corrupted context may fail together. Independence may require different code, evidence, model family, reviewer, or environment.

Preconditions, invariants, completion conditions, and non-goals

These categories answer different temporal questions.

Preconditions

A precondition must hold before useful work can begin:

• the requested input snapshot exists;
• the tenant may access it;
• the required model capability is available;
• a schema version is supported;
• an approval exists for a planned high-impact action.

A false precondition should normally prevent execution. Discovering it after effects have occurred is too late. Some preconditions may expire, so a long run can need revalidation before an effect.

Invariants

An invariant must remain true across the relevant execution interval:

• repository digest remains unchanged during read-only analysis;
• every tool call has an authorized capability;
• tenant data never crosses its isolation boundary;
• resource reservations never underflow;
• derived evidence preserves lineage to an acquired root.

An invariant is stronger than an end-state preference. Checking only before and after can miss a temporary violation. Choose observation points that match the threat: capability checks per call, transaction constraints per write, or sandbox enforcement continuously.

Completion conditions

A completion condition establishes the requested outcome:

• a risk is supported by an exact source span and passing verifier;
• every requested file has a compiled result;
• the generated image passes dimension and safety gates;
• the incident timeline contains no unsupported causal edge;
• an external system reports the intended durable state.

At least one completion condition is required. Otherwise “nothing happened” could satisfy a set of preconditions and invariants.

Non-goals

Non-goals control scope but are not all forbidden effects:

• do not optimize the whole repository;
• do not infer business impact outside supplied evidence;
• do not migrate the database in this phase;
• do not guarantee legal correctness.

Represent a non-goal as a forbidden effect, explicit output exclusion, or documented limitation when it is mechanically enforceable. Do not create a fake predicate for an unobservable promise.

Output contracts cover layers, not merely JSON

JSON validity establishes syntax. A JSON Schema can establish structure: required properties, types, bounds, and combinations. Draft 2020-12 separates core and validation vocabularies and defines behaviors such as tuple validation and unevaluated items; pin the dialect rather than saying only “JSON Schema” (JSON Schema Draft 2020-12).

Neither syntax nor structure proves the answer is supported. MP-20 names five validation layers:

Layer	Establishes	Does not establish
Syntactic	Bytes parse as the representation	Required fields or domain truth
Structural	Shape satisfies the pinned schema	Semantic validity
Semantic	Domain constraints hold	Evidence supports claims
Evidence	Claims join to acceptable sources/receipts	Action authority
Policy	Content/effects satisfy current policy	Universal safety or truth

The reference contract requires at least syntactic and structural layers. It may require more:

{
  "schema_id": "risk-report-v1",
  "required_validation": [
    "syntactic",
    "structural",
    "semantic",
    "evidence"
  ],
  "required_artifacts": [
    {
      "id": "risk-report",
      "media_type": "application/json"
    }
  ]
}

The output observation repeats the schema identity and the layers that passed, plus a content digest and byte count. The evaluator rejects disagreement between output.bytes and usage.output_bytes. Two reports claiming different byte counts for the same output cannot both be accepted as one coherent observation.

At production depth, validation receipts should name validator release, schema digest, subject digest, result, time or sequence, and any environment assumptions. A list of layer names is the smallest useful contract, not the final trust model.

Evidence grounds predicates

The behavior specification declares the evidence kinds each predicate requires:

pub struct PredicateSpec {
    pub id: String,
    pub description: String,
    pub required_evidence: Vec<EvidenceKind>,
}

The observation does not embed arbitrary proof text in the predicate. It references evidence:

pub struct PredicateObservation {
    pub predicate_id: String,
    pub passed: bool,
    pub evidence_ids: Vec<String>,
}

pub struct EvidenceObservation {
    pub id: String,
    pub kind: EvidenceKind,
    pub artifact_sha256: String,
    pub source_locator: String,
}

For risk.supported, the fixture requires both Text and TestReceipt. A passing observation that references only text remains incomplete. A reference to an absent evidence ID makes the observation invalid. These outcomes differ:

• Incomplete: the run may be internally coherent, but required work or evidence is absent or failed.
• Invalid observation: the claimed record is malformed, contradictory, unauthorized, or cannot be interpreted safely.

Typed evidence kinds do not prove quality. A string labeled test_receipt is not automatically trustworthy. The next maturity steps are:

1. bind the receipt to the exact subject digest;
2. bind it to an oracle and release identity;
3. validate integrity or signature;
4. validate freshness and environment;
5. validate authority of the producer;
6. preserve raw evidence or protected reference for audit.

Multimodal predicates use the same idea without erasing modality. An image claim may require an image region, geometry map, OCR receipt, and source artifact. A video claim may require a native PTS range and canonical clock mapping. Do not flatten them into prose citations that cannot be restored to source.

Effects are not outcomes

An outcome describes what became true. An effect describes an interaction with the environment. Reading a file, writing a report, sending a message, charging a card, and deleting a resource have different authority and recovery properties even if each contributes to one outcome.

MP-20 starts with six effect classes:

pub enum EffectClass {
    ReadOnly,
    LocalWrite,
    ExternalCommunication,
    PrivilegedOperation,
    FinancialTransaction,
    IrreversibleChange,
}

The specification can require a minimum count and forbid classes:

{
  "required_effects": [
    { "class": "read_only", "minimum_count": 1 }
  ],
  "forbidden_effects": [
    "local_write",
    "external_communication",
    "irreversible_change"
  ]
}

This is intentionally conservative. Every observed effect must be marked authorized and carry a unique receipt. Any forbidden or unauthorized effect makes the observation invalid, even if the requested output is excellent.

Keep five concepts separate:

Concept	Meaning
Required	Completion needs at least this effect
Allowed	Policy permits it under named conditions
Forbidden	It must not occur in this contract
Authorized	A concrete attempt was approved for exact scope
Observed	A receipt says an attempt or effect occurred

An effect receipt alone is not authorization. An authorization alone is not proof of occurrence. Neither proves the intended durable outcome. Sending an HTTP request is not the same as the remote system committing the change. Later chapters model capability tokens, idempotency keys, result receipts, and postcondition observations separately.

Reject contradictory contracts before execution. Requiring ReadOnly while forbidding ReadOnly is an invalid specification, not an impossible task that should consume model tokens.

Budgets are behavioral limits

A harness that obtains the right answer after unlimited retries, unbounded cost, or a missed deadline did not satisfy the same product contract. MP-20 checks:

• model calls;
• tool calls;
• elapsed milliseconds;
• cost microunits;
• output bytes.

pub struct BehaviorBudget {
    pub max_model_calls: u32,
    pub max_tool_calls: u32,
    pub max_elapsed_ms: u64,
    pub max_cost_microunits: u64,
    pub max_output_bytes: u64,
}

This is a completion envelope, not yet a concurrency-safe reservation ledger. The observation reports consumed resources and the evaluator compares totals. A production loop must also reserve before expensive work so two concurrent branches cannot each spend the last available budget. It must identify which layer owns retries; otherwise provider, client, tool, and workflow retries can multiply silently.

Budget exhaustion is incomplete in this lab: the observation can be well formed while the desired behavior was not achieved inside its envelope. A product may instead route to NeedsReview, return a partial artifact, or record a failure. What matters is that policy is explicit and the harness never relabels over-budget completion as success.

Zero is domain-specific. This implementation permits zero tool calls and zero cost but requires a positive model-call, time, and output-byte bound. A fully deterministic task might need zero model calls, so a later generalized contract may represent disabled resources separately from bounded resources. That is an honest limitation of version 1.

Version identity is part of meaning

Suppose a report says risk.supported = true. Its meaning can change when any of these change:

• objective or predicate wording;
• required evidence kinds;
• output schema;
• validator or grader;
• allowed or forbidden effects;
• budgets;
• serialization rules;
• evaluator logic.

MP-20 includes a schema version and exact evaluator release in the specification. It validates both, serializes the strict Rust struct, hashes the bytes with SHA-256, and places the digest in the observation. The RustCrypto sha2 crate supplies the SHA-2 implementation (crate documentation).

pub fn identity_sha256(&self) -> Result<String, BehaviorSpecError> {
    self.validate()?;
    digest_serialized(self)
}

A hash establishes byte identity under a named serialization procedure. It does not establish:

• truth or quality of the specification;
• authorship or authorization;
• secrecy;
• freshness;
• semantic equivalence with differently serialized data;
• portability across implementations that order fields or encode values differently.

The current identity is stable inside the pinned Rust representation and serializer. Cross-language or long-term interchange should use an explicit canonical serialization or a signed envelope whose format is itself versioned. Never hash an in-memory map with unspecified order and call it a portable protocol identity.

Versioning needs migration policy:

• A historical report remains evaluated under its historical contract and evaluator.
• A changed contract gets a new identity even if its human label stays the same.
• A new evaluator should not silently reinterpret an old report.
• Compatibility adapters emit a new observation and preserve the original.
• Release gates name the contract and evaluator identities they accepted.

Strict Rust contracts fail before evaluation

The specification and every nested structure use Serde’s deny_unknown_fields. Serde documents it as a container attribute that rejects fields outside the declared contract (Serde attributes). This catches misspellings such as max_models_calls instead of silently ignoring the intended bound.

Strict parsing is necessary but insufficient. Domain validation then checks:

• supported schema and evaluator versions;
• non-empty, bounded, visible identifiers and descriptions;
• at least one completion condition;
• bounded collections;
• globally unique predicate IDs;
• unique evidence requirements and validation layers;
• syntactic and structural output validation;
• unique artifact IDs;
• unique required and forbidden effect classes;
• non-zero required-effect counts;
• no required/forbidden contradiction;
• positive and hard-bounded resource values.

This is why parse errors and specification errors remain distinct. JSON can be perfectly valid while the contract is contradictory.

pub fn validate(&self) -> Result<(), BehaviorSpecError> {
    if self.schema_version != BEHAVIOR_SPEC_SCHEMA_VERSION {
        return Err(BehaviorSpecError::UnsupportedSchema);
    }
    if self.completion_conditions.is_empty() {
        return Err(BehaviorSpecError::MissingCompletionCondition);
    }
    // Unique IDs, validation layers, effects, and hard bounds follow.
    Ok(())
}

The evaluator accepts borrowed references and returns a report. It does not mutate the specification or observation, call a provider, execute a tool, or read global state. Those properties make it simple to test and replay.

Evaluation as checked joins

Treat evaluation as a small relational program:

specification predicates ──join predicate_id── observation predicates
                                      │
                                      └──join evidence_id── typed evidence

required artifacts ────────join artifact_id──── observed artifacts
required effects ──────────group effect_class── observed effect receipts
output contract ───────────compare───────────── output receipt
budgets ───────────────────compare───────────── usage receipt
specification digest ──────compare───────────── observation binding

The implementation indexes observations into ordered maps while detecting duplicate IDs. It rejects unknown predicates and dangling evidence references before treating a passed predicate as grounded. It then checks output, artifacts, effects, and budgets.

Verdict precedence is important:

let verdict = if evaluation.invalid {
    CompletionVerdict::InvalidObservation
} else if evaluation.violations.is_empty() {
    CompletionVerdict::Complete
} else {
    CompletionVerdict::Incomplete
};

If one record is missing a required artifact and also contains an unauthorized effect, the result is invalid observation, not merely incomplete. Callers should not retry malformed or policy-breaching records as though they only needed more model effort.

The report preserves:

• evaluator release;
• specification label and digest;
• observation digest;
• final verdict;
• satisfied and required predicate counts;
• evidence, artifact, and effect counts;
• typed violations with subject and explanation.

Counts are diagnostic, not scores. Satisfying 9 of 10 required predicates does not mean “90% complete” unless the product has explicitly defined partial value. One failed safety invariant may dominate all other success.

Terminal states and truthful stopping

MP-20 observations can report:

• Completed;
• Failed;
• Cancelled;
• BudgetExhausted;
• NeedsReview.

Only Completed can yield a complete verdict, and only when every other condition passes. The terminal label alone is never sufficient.

Do not collapse these states:

State	Meaning	Typical next action
Complete	Exact contract satisfied	Publish or advance
Incomplete	Coherent run lacks required outcome/evidence/budget	Repair, retry, partial, or stop
Invalid observation	Record is unsafe or internally untrustworthy	Quarantine and investigate
Invalid specification	Requested contract cannot be evaluated safely	Fix before execution
Failed	Execution ended with a typed operational/domain error	Retry only by owned policy
Cancelled	Authorized cancellation won	Release resources; do not publish
Budget exhausted	Envelope ended before completion	Escalate, partial, or stop
Needs review	Machine authority or confidence is insufficient	Human decision

A harness stops when an external evaluator establishes completion, a hard terminal condition wins, or policy routes to review. “Continue until the model says done” gives the probabilistic component control over both proposal and acceptance.

Mutation design: one violated reason at a time

The lab starts with one valid fixture, clones it, and applies a named mutation. This preserves a diagnostic counterfactual: the valid and invalid cases differ at one responsible seam.

Mutation	Expected class	Why
none	complete	Every predicate, artifact, effect, and budget satisfies
clear predicates	incomplete	Output exists but no completion facts are established
wrong specification identity	invalid observation	Record belongs to another contract
fail completion predicate	incomplete	Required condition honestly failed
drop required evidence	incomplete	Passed claim lacks its verifier kind
reference unknown evidence	invalid observation	Observation contains a dangling join
add forbidden local write	invalid observation	Policy boundary was crossed
make read unauthorized	invalid observation	Occurrence lacks authorization
exceed model-call budget	incomplete	Outcome missed its resource envelope
remove required artifact	incomplete	Deliverable is absent
require and forbid read	invalid specification	Contract contradicts itself

This table is also a product decision. Another system might classify budget violation as Failed, or a forbidden effect as a separate PolicyIncident. The important rule is to decide before execution and test the exact classification.

Avoid mutations that change five fields simultaneously. If wrong schema, missing artifact, budget excess, and forbidden effect all occur, the evaluator may correctly report many violations, but the test does not identify which individual check regressed.

Failure investigations

1. Fluent output is incomplete

Symptom: the report is valid JSON and reads well, but verdict is incomplete.

Investigate: inspect missing predicate IDs before changing prompts. Confirm whether validation, source, test, policy, and artifact receipts exist.

Root cause: the output receipt proves representation and shape, not completion conditions.

Repair: run the responsible independent checks and attach their receipts. Do not ask the model to claim that checks ran.

2. Everything passes against the wrong contract

Symptom: every predicate ID appears satisfied, but verdict is invalid_observation.

Investigate: compare specification_sha256 before reading predicate details.

Root cause: a stale or cross-task observation was joined to the current specification.

Repair: evaluate it under its original contract or rerun the current contract. Never overwrite the stored digest.

3. A passed predicate lacks evidence

Symptom: passed = true, verdict incomplete.

Investigate: compare required evidence kinds with referenced evidence kinds.

Root cause: a claim is asserted but not grounded in the declared oracle outputs.

Repair: produce the missing receipt or mark the predicate failed/unknown. Do not weaken the contract merely to admit the run.

4. Unknown evidence makes the record invalid

Symptom: the evidence existed in an earlier trace, but the completion record is invalid.

Investigate: resolve every evidence ID inside the exact observation bundle and retention scope.

Root cause: a dangling reference prevents self-consistent interpretation.

Repair: include a protected reference that can be resolved, preserve the source bundle, or emit a new incomplete observation. Guessing by ID is unsafe.

5. Great result after a forbidden effect

Symptom: the intended risk report is correct, but an intermediate tool wrote a file.

Investigate: read effect receipts and authorization before output quality.

Root cause: the task outcome does not erase a policy violation.

Repair: quarantine the run, restore or compensate when possible, and execute in an enforced read-only environment. Do not relabel it as successful because the final bytes look good.

6. Receipt exists but authorization does not

Symptom: a read receipt is present and required count is met, yet observation is invalid.

Investigate: inspect the authorization decision’s subject, scope, capability, policy version, and lifetime.

Root cause: observation of an effect and permission for that effect were collapsed.

Repair: authorize before execution and bind authorization to the exact attempt. A fabricated post-hoc authorized: true field is not sufficient for production.

7. Counts disagree

Symptom: output says 256 bytes while usage says 240.

Investigate: identify whether counting occurred before/after encoding, compression, framing, or redaction.

Root cause: two receipts refer to different byte subjects or phases.

Repair: name representation and subject identity, then count once at the ownership boundary.

8. Historical reports change meaning

Symptom: rerunning an old observation produces a different verdict after a deployment.

Investigate: compare evaluator release, schema, canonicalization, validator releases, and policy.

Root cause: mutable evaluator behavior was addressed by a stable label.

Repair: preserve executable release identity or migration logic. Emit a new report; never mutate historical evidence invisibly.

9. Duplicates hide disagreement

Symptom: two observations use the same predicate ID with different results.

Investigate: detect duplicates before collecting into a map.

Root cause: ordinary map insertion silently replaced one record.

Repair: reject duplicates as invalid. An explicit aggregation contract is required if multiple oracle results are intended.

10. The evaluator becomes a denial-of-service target

Symptom: a provider-free validation endpoint consumes excessive memory or CPU.

Investigate: count collections, identifier sizes, locator sizes, nested references, and report violations.

Root cause: deterministic does not mean bounded.

Repair: enforce parse/body limits before allocation, hard collection and string bounds during domain validation, and bounded violation output.

MP-20 bounds inputs but does not yet cap the number of emitted violations independently. With 1,024 predicates and several faults each, the report can still grow materially. A network-facing version should cap detailed violations while preserving total counts and a truncation receipt.

Security implications

The completion evaluator belongs on the trusted side of the model boundary. Model output, retrieved content, tool observations, and memory are untrusted inputs.

Key rules:

• never let evidence text modify the behavior specification;
• never grant authority because content requests it;
• authorize effects before execution, not while evaluating the result;
• bind receipts to subject, producer, release, scope, and freshness;
• reject unknown fields and duplicate identities;
• keep secrets and raw sensitive content out of public completion reports;
• separate content digest from access authorization;
• preserve tenant and retention scope when resolving evidence;
• sign or integrity-protect reports when they cross trust boundaries;
• treat invalid observations as investigation inputs, not retry prompts;
• bound parsing, joins, locator resolution, and diagnostic output.

The NIST AI RMF describes measurement as documented test, evaluation, verification, and validation connected to deployment context, and notes that independent review can reduce conflicts of interest (AI RMF Core, Measure). A completion contract is one engineering mechanism for making those measurements explicit. It is not by itself a full risk-management program.

Performance implications

The reference implementation serializes the specification and observation for identities, builds ordered indexes, checks joins, and constructs a violation vector. Ordered maps make deterministic iteration and duplicate reasoning straightforward; hash maps may offer lower average lookup cost for large workloads but need deterministic output handling and collision-resistance considerations.

Measure at least:

• parse and domain-validation time;
• identity serialization and SHA-256 time;
• indexing and join time;
• peak allocation;
• report bytes;
• valid versus maximally invalid observations;
• predicate/evidence fan-out;
• cold and warm behavior.

Do not optimize before defining limits. A 1,000-predicate completion contract may be appropriate for batch evaluation and absurd for an interactive endpoint. Separate input admission from evaluation so oversized records fail before large allocations.

Caching a report is safe only under the complete identity of specification, observation, evaluator, and external oracle receipts. If authorization, revocation, or freshness is evaluated dynamically, a historical complete report may remain historical evidence without granting current permission.

Alternatives and trade-offs

Plain prose checklist

Useful during discovery and review; poor for automated joins, version identity, and replay. Keep prose descriptions alongside executable fields, not instead of them.

JSON Schema alone

Excellent for representation and structural constraints. It cannot by itself prove repository state, authorization, evidence grounding, or tool outcomes. Use it as one validation layer.

General policy engine

OPA/Rego, Cedar, or another policy system can make authorization and policy predicates expressive. That may be better for organization-wide rules. You still need subject/effect receipts, output validation, budgets, and a completion join. Avoid placing every semantic check into one opaque policy document.

Temporal/state-machine specification

TLA+, Alloy, typestates, or explicit transition systems can establish richer safety and liveness properties. They are valuable for durable workflows and concurrency. MP-20 is intentionally an end-observation evaluator plus declared invariants; it does not prove every intermediate state.

Model grader

Useful for semantic qualities that deterministic code cannot capture. It needs a rubric, pinned identity, calibration, abstention, disagreement handling, and held-out evaluation. Never make the same uncalibrated model sole author and judge for a high-impact claim.

Human review

Appropriate for ambiguous, high-impact, normative, or authority-bearing decisions. It is slower and variable. Capture reviewer scope, evidence shown, decision options, disagreement, and expiry rather than reducing review to an unscoped boolean.

Signed event/receipt graph

Best when observations cross services or organizations. It strengthens integrity and attribution but increases key management, canonicalization, and revocation complexity. It still cannot prove a dishonest producer’s claim is true.

Mini-project milestone: compile the definition of done

Complete MP-20 · Behavior Contract Compiler:

1. read the strict specification and complete observation fixtures;
2. predict every case classification;
3. trace risk.supported to text and test receipts;
4. run the 15 focused tests;
5. run the 11-case mutation executable;
6. inspect the content-addressed artifact;
7. add a human-approval evidence extension;
8. deliberately remove one evaluator rule and watch the example fail;
9. restore it with a focused regression;
10. measure 1, 10, 100, and 1,000 predicates;
11. write the design defense;
12. run the full workspace gate.

This lab becomes part of Forge. Later chapters will feed it context manifests, tool receipts, memory write receipts, replay identities, and evaluation results rather than replacing it.

Exercises

Recall and classify

1. Explain why valid JSON is neither a completion condition nor evidence of semantic correctness.
2. Classify each as precondition, invariant, completion condition, effect, or non-goal: input readable; repository unchanged; report cited; file read; do not optimize unrelated code.
3. Distinguish an oracle from a receipt.
4. Name three properties SHA-256 identity does not prove.
5. Explain why a forbidden effect dominates a missing artifact in verdict precedence.

Predict and trace

6. Predict the verdict for terminal_status = cancelled with all predicates present.
7. Trace a required Image predicate through artifact digest, region locator, geometry map, and vision-verifier receipt.
8. Draw the identities needed to replay a report after an evaluator upgrade.
9. Predict which error occurs when a specification has duplicate predicate IDs across preconditions and completion conditions.
10. Explain whether a budget violation should invalidate the observation or leave it incomplete in your product.

Implement and repair

11. Add a HumanApproval evidence kind and scope-mismatch mutation.
12. Add independent caps for total evidence references and emitted violations.
13. Bind artifact observations to output digest where the required artifact is the primary output.
14. Add an Unknown predicate result instead of overloading passed: false; define verdict rules.
15. Add a receipt producer release and subject digest, then reject mismatched subjects.
16. Add property tests: reordering semantically ordered vectors changes identity; duplicate IDs are rejected; adding an unknown JSON field fails.

Design and measure

17. Design a contract for code modification: compilation, tests, permitted paths, repository diff, and forbidden network effects.
18. Design a multimodal contract for answering a question about a video with temporal citations.
19. Design a generation contract where human preference matters but safety and file structure are deterministic.
20. Benchmark ordered versus hash indexes at 10, 100, 1,000, and the hard bound. Include stable output requirements in the comparison.
21. Propose a canonical cross-language identity format and migration plan.
22. Threat-model a remote service that accepts third-party receipt bundles.

Solution and review guidance

1. Syntax answers “can I parse it?” Completion answers “did the exact requested behavior occur under its constraints?”
2. Input readable is a precondition; repository unchanged is an invariant; report cited is a completion condition; file read is an effect; unrelated optimization is a non-goal.
3. The oracle performs judgment; the receipt is stable evidence of a particular oracle result.
4. A digest does not prove truth, authorship, authorization, secrecy, or freshness.
5. A policy breach makes the record unsafe to accept even if more ordinary work is also absent.
6. Cancelled remains incomplete because terminal status is not completed; product policy may expose cancellation separately outside the three completion verdicts.
7. Preserve the image artifact identity, source region, coordinate transform identity, and verifier subject/release. A prose “see image” citation is insufficient.
8. Preserve specification, observation, evaluator, validators/oracles, evidence artifacts, and policy/authorization releases.
9. Domain validation returns a duplicate predicate identifier before evaluation.
10. Either can be defensible. In MP-20, a truthful usage receipt over budget is coherent but incomplete. Falsified or contradictory usage is invalid.
11. Approval must bind actor, effect, subject, scope, policy, and validity window; type alone is not enough.
12. Reject or truncate deterministically before constructing unbounded diagnostics. Record totals and truncation.
13. Compare the required artifact’s digest and byte count with the output receipt when the contract declares that relationship.
14. Unknown is useful when the oracle could not run. Decide whether it routes to repair, retry, review, or incomplete; never coerce it to pass.
15. A well-formed receipt for a different subject cannot support the current predicate.
16. Separate identity properties from semantic equivalence. Some reorderings should change identity; if order is declared irrelevant, canonicalize it explicitly.
17. Permit only named paths/effects, require a clean baseline, record diff digest, compile/test receipts, and verify final repository state.
18. Require exact video artifact, PTS/canonical clock, sampled coverage, citation ranges, and evidence-verifier releases.
19. Deterministic validators can establish dimensions, encoding, policy screens, and provenance. Calibrated human/model preference can rank acceptable candidates without replacing safety gates.
20. Benchmark the whole evaluator including deterministic report construction; lookup timing alone is incomplete.
21. Pin field order, numeric/string normalization, set ordering, Unicode policy, and version. Test cross-language golden vectors.
22. Authenticate producers, verify signatures and subject scope, enforce tenant/retention policy, prevent replay, bound input, and distrust receipt content until validated.

Check your understanding

• Can a run be operationally Completed while its completion verdict is Incomplete?
• Can an observation be invalid even when every required predicate says it passed?
• What is the earliest safe time to reject a contradictory effect contract?
• Why must duplicate IDs be rejected before collecting into a map?
• What evidence would establish that a repository remained unchanged?
• When is before/after hashing insufficient for an invariant?
• Why should a model not control both proposal and acceptance?
• Which contract changes require a new identity?
• What makes a receipt independent enough for the failure being tested?
• How would you preserve current authorization separately from historical completion?

If any answer relies on “the model said so,” identify the missing oracle and receipt.

Primary references

• RFC 2119: requirement levels
• RFC 8174: capitalization clarification for BCP 14
• JSON Schema Draft 2020-12
• Serde container attributes
• Serde attributes and deny_unknown_fields
• RustCrypto SHA-2 crate
• NIST AI RMF Core: Measure
• NIST AI 600-1: Generative AI Profile

Definition of done

You are finished when you can demonstrate all of the following:

• the behavior is stated as observable predicates, effects, artifacts, and budgets;
• preconditions, invariants, completion conditions, and non-goals are distinct;
• strict parsing and domain validation reject unknown, duplicate, unbounded, and contradictory data;
• observations bind to the exact specification and evaluator release;
• every passed predicate carries its required typed evidence;
• output validation layers are not confused with evidence or policy;
• required effects are present and every effect is authorized and receipted;
• forbidden effects invalidate the observation;
• budgets are checked as part of completion;
• complete, incomplete, invalid observation, and invalid specification are explained and tested;
• the 11 mutation cases pass for diagnosed reasons;
• a content-addressed report is reproducible in the pinned workspace;
• performance and diagnostic growth are bounded and measured;
• limitations of hashes, receipt trust, and end-state evaluation are documented;
• the full companion quality gate passes.