MP-20 · Behavior Contract Compiler

This lab turns behavior specifications and completion conditions into executable Rust. You will not ask a model whether its own work is finished. You will evaluate a strict observation against a versioned behavior contract, grounded evidence, effect receipts, required artifacts, and resource budgets.

Time box: 3–6 focused hours

Requires: Rust Core contracts, provider-neutral capabilities, and the multimodal adversarial lab

Needs no: network, model provider, model weights, paid API, or accelerator

The behavior

Given a strict specification and one run observation, the evaluator must:

reject malformed or contradictory specifications before execution;
bind the observation to the exact specification identity;
require every precondition, invariant, and completion predicate;
ground passed predicates in their declared evidence kinds;
validate output schema, validation layers, artifacts, and receipts;
reject forbidden, unauthorized, or receipt-free effects;
enforce model, tool, time, cost, and output-byte budgets;
return complete, incomplete, or invalid_observation.

A contradictory specification is a construction error and therefore returns an error before those three observation verdicts are available.

Run the reference acceptance suite

From the companion root:

cd rust/rust-ai-engineering
cargo test -p mosaic-harness behavior
cargo run -q -p mosaic-harness --example behavior_contract_lab

The focused module has 15 tests. The executable evaluates 11 cases and writes:

target/labs/mp-20/
  1c9e4ffceffad318b940fe531e173295d9e0cc8dc7198ad3fb089d1f1667f89e.json

The pinned result is:

artifact bytes              3,079
artifact SHA-256             0c503414b0466fa354d321486b851447e3501bcbb33e00bc7b462cfd0a09a88e
specification SHA-256        1c9e4ffceffad318b940fe531e173295d9e0cc8dc7198ad3fb089d1f1667f89e
cases                        11
passed                       11
complete                     1
incomplete                   5
invalid observations         4
invalid specifications       1

The implementation is crates/mosaic-harness/src/behavior.rs. The executable is crates/mosaic-harness/examples/behavior_contract_lab.rs. The strict fixtures live under labs/harness/mp-20-behavior-contract/fixtures.

Predict before running

Classify every mutation first:

Case	Your prediction	First responsible boundary
valid complete
fluent output without checks
wrong specification identity
failed completion predicate
missing predicate evidence
unknown evidence reference
forbidden local write
unauthorized read
model-call budget exceeded
required artifact missing
required effect also forbidden

Do not use “fail” for every row. Explain why:

missing required evidence is an honest incomplete run;
a reference to evidence that does not exist makes the observation internally invalid;
a forbidden effect is a policy breach, not unfinished work;
requiring and forbidding the same effect makes the specification contradictory.

Trace one claim

Trace risk.supported through four joins:

PredicateSpec("risk.supported")
  ├── requires Text
  └── requires TestReceipt
          │
          ▼
PredicateObservation("risk.supported", passed = true)
  ├── evidence "source" ─► Text + digest + exact locator
  └── evidence "tests"  ─► TestReceipt + digest + verifier locator
          │
          ▼
CompletionReport.satisfied_predicates += 1

Now apply drop_required_evidence. The observation still says passed = true, but it references only the text source. The evaluator emits missing_predicate_evidence for the absent test receipt and returns incomplete. The model’s assertion does not manufacture the missing receipt.

Implement an extension

Add HumanApproval to EvidenceKind, then require it for one effect:

extend the enum without weakening strict parsing;
add a fixture receipt that names the approved scope;
make an effect predicate require human_approval;
add one case with matching approval;
add one case with an approval for a different effect or scope;
make the second case fail at a deterministic join;
retain all existing outcomes.

Do not treat a boolean approved: true as sufficient. An approval receipt needs identity, actor, scope, policy version, time or generation, and integrity protection appropriate to the system.

Break and diagnose

Choose one mutation and temporarily remove the rule that catches it. Run:

cargo test -p mosaic-harness --example behavior_contract_lab

The example test must fail because at least one actual classification no longer matches the fixture. Restore the rule and add the narrowest regression test that explains why it exists.

For the required investigation, use fluent-output-without-checks:

confirm the JSON output receipt is present and structurally valid;
inspect the report’s missing predicate IDs;
verify that the specification contains one precondition, one invariant, and one completion condition;
prove that no predicate observations or receipts were supplied;
identify the root cause as missing completion evidence, not prompt quality;
reject “tell the model to say the tests passed” as a repair;
restore independently produced evidence receipts.

Measure the evaluator

Generate valid observations with 1, 10, 100, and 1,000 predicates. Measure:

construction and serialization time;
evaluator wall time;
allocated bytes or peak resident change;
report size;
effect of missing versus complete evidence joins.

The reference uses ordered maps and sets to produce predictable joins and duplicate detection. Its primary work is approximately collection indexing plus predicate/evidence lookups; exact complexity still depends on string comparison and serialization. Report measured results instead of claiming constant factors from notation.

The hard 1,024-object bounds matter even without a provider. The parser and evaluator still accept untrusted collection sizes and strings. A provider-free component can exhaust memory or CPU.

Defend the design

Write a short review answering:

Why is InvalidObservation different from Incomplete?
Which predicates can deterministic code establish?
Which semantic predicates need an independent tool, grader, or human?
Why is an artifact digest not proof that its contents are correct?
What must be versioned before a stored report can support a release decision?
When should exceeding a budget be incomplete, failed, or a policy incident?
Why does an effect receipt prove occurrence but not necessarily authorization?
What would make the SHA-256 identity non-portable across implementations?

Acceptance rubric

Strong completion evidence includes:

15 or more focused tests passing;
all 11 reference mutations classified correctly;
one new extension with a before/after failing test;
a saved content-addressed report with evaluator and specification identities;
a measurement table with environment and method;
a root-cause investigation that separates symptom from violated invariant;
a design note that distinguishes claims, evidence, authority, and completion;
no hidden provider or network dependency.

Solution guidance

The reference implementation:

derives strict Serde contracts with deny_unknown_fields;
validates schema/evaluator versions before computing identity;
bounds identifiers, descriptions, locators, and collections;
rejects duplicate predicate, artifact, effect, receipt, and evidence references;
uses SHA-256 over the pinned serialized representation;
indexes observations once, then joins by stable IDs;
treats any invalid-observation violation as stronger than incomplete violations;
checks output usage agreement, artifacts, effects, and every budget;
serializes a report that includes specification and observation identities.

Other designs are valid. A database-backed evaluator, signed receipt graph, or generated JSON Schema may be better at larger scale. Preserve the observable contract: exact binding, independent evidence, explicit authority, bounded resources, stable version identity, and honest verdicts.