Strict Replay and Forked Replay

A second run with the same prompt is not a replay. It is another experiment.

The model may sample different tokens. A provider may silently deploy a new revision. Search results, clock readings, remote files, permissions, and tool state may have changed. Even with temperature zero, concurrency, floating-point kernels, batching, and service-side implementation details can change an observation.

Strict replay makes a narrower and stronger claim:

Given the exact recorded observations, dependency releases, causal records, receipt flow, and initial state, the deterministic harness produces the same state transition after every event.

Forked replay asks a different question:

Keeping an authenticated parent run and prefix fixed, what changes after one explicitly authorized dependency is replaced inside an isolated, effect-free replay?

Those definitions turn replay from a demo button into an engineering instrument. Strict replay detects drift in deterministic system logic. Forked replay supports controlled counterfactual experiments. Neither pretends that a stochastic model can be reconstructed from a prompt alone.

The reference implementation is mosaic_harness::replay. Its companion strict_and_forked_replay compiles a causal trace, replays five frames, forks after the recorded model observation, replaces an evaluator release, and proves the first divergence occurs at validation.

Learning objectives

By the end you will be able to:

distinguish playback, live rerun, deterministic re-execution, strict replay, and forked replay;
inventory every dependency that can influence a replayed decision;
replace nondeterministic model and tool calls with authenticated recorded observations;
make time, randomness, ordering, environment, and reducer releases explicit inputs;
bind replay frames one-to-one to secret-safe causal records and receipt flow;
derive a deterministic state chain and verify a terminal replay receipt;
prohibit live effects while still verifying their recorded receipts;
construct an immutable fork prefix with authorized, exact old-to-new overrides;
identify the first state divergence without claiming causal certainty beyond the experiment;
investigate missing artifacts, silent dependency drift, ordering failures, and dishonest forks;
design retention, access, privacy, performance, and migration policies for replay evidence.

Completion artifact

Run:

cd rust/rust-ai-engineering
cargo test -p mosaic-harness replay
cargo test -p mosaic-harness --example strict_and_forked_replay
cargo run -q -p mosaic-harness --example strict_and_forked_replay

Pinned reference result:

source causal records                  5
strict replay frames                   5
fork point                             after frame 3
first divergence                      frame 4
mutation cases                        12 / 12
policy SHA-256                         1ce4d46ec7ea7d8c3ab07f67cad8cb00713228ea3445e138c3ba4c379689a4fb
strict receipt SHA-256                 145631e33f5f849d0f592d889b7dda296360e5787aaeddaae6faf611e5760f96
fork plan SHA-256                      c69148c7238e0e94456314032a817a06d1d7978debd5a3952fcb3204fdb23ae2
fork receipt SHA-256                   c002f57ffccf70064aa31d4668fb91ffb5e6707583a47d3401cf92f277feda36
strict final state SHA-256             1ea467bada636ad433ed03a0744b86c15cde4c990ddf96702c101af98bfdc9b8
fork final state SHA-256               6080779d264e471982d6d209ed56275f2405d4d772e115d219ccae29ce945f13
report bytes                           2,425
report SHA-256                         d2b1caa6c7049a488f7ed2688a5dec96903e9841cc1e83dfdb92d6e7f65db658

Nine module tests and two example tests exercise successful replay, trace and frame drift, state drift, missing artifacts, live-effect rejection, receipt mutation, authorized divergence, stale parents, non-forkable dependencies, unused overrides, and deterministic report serialization.

Five operations people call “replay”

The word is overloaded. Name the operation before claiming a guarantee.

Operation	External calls	Recorded observations	Main question
UI playback	no	displayed	What appeared to happen?
Live rerun	yes	usually no	What happens now?
Seeded rerun	yes or local	maybe	Is this implementation approximately repeatable?
Strict replay	no	required	Does deterministic harness state reproduce exactly?
Forked replay	no	prefix fixed, suffix controlled	What changes under one authorized substitution?

Playback can animate a trace without executing reducers. It is useful for inspection but proves nothing about code behavior.

A live rerun sends requests again. It is valuable for regression or availability testing, but the world has changed. Never label its output “the replayed answer.”

A seeded rerun controls a random-number generator. That is helpful only when the seed controls all relevant randomness and the runtime, kernels, ordering, inputs, and releases remain compatible. Model-provider APIs rarely promise that.

Strict replay consumes the recorded boundary observations and reruns only deterministic logic. It does not contact the original provider, fetch today’s document, read the wall clock, or execute a tool.

Forked replay preserves the strict parent and a prefix, then substitutes an approved forkable dependency. It is a causal experiment over a recorded world, not a historical statement about what would certainly have happened.

The replay boundary

Place the boundary where nondeterminism enters the harness:

live run

model provider ── observed response receipt ─┐
tool system ──── observed result receipt ────┼── deterministic harness reducer
clock/random ─── recorded values ────────────┤        │
retrieval ────── snapshot-bound evidence ────┘        ▼
                                                committed state receipt

strict replay

recorded model observation ──────────────────┐
recorded tool observation ───────────────────┼── same reducer release
recorded time/random values ─────────────────┤        │
recorded retrieval snapshot ─────────────────┘        ▼
                                                same state receipt

Anything left outside the recording boundary can make replay diverge. Anything unnecessarily placed inside it increases storage, privacy, and retention cost. The boundary is therefore a design decision, not an afterthought.

The reference records observations rather than reissuing operations:

pub enum ReplayOperationClass {
    DeterministicReducer,
    RecordedModelObservation,
    RecordedToolObservation,
    EffectReceiptVerification,
    ExternalEffectDispatch,
}

ExternalEffectDispatch is rejected in both strict and forked replay. A replay may check that a recorded authorization and effect receipt were valid. It must not send the email, charge the card, write the database row, or call the actuator again.

Dependency completeness

A replay policy is a manifest of facts required to interpret and reduce the trace:

pub struct ReplayDependency {
    pub dependency_id: String,
    pub kind: ReplayDependencyKind,
    pub release_sha256: String,
    pub required: bool,
    pub forkable: bool,
    pub availability: ReplayAvailability,
}

The reference requires at least:

run envelope;
behavior specification;
policy bundle;
environment contract;
reducer release.

A real system will usually add prompt program, tokenizer, schema bundle, evaluator, tool contract, model observations, retrieval snapshot, memory snapshot, locale data, feature flags, and configuration. For local inference, also bind model weights, architecture configuration, runtime, kernel family, quantization plan, and device-class assumptions.

Release identity is not availability

A digest says which bytes are required. It does not prove the bytes can still be loaded.

The manifest therefore separates:

release_sha256     identity of the dependency used by the original run
availability       recorded receipt, local artifact, or unavailable

Preflight must fail before frame one when a required artifact is unavailable. Partially replaying and then substituting “something close” produces persuasive fiction.

Completeness test

For every branch in the reducer, ask:

Can this value differ between executions?
If yes, is it:
  1. derived only from already-bound inputs, or
  2. captured as an authenticated observation?

Commonly forgotten inputs include:

current time, time zone, locale, and daylight-saving database;
randomized IDs, seeds, shuffles, sampling, and jitter;
iteration order from unordered collections;
environment variables and process working directory;
filesystem metadata and network resolution;
feature-flag evaluation;
tenant policy and authorization state;
database isolation result and concurrent writer order;
library, compiler, target, CPU feature, and floating-point behavior;
remote response headers, retry hints, and rate-limit state.

If a forgotten input changes a decision, strict replay should fail rather than normalize the difference away.

Frame-to-trace binding

The causal trace from Chapter 15 is the event authority. Replay does not invent a second event history. There is exactly one frame for every causal record, in the same order.

pub struct ReplayFrame {
    pub sequence: u32,
    pub event_id: String,
    pub event_record_sha256: String,
    pub input_receipt_sha256: Vec<String>,
    pub output_receipt_sha256: Vec<String>,
    pub operation: ReplayOperationClass,
    pub dependency_ids: Vec<String>,
    pub previous_state_sha256: String,
    pub state_sha256: String,
}

The engine verifies:

trace identity equals the policy’s trace identity;
trace receipt reproduces the record collection and terminal record;
frame count equals causal-record count;
sequence, event ID, record digest, and receipt vectors match exactly;
every dependency ID exists and is available;
previous state equals the prior frame’s committed state;
derived state equals the frame’s stored state.

This prevents a trace from one run being paired with states from another. It also prevents a replayer from deleting an inconvenient event, changing receipt flow, or silently skipping a failed validation.

Deterministic state transitions

The reference state identity is:

S[n] = SHA-256(
  replay-engine-release,
  S[n-1],
  causal-record[n],
  operation-class[n],
  ordered dependency releases[n],
  input receipts[n],
  output receipts[n]
)

This is a compact teaching reducer. A production reducer normally hashes a canonical serialization of the actual typed application state too. The important properties are:

all decision inputs are explicit;
ordering is stable;
the reducer release is bound;
serialization is canonical and versioned;
overflow, invalid enum values, and unknown fields fail closed;
no I/O hides inside the reducer.

Functional core, imperative shell

Keep replayable transitions pure:

fn reduce(state: State, event: RecordedEvent, deps: &ResolvedDeps)
    -> Result<(State, Vec<Intent>), ReduceError>;

During a live run, the imperative shell may authorize and execute an Intent, then record an observation. During replay, the engine validates the recorded intent/receipt relationship but never executes the intent.

This separation makes replay practical. Trying to virtualize arbitrary I/O from application code is far more complex and easier to get wrong.

Time, randomness, and ordering

“Use a fixed seed” solves only one source of nondeterminism.

Time

Pass a typed clock observation into reducers. Record both semantic time and provenance:

value                 2026-07-29T14:05:00Z
source                scheduler_clock
observation receipt   sha256:...
precision             milliseconds

Do not call SystemTime::now() inside replayable code. Deadline decisions should consume recorded elapsed time or a simulated clock.

Randomness

Record either:

the exact random observations used by decisions; or
a seed plus a pinned generator algorithm and consumption order.

The first is more robust across implementation changes. The second is smaller but fragile: one new random draw shifts every later value.

Concurrency

Wall-clock timestamps do not establish a total order in distributed systems. Capture causal parents, receipt production/consumption, scheduler decisions, and conflict outcomes. If two events were truly concurrent, preserve that fact rather than inventing causality.

For deterministic reducers, feed a stable linearization chosen and recorded by the live scheduler. For testing alternative schedules, use a simulator or a fork whose scheduling substitution is an explicit experimental variable.

FoundationDB’s deterministic simulation documentation is a useful production example: its simulator controls time, randomness, networking, and failures so a failing seed can be rerun. That is broader than this application-level replay, but the lesson is the same—uncontrolled concurrency breaks repeatability.

Strict replay algorithm

Use a two-phase workflow.

Phase 1: preflight

Decode with deny_unknown_fields.
Validate schema and engine release.
Authenticate policy, trace, trace receipt, and artifact identities.
Check required dependency kinds and artifact availability.
Check retention and access authorization for protected content.
Verify record count, frame count, size limits, and terminal event.
Select a no-network, no-write replay runtime.

Preflight is atomic: return a typed reason before reduction begins.

Phase 2: reduction

For each causal record:

match the next frame;
resolve only declared dependencies;
verify any recorded observation receipt;
derive the next state;
compare exact state identity;
append the verified frame to the replay receipt.

The terminal receipt binds policy, causal trace, trace receipt, complete frame collection, frame count, and final state.

let receipt = strict_replay(&policy, &trace, &trace_receipt, &frames)?;
verify_strict_replay_receipt(
    &policy,
    &trace,
    &trace_receipt,
    &frames,
    &receipt,
)?;

Reverification recompiles the receipt. It does not merely check that stored fields look valid.

Forked replay

A fork is not “edit the trace and press replay.” The parent history and prefix are immutable.

The plan binds:

pub struct ForkReplayPlan {
    pub parent_replay_receipt_sha256: String,
    pub fork_after_sequence: u32,
    pub isolated_namespace_sha256: String,
    pub overrides: Vec<ReplayOverride>,
}

Every override names:

dependency ID;
exact release expected in the parent;
exact replacement release;
authorization receipt.

The old-release field prevents a stale plan from applying after the baseline changes. The authorization receipt makes experimental authority reviewable. The isolated namespace prevents fork artifacts from being mistaken for production state.

Forkability is a policy

The reference permits prompt and evaluator releases to be forkable. Run envelopes, policy, environment, reducer, and recorded observations are not.

That is intentionally conservative. A different reducer can reinterpret every event. A different policy can change what was authorized. Replacing a recorded model observation changes the factual world of the parent. Those may be valid experiments, but they need a different fork class and stronger review—not an ordinary dependency override.

Prefix and suffix rules

For a fork after frame k:

frames 1..k       exact authenticated parent prefix
state k           fork checkpoint
frames k+1..N     recomputed suffix under replacements
effects           always disabled

The engine verifies the parent receipt against the entire original frame set before accepting the prefix. It then derives each suffix state using replacements only for matching dependency IDs.

An override that is never consumed is rejected. Otherwise a report could claim “evaluator B caused no change” when the fork never used evaluator B.

The receipt records the first sequence whose state differs from the parent. Zero means the authorized replacement was consumed but produced no state difference within the suffix.

What a fork proves

Suppose the parent accepted a model response with evaluator A. The fork uses evaluator B, diverges at validation, and ends rejected.

Supported claim:

For this recorded observation, fixed prefix, fixed reducer, fixed policy, and recorded dependency
set, replacing evaluator release A with B changed state first at validation and produced a different
terminal state.

Unsupported claims:

Evaluator B is better for all traffic.
The original production outcome would definitely have been rejected.
The evaluator alone caused every downstream difference in the live world.

Generalization requires an evaluation suite, representative sampling, uncertainty analysis, and possibly online experiments. Fork replay gives a controlled case study and a debugging primitive.

Failure investigations

“Replay passes on my machine but not CI”

Likely causes:

reducer release was identified by version string rather than bytes;
canonical serialization depends on map order;
target architecture or floating-point behavior matters;
locale/time-zone data differs;
CI cannot access a protected recorded artifact;
a hidden environment lookup remains in the reducer.

Investigation:

compare dependency manifests by digest;
locate the first divergent frame;
compare resolved releases and receipt vectors;
instrument reducer inputs, not sensitive content;
convert the newly discovered input into a dependency or observation;
add the incident as a mutation test.

Do not weaken equality to make the test green.

“The trace exists but replay cannot start”

This is often a retention mismatch. Metadata outlived the protected prompt, model output, tool result, or evidence artifact.

Choose one:

retain the required encrypted artifact for the declared replay window;
shorten the promised replay window;
produce a deliberately weaker metadata-only diagnostic artifact;
cryptographically erase sensitive content and report replay as unavailable.

Never reconstruct missing content from logs.

“The fork reports no difference”

Check:

override ID matches the dependency consumed by suffix frames;
replacement bytes differ from expected bytes;
fork starts before the dependency is consumed;
suffix was actually recomputed rather than copied;
comparison includes semantic state, not only terminal status.

The reference rejects unused overrides, but a used evaluator can legitimately return the same decision.

“Replay sent a notification”

Treat this as a safety incident. The replay runtime had an effect-capable adapter or ambient credential.

Immediate actions:

disable the replay worker;
identify effects by idempotency key and receipt;
revoke replay-runtime credentials;
separate effect interfaces from recorded-observation interfaces;
add a process/network sandbox and deny-all egress;
add regression tests that every effect operation returns ExternalEffectForbidden.

An if replay { ... } check scattered through business code is not a sufficient boundary.

Security and privacy

Replay bundles are high-value. They join identities, decisions, protected content references, authorization receipts, and system releases.

Apply:

tenant-scoped authorization on every artifact dereference;
encryption with separable keys for content and metadata;
purpose-limited replay roles;
short-lived worker credentials;
append-only audit records for replay and fork creation;
isolated output namespaces;
egress denial and no production write credentials;
content-aware retention and cryptographic deletion;
rate and concurrency limits;
export controls and legal-hold handling;
integrity authentication, not bare hashes alone.

A SHA-256 identity detects mutation only when the expected digest comes from a trusted source. Sign or authenticate the root manifest and authorization receipts.

Do not record raw secrets or stable hashes of low-entropy secrets. The causal-trace capture policy still applies. Replay workers should dereference protected content only when authorized and should not copy it into ordinary diagnostics.

Performance and storage

Replay cost has four components:

capture overhead + artifact storage + preflight reads + reducer execution

Optimize without weakening semantics:

content-address and deduplicate immutable artifacts;
checkpoint typed state periodically for long runs;
preserve the complete state chain even when starting from a checkpoint;
tier cold replay bundles to cheaper encrypted storage;
batch digest verification and bounded reads;
keep indexes on run, release, terminal status, and retention deadline;
cache authenticated immutable dependencies by digest;
separately measure replay queue delay, preflight time, reduction time, bytes read, and failure reason.

Do not sample away the only failed runs. A reasonable policy retains all severe failures for a bounded window, a controlled sample of successes, and aggregate metrics beyond content retention.

Checkpoints create a trust boundary. A checkpoint must bind the prefix receipt, state schema, reducer release, and state digest. Otherwise it is merely a faster unverified starting point.

Schema evolution

Old replay bundles will outlive code releases. Plan migrations explicitly.

Safe options:

retain the old replay engine in an isolated compatibility worker;
migrate the bundle with a receipt that binds source bytes, migration release, and destination bytes;
declare the release unsupported and preserve inspectability without claiming replay.

Never deserialize an old enum into a default new variant. Unknown fields and variants must fail closed. Never recompute the parent receipt under the new schema and call it the original identity.

Test every supported historical fixture in CI. The release chapter will extend this into compatibility windows and rollback criteria.

Practical implementation walkthrough

Step 1: compile a source trace

Use the Chapter 15 compiler. The companion produces five records:

run started
model dispatched
model observed
validation completed
run terminal

The model observation is a recorded boundary value. Replay never dispatches the model.

Step 2: build the manifest

Declare the five mandatory dependency kinds plus the recorded model observation and evaluator. Mark only the evaluator forkable.

Step 3: derive strict frames

For each causal record:

copy exact sequence, event ID, record digest, and receipt vectors;
select the operation class;
list dependency IDs in stable order;
derive the next state from the previous state.

Step 4: run and verify

strict_replay returns a terminal receipt. verify_strict_replay_receipt reruns the entire derivation and requires exact equality.

Step 5: authorize a fork

The companion forks after frame three, where the model observation is fixed. It replaces evaluator release A with B. Both suffix frames consume B, so state first diverges at frame four.

Step 6: mutation-test claims

The companion changes trace identity, event identity, state, operation class, dependency availability, stored receipt, parent identity, forkability, and override consumption. Every change must fail for the intended typed reason.

Exercises

Exercise 1: clock dependency

Add a ClockObservation dependency and a deadline event. Prove that reading live time is impossible in strict replay. Mutate the clock receipt and expect failure before state transition.

Exercise 2: recorded tool result

Add authorization and tool-observation records. Replay the result through RecordedToolObservation, verify the effect receipt, and prove the tool adapter’s dispatch method is never reachable.

Exercise 3: checkpoint

For a 1,000-frame trace, create a checkpoint after frame 500. Bind the prefix-frame digest, state, schema, and reducer release. Show that replay from the checkpoint has the same terminal receipt as full reduction, or explain why your receipt design needs a separate “accelerated replay” identity.

Exercise 4: schedule fork

Model two concurrent observations with a recorded scheduler order. Design a distinct fork policy that permits a schedule substitution while preserving causal constraints. Explain why the ordinary evaluator override is insufficient.

Exercise 5: retention failure

Mark the protected model observation unavailable. The policy must fail preflight. Add a metadata-only diagnostic command whose output clearly says it is not strict replay.

Exercise 6: cross-version fixture

Serialize a version-one replay bundle. Implement a version-two migration that adds a field without defaulting unknown semantics. Produce a migration receipt and preserve the original bundle.

Solution sketches

Inject a clock port into live code and a recorded-clock adapter into replay. Bind observation identity in the frame dependencies.
Separate ToolDispatcher from RecordedToolObservationReader. Give the replay worker only the latter capability.
A checkpoint can accelerate verification, but if intermediate frames are not rechecked in that execution, its operational receipt should disclose checkpoint use even when the logical terminal state matches.
Scheduling changes the execution model. Authorize it with a schedule-specific plan, validate happens-before constraints, and isolate all effects.
Return MissingDependency; do not call a provider. Metadata-only inspection can still show event kinds and available receipts.
Hash source, migration program, and destination. Make the compatibility worker verify the migration receipt before replay.

Review checks

You understand this chapter when you can answer:

Why is a second temperature-zero provider request not strict replay?
Which values belong in the dependency manifest?
Why must model and tool observations be captured at the nondeterministic boundary?
How does a frame prove it belongs to one causal record?
Why are live effects forbidden even when idempotency keys exist?
What does the state chain detect?
How is a fork’s parent protected?
Why include both expected and replacement releases in an override?
Why reject an unused override?
What exactly does first divergence prove?
How can retention make a valid trace unreplayable?
Why does a hash need an authenticated root?
When should an old bundle be declared unsupported rather than silently migrated?

Exit checklist

When every item is true, replay becomes a reliable debugging and experimentation surface rather than a theatrical rerun.