Agent Loops, Progress, Stopping, and Bounded Repair

An agent loop is easy to sketch:

ask model → call tool → inspect result → repeat until done

That sketch hides the engineering problem. What is “done”? Who decides that progress occurred? What prevents an expensive loop from repeatedly proposing the same action? What happens when a write times out after the remote system may have applied it? Can the process restart without forgetting its budget? Does cancellation stop computation, or does it merely stop observing an effect that continues elsewhere?

A production loop must answer those questions without relying on the model’s narration. Its real shape is:

immutable run envelope + behavior specification + loop policy
                              │
                              ▼
                 reserve next permitted action
                              │
                              ▼
                  execute through typed boundary
                              │
                              ▼
               validate outcome and new evidence
                              │
                              ▼
      append state transition + usage + receipts durably
                              │
              ┌───────────────┴────────────────┐
              ▼                                ▼
       another bounded step          typed terminal receipt

The model may propose. The harness owns authority, accounting, progress, and termination.

This chapter implements that boundary in mosaic_harness::agent_loop. The provider-free companion, bounded_agent_loop_lab, is also MP-24 · Replayable Model Loop.

Learning objectives

By the end you will be able to:

• distinguish an agent loop from a deterministic workflow;
• represent loop execution as a finite, append-only state machine;
• define progress as a verified state change plus new evidence;
• bind every step to exact action, state, policy, and receipt identities;
• enforce step, model, tool, repair, cost, and deadline budgets;
• stop no-progress and repeated-action cycles deterministically;
• require an evidence-backed completion receipt rather than model self-report;
• route uncertain external effects to reconciliation instead of blind retry;
• bound repair by defect, attempt count, and revalidation;
• preserve cancellation safety and durable-resumption invariants;
• test terminal precedence, malformed traces, receipt mutation, and false completion;
• explain which guarantees are deterministic and which require external observation.

Prerequisites and dependency order

Complete the earlier Harness Engineering chapters first:

1. the behavior specification defines acceptance and completion;
2. the run envelope binds task, environment, policy, budget, and trace;
3. prompt programs and provider-neutral requests define model input;
4. context, retrieval, and memory produce bounded evidence candidates;
5. structured and semantic validators classify candidate output;
6. typed tools authorize effects and verify execution receipts;
7. tool observations enter the loop as untrusted evidence.

The loop does not replace those layers. It sequences them.

Completion artifact

Run:

cd rust/rust-ai-engineering
cargo test -p mosaic-harness agent_loop
cargo test -p mosaic-harness --example bounded_agent_loop_lab
cargo run -q -p mosaic-harness --example bounded_agent_loop_lab

Pinned reference result:

baseline terminal                  completed
mutation cases                     11 / 11
policy SHA-256                     3ad80edb851be81551ea9a20b9ba9d17e5bfcd7f7c97892622b031406d0538df
ordered steps SHA-256              ff42896e414d2e4d5929ab44ded863e7b867a4a845f91d945bd178179607ab64
loop receipt SHA-256               11a64af69ff66eec8e0af26154f52e15a06a410248c2683d612119046dde871f
report bytes                       1,896
report SHA-256                     35713c204d835ee83517dad10ee778afd7682877cfd0bd3f22489d904df57f33

Nine module tests and two example tests cover success, replay, all resource budgets, deadline, no-progress, repeated action, indeterminate effects, false completion, input exhaustion, strict usage deltas, state-chain integrity, sequence and time ordering, duplicate evidence, post-terminal steps, malformed digests, strict decoding, and stored-receipt mutation.

Model loop or workflow?

A workflow has a known graph. A loop chooses at runtime among a constrained action set.

Property	Workflow	Agent loop
Next step	declared transition	policy-filtered proposal
State space	largely anticipated	bounded but partly discovered
Completion	workflow terminal	behavior proof plus terminal receipt
Retry	per-step policy	per-effect policy plus loop budgets
Main risk	orchestration defect	orchestration defect plus model variance
Best use	repeatable business process	uncertain investigation or synthesis

If the sequence is known, use a workflow. “Have the model decide every next step” adds latency, variance, cost, and attack surface. An agent loop is justified when observations determine which of several safe next actions is useful and encoding the decision graph explicitly would be brittle.

Even then, keep deterministic work outside the model. Parsing, authorization, budget arithmetic, receipt verification, schema validation, sorting, hashing, and terminal classification belong in ordinary Rust.

The loop is a state machine

Treat a loop as a transition over immutable snapshots:

(policy, state_n, evidence_n, usage_n)
              │ action_n+1
              ▼
(state_n+1, evidence_n+1, usage_n+1, outcome_n+1)

Each reference step carries:

pub struct AgentLoopStep {
    pub sequence: u32,
    pub action_kind: LoopActionKind,
    pub action_sha256: String,
    pub state_before_sha256: String,
    pub state_after_sha256: String,
    pub new_evidence_receipt_sha256: Vec<String>,
    pub outcome: LoopStepOutcome,
    pub completed_at_unix_ms: u64,
    pub model_calls_delta: u32,
    pub tool_calls_delta: u32,
    pub repairs_delta: u32,
    pub cost_microunits_delta: u64,
    pub completion_receipt_sha256: Option<String>,
}

The digests are joins, not decoration. A step cannot silently begin from a different state. Its sequence cannot skip. Its completion time cannot move backward. Evidence identities cannot repeat within a step. Usage deltas must agree with the action kind.

The complete ordered step collection has its own digest. The receipt binds that digest to the policy identity and final accounting. Replaying the same policy and steps must produce the same receipt.

State should be canonical

Hashing arbitrary in-memory data is not sufficient. First define a stable schema and serialization:

• version the state envelope;
• reject unknown fields when forward compatibility would be unsafe;
• use deterministic ordering for sets and maps;
• normalize only where the domain permits it;
• distinguish absent, empty, unknown, and redacted;
• use content identities for external artifacts;
• keep transient handles and local timestamps outside semantic state.

The state digest is useful only if equal semantic states serialize equally and meaningful changes serialize differently.

Progress is an externally checkable predicate

The reference evaluator uses:

verified_progress(step) =
    step.outcome == succeeded
    AND state_before != state_after
    AND count(new, previously unseen evidence receipts) > 0

All three terms matter.

• Success without a state change may be an idempotent read, but it did not advance this loop.
• A changed model-authored plan without evidence may be churn.
• New evidence attached to unchanged state may be useful for diagnosis, but the state reducer has not incorporated it.
• Reusing an earlier receipt cannot manufacture progress.

This predicate is intentionally conservative. A different product may define progress as shrinking a verified unresolved-claim set, increasing satisfied acceptance checks, or committing a new artifact. Encode that predicate deterministically and version it. Do not ask the model “did you make progress?”

Progress functions

For richer tasks, define a well-founded progress measure:

P(state) = (
  unresolved_required_claims,
  failing_acceptance_checks,
  unverified_effects,
  remaining_repairs
)

Order the tuple lexicographically or by a documented partial order. A step advances only when a verified component improves without violating a more important invariant. This resembles a termination argument: the measure cannot decrease forever because every component is bounded below.

Do not use a single opaque “confidence” score. Confidence can rise while evidence quality falls.

A policy is an immutable resource contract

The reference policy binds:

• schema and evaluator release;
• policy, run-envelope, and behavior-specification identities;
• start and deadline;
• maximum steps;
• maximum model calls;
• maximum tool calls;
• maximum repairs;
• maximum cost in integer microunits;
• consecutive no-progress threshold;
• consecutive repeated-action threshold.

Use integers for exact accounting. Floating-point currency introduces comparison and serialization ambiguity. Convert provider billing units into a documented internal integer scale.

Reserve before dispatch

The compact evaluator receives completed steps and detects the first exceeded bound. A live orchestrator should additionally reserve before starting:

lock run
  verify current revision
  verify no terminal exists
  reserve one action slot
  reserve worst-case cost/tokens/tool concurrency
  append dispatch intent with idempotency key
unlock run
dispatch
append observed outcome
commit actual usage and release unused reservation

Post-hoc accounting alone can oversubscribe under concurrency. Two workers can both see one remaining call and both dispatch. A database compare-and-swap, serializable transaction, or single-owner actor should make reservation atomic.

Reserve worst-case permitted usage, then commit actual usage. If a provider does not report exact usage, retain the conservative reservation or mark accounting incomplete.

Terminal precedence is policy

One step can violate several conditions. A late tool call may cross deadline, tool-call, and cost budgets simultaneously. The reference order is:

1. deadline;
2. step budget;
3. model-call budget;
4. tool-call budget;
5. repair budget;
6. cost budget;
7. indeterminate effect;
8. finish validation;
9. repeated action;
10. no progress;
11. input exhausted when no earlier terminal exists.

This is not the only valid order. It is a versioned choice. The primary terminal should be stable for telemetry and replay; secondary violations may be recorded separately in a larger receipt.

Typed terminals make failure actionable

The terminal enum is:

pub enum AgentLoopTerminal {
    Completed,
    StepBudgetExhausted,
    ModelBudgetExhausted,
    ToolBudgetExhausted,
    RepairBudgetExhausted,
    CostBudgetExhausted,
    DeadlineExceeded,
    NoProgress,
    RepeatedAction,
    IndeterminateEffect,
    InvalidCompletion,
    InputExhausted,
}

Avoid a Boolean success. These terminals drive different product behavior:

Terminal	Safe next owner
Completed	downstream consumer
budget or deadline	product policy, user, or scheduler
NoProgress	diagnosis, more evidence, or escalation
RepeatedAction	planner defect or adversarial-input review
IndeterminateEffect	reconciliation worker
InvalidCompletion	behavior evaluator or model-output review
InputExhausted	orchestrator/data-source diagnosis

An automated resume must name which terminals are resumable and under what new policy identity. Never silently increase a bound while keeping the old policy digest.

Completion requires proof

The Finish action completes only when:

• the step outcome is successful;
• a completion-receipt identity is present;
• the same identity appears among the step’s new evidence receipts.

In a fuller implementation, replay that receipt against the exact behavior specification and artifact set. The loop receipt should join:

behavior specification
run envelope
artifact identities
validation receipts
effect receipts
usage receipt
terminal classification

The model saying “done,” producing polished prose, or setting {"complete": true} is an observation. It may request evaluation; it cannot satisfy evaluation by itself.

Repair is a bounded transition, not a second unlimited agent

A repair should receive a narrow defect report:

expected schema identity
candidate output identity
failing validation codes
allowed fields or operations
previous repair identities
remaining repair budget

Then:

1. classify whether the defect is repairable;
2. reserve one repair;
3. request or compute the smallest permitted change;
4. create a new candidate identity;
5. rerun every invalidated check;
6. record the result and stop at the repair bound.

Never repair an authorization failure into authorization. Never let a repair remove evidence requirements, increase its own budget, reinterpret an indeterminate effect as failure, or mutate the behavior specification.

Local repair versus regeneration

Local deterministic repair is appropriate for non-semantic canonicalization that the contract explicitly permits: trimming a transport wrapper, parsing a known integer representation, or sorting a set before stable serialization. Record the transformation.

Model repair is appropriate when the candidate must be regenerated from the original evidence and specific validation feedback. It consumes model and repair budgets. It must not see hidden expected answers from evaluation fixtures.

Full regeneration may be safer when a local patch would produce a structurally valid but semantically incoherent object. The trade-off is extra cost and variance. Measure both on a versioned defect corpus.

No-progress and repetition guards

Two consecutive no-progress steps stop the reference loop. Two consecutive identical action identities stop it as repeated action.

The action identity should cover semantic intent:

action kind
tool or model route
normalized parameters
relevant state revision
evidence inputs
policy release

If it includes sequence number or wall-clock time, identical actions will hash differently and the guard becomes useless. The lab deliberately overwrites the second action digest with the first to exercise this invariant.

Real systems may detect cycles longer than one:

A → B → A → B

Keep a bounded window of action/state pairs and stop repeated subsequences. Bound the window so cycle detection cannot become unbounded memory growth. A Bloom filter is compact but has false positives; an exact bounded map is often better for high-value runs.

Indeterminate effects require reconciliation

A timeout says the client stopped waiting. It does not say whether an external effect happened. The safe state is:

IndeterminateEffect {
  invocation_identity,
  idempotency_key,
  dispatch_receipt,
  last_observation,
  reconciliation_owner
}

Do not let the planner issue another write. A reconciliation worker should query the idempotency record, effect ID, resource generation, audit log, and independently observed target state. It then emits verified success, verified no-effect failure, or remains indeterminate.

This boundary prevents a classic failure: the first write succeeds, its response is lost, the loop retries, and the second write applies the effect again.

Cancellation and deadlines

Cancellation is a protocol, not a dropped future.

For local, cancel-safe computation, dropping a future may be sufficient. For a database write, remote tool, process, or provider request, dropping the local future may leave work running. Record dispatch before awaiting the effect and reconcile after cancellation if execution may have crossed the boundary.

A structured async runner should:

• give each run one owner;
• keep child tasks in a tracked set;
• propagate a cancellation token;
• stop admitting new work after cancellation;
• await or abort children according to each operation’s contract;
• release reservations exactly once;
• flush the final durable event;
• classify unresolved effects as indeterminate;
• finish within a separate shutdown budget.

Use monotonic time for in-process duration and deadlines. Persist absolute timestamps only with the clock source and uncertainty needed for cross-process recovery. After restart, recompute remaining time conservatively.

Durable resumption

Persist events before exposing derived state:

run_created
action_reserved
action_dispatched
outcome_observed
evidence_validated
state_committed
terminal_committed

Each event includes run ID, sequence, previous-event digest, policy identity, state identity, idempotency key where applicable, and producer release. On recovery:

1. load the immutable run envelope and policy;
2. verify the hash-linked event prefix;
3. reproduce state and usage;
4. locate reservations without committed outcomes;
5. reconcile possibly executed effects;
6. acquire exclusive ownership through a lease or compare-and-swap;
7. resume only if the terminal policy permits it.

An event append and a materialized-state update should be atomic or reconstructable. Do not accept a database row saying step 8 when the durable event stream ends at step 7.

Security implications

An agent loop concentrates authority and therefore deserves a direct threat model.

Prompt injection

Retrieved text, web content, tool output, OCR, transcripts, and generated media metadata remain untrusted evidence. They cannot alter policy, grants, approvals, budgets, or terminal rules.

Resource exhaustion

Bound every count and byte collection, including receipts per step, total evidence identities, trace size, tool output, context assembly, repair feedback, and cycle history. Step bounds alone do not prevent one enormous step.

Confused deputy

Bind tool invocations to tenant, principal, grant, contract, run envelope, exact input, and policy. The model chooses among permitted actions; it does not choose its authority.

Completion spoofing

Make completion receipt verification independent of model output. Ensure a model cannot write its own validator result into the trusted receipt store.

Receipt laundering

A digest proves identity, not truth. Validate the issuer, release, scope, freshness, and upstream receipt chain. Keep trusted control data separate from untrusted content.

Replay and rollback

Use idempotency keys for effects. Bind receipts to sequence and policy. Prevent an old valid completion receipt from satisfying a new behavior specification.

Performance implications

The loop’s deterministic control plane should be cheap relative to inference, but measure it separately:

• policy and step validation;
• canonical serialization;
• digest computation;
• durable event append;
• state reconstruction;
• receipt-set insertion;
• cycle detection;
• authorization and reconciliation lookup.

Use bounded collections and incremental state. Do not repeatedly serialize the entire transcript when only a content-addressed event is new. Store large artifacts outside the trace and join them by digest, byte length, media type, and provenance.

Batching model calls can increase throughput but complicates per-run deadlines and cancellation. Fair scheduling should prevent one long loop from occupying every inference slot. Reserve capacity by tenant and priority, then record queue time separately from model time.

Failure investigations

The loop spends money but never advances

Inspect action identities, state before/after, new evidence identities, validator outcomes, and usage per step. Common causes are:

• progress defined as “model returned text”;
• repeated evidence receiving new random IDs;
• state including timestamps, so every attempt looks different;
• a validator result not incorporated into state;
• no-progress counter reset by an irrelevant change.

Repair the progress predicate and add a two-step cycle fixture.

Completion appears before an effect is known

Trace the completion receipt to every required effect. If a write is indeterminate, completion must fail. Freeze the run, reconcile, invalidate the premature receipt, and add a regression where the transport times out after dispatch.

Budget is exceeded under load

Look for post-hoc rather than pre-dispatch accounting, multiple workers owning one run, reservation release races, provider usage arriving late, and integer-unit conversion differences. Make the reservation atomic and replay the concurrency schedule.

Restart repeats the last tool

Inspect whether action_dispatched was durable before the call and whether recovery recognizes an unmatched dispatch. Reconcile by idempotency key. Do not infer “not executed” from the absence of a local response.

Repair passes schema but changes meaning

Compare the original evidence, defect report, repaired candidate, and semantic receipts. Re-run all checks invalidated by the changed fields. Add a mutation where fixing one field contradicts another.

Alternatives and trade-offs

Declarative workflow engine

Prefer it when transitions are known, long-running, and operationally important. It offers strong retries, timers, and durable history. Keep model choice inside small activities.

Planner–executor split

A planner creates a bounded plan; an executor runs typed actions. This improves reviewability but plans become stale as evidence changes. Replan only from a new state revision and charge the model budget.

Search over candidate trajectories

Beam search, tree search, and generate–check–repair can improve difficult tasks. They multiply cost. Each branch needs its own state/evidence identity and budget; shared external effects should usually be forbidden until one branch is committed.

Human checkpoint

Human approval is appropriate for high-impact or ambiguous actions. Approval must bind exact parameters and expire. It does not replace effect verification or completion evaluation.

Stateless single response

Use it when one bounded generation plus validation is sufficient. It is the simplest, cheapest, and easiest system to evaluate.

MP-24 · Replayable Model Loop

The lab runs one valid four-step sequence:

model generation → tool call → verification → finish

Each step changes state and adds new evidence. The finish step introduces and points to the same completion receipt. Eleven cases then cover:

1. valid completion;
2. consecutive no progress;
3. repeated action;
4. finish without proof;
5. model-call budget;
6. tool-call budget;
7. repair budget;
8. cost budget;
9. deadline;
10. indeterminate effect;
11. input exhaustion.

The generated report lives under target/labs/mp-24/<policy-sha256>.json. It contains no provider secrets and needs no network. Complete the guided work in MP-24 · Replayable Model Loop.

Exercises

1. Recall

Classify each item as proposal, evidence, authority, progress, or completion proof:

• model text saying a rollback succeeded;
• a capability grant;
• a verified target-generation receipt;
• a changed plan with no new observation;
• a replayed behavior-evaluation receipt.

2. Predict

A policy permits two tool calls and costs 1,000 microunits. The second tool call finishes exactly at the deadline and raises cost to 1,100. Which reference terminal wins, and why?

3. Implement

Add max_total_evidence_receipts to the policy and a typed terminal. Reserve the collection bound before insertion. Add exact-bound, one-over-bound, duplicate, replay, and stored-mutation tests.

4. Repair

Extend the lab with a repair that changes state but adds no new validation receipt. Predict and confirm its terminal. Then add a valid repair that introduces a new receipt and is followed by a fresh verification step.

5. Durable recovery

Design an event sequence for a tool dispatch whose response is lost during process shutdown. State which event is the recovery boundary and how ownership is reacquired.

6. Cycle detection

Implement a bounded detector for A → B → A → B. Define the semantic action identity and prove the detector cannot grow without bound.

7. Design

Choose a real AI task. Argue whether it should use a stateless response, workflow, bounded agent loop, planner–executor split, or search. Include latency, cost, authority, evaluation, and failure recovery.

Solution guidance

1. Model text is a proposal/observation, the grant is authority, the target receipt is evidence, the changed plan is not verified progress, and the replayed behavior receipt is completion proof only if its identities match this run.
2. DeadlineExceeded wins because deadline precedes resource budgets in the versioned reference terminal order. A richer receipt may also record the cost violation.
3. Validate the bound, use checked arithmetic, define whether duplicates count, add it to policy identity, and version the evaluator release.
4. The first repair contributes no verified progress. The second needs successful outcome, state change, and a previously unseen validation receipt.
5. Persist reservation and dispatch intent before the external boundary. Recovery treats unmatched dispatch as possibly executed and reconciles it under exclusive ownership.
6. Hash semantic action plus relevant state, keep an exact fixed-size deque or map, and evict by a documented policy.
7. Prefer the least autonomous form that handles real uncertainty. High-impact writes strongly favor explicit workflows and approvals.

Check your understanding

1. Why is state change alone insufficient evidence of progress?
2. Why should action identity omit sequence and wall-clock time?
3. What is the difference between a retryable failure and an indeterminate effect?
4. Why reserve budget before dispatch?
5. Can an approval receipt prove an effect occurred?
6. When must repair rerun semantic validation?
7. What does dropping a Rust future prove about remote work?
8. Why must a resumed run retain or explicitly replace the original policy identity?
9. When is a workflow better than an agent loop?
10. What makes a completion receipt portable across replay but not across unrelated runs?

Practical completion checklist

• The loop policy and evaluator release are immutable and content-addressed.
• State and action serialization are canonical and versioned.
• Exactly one owner can reserve the next action.
• Step, model, tool, repair, cost, token, and deadline bounds are explicit where applicable.
• Progress requires an independently checkable monotonic change.
• No-progress, repeated-action, and longer-cycle behavior are bounded.
• Tool effects use idempotency and typed reconciliation.
• Completion replays the exact behavior specification and required receipts.
• Repairs receive narrow defects and rerun invalidated checks.
• Cancellation stops admission, accounts usage, joins children, and handles uncertain effects.
• Durable recovery verifies the event prefix and unmatched dispatches.
• Every terminal is typed, observable, and assigned an owner.
• Normal, boundary, mutation, restart, concurrency, and security tests pass.
• Measurements separate deterministic overhead, queueing, inference, tools, and storage.
• Known limitations and resumable-terminal policy are documented.

Authoritative references

• Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models”: an early formulation that interleaves reasoning traces and environment actions.
• Shinn et al., “Reflexion: Language Agents with Verbal Reinforcement Learning”: useful context for feedback-driven retries; production systems must still bound and verify them.
• Zhou et al., “Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models”: a search-oriented alternative whose branches need explicit compute budgets.
• Rust Async Book, “Cancellation”: the relationship between dropping futures, cancel safety, and operation semantics.
• OWASP, AI Agent Security Cheat Sheet: threat categories and controls for autonomous tool-using systems.
• NIST, AI Risk Management Framework 1.0: governance, measurement, and lifecycle framing for trustworthy AI systems.