Workflow-versus-Agent Design Case Studies

“Use an agent” is not a behavior specification. It is a control-flow decision.

A workflow chooses most transitions in code before the run starts. An agent delegates some transition selection to a model while the run is in progress. That delegation can be useful when the next productive action depends on evidence that cannot be known in advance. It also enlarges the set of reachable states, effects, costs, and failure paths.

The correct question is therefore not:

Could a model decide the next step?

It is:

What is the least autonomous system that can satisfy the behavior contract under the available evidence, authority, verification, time, and cost?

This chapter turns that question into executable policy. The reference implementation is mosaic_harness::orchestration_design. Its casebook covers deterministic formatting, document extraction, ticket routing, mixed-media triage, video candidate funnels, repository investigation, sandboxed code repair, schema migration, public-web research, and irreversible refunds.

Learning objectives

By the end you will be able to:

distinguish deterministic, model-enhanced, router, bounded-agent, and human-led control flow;
select an architecture from discovery scope, transition knowledge, effect risk, and verification;
explain why model capability and model authority are independent decisions;
calculate how agentic branching expands the reachable state and test surface;
bind modalities, model calls, tools, branches, time, cost, effects, approval, replay, and fallback;
recognize over-agentic, underpowered, and unsafe designs;
keep tool observations as evidence rather than instructions;
place human approval before a consequential effect rather than after it;
investigate no-progress loops, tool thrashing, false completion, and automation bias;
design text, image, audio, video, and structured-data workflows without pretending every multimodal system needs an agent;
extend a typed Rust casebook and reproduce its assessment receipt.

Completion artifact

Run:

cd rust/rust-ai-engineering
cargo test -p mosaic-harness orchestration_design
cargo test -p mosaic-harness --example workflow_agent_casebook
cargo run -q -p mosaic-harness --example workflow_agent_casebook

Pinned result:

accepted designs                        10 / 10
deliberate rejected designs               5 / 5
deterministic workflows                       2
model-enhanced workflows                      1
router workflows                              3
bounded agents                                3
human-led processes                           1
casebook SHA-256             9a7251021563e892c65d492e253287ece96752ce9ed0f82143693b6312580089
report bytes                              7,797
report SHA-256               6adc34518cf31592fb536468aebadbeeb462913d055cf4c55d07b563784e91cc

Twelve module tests and two example tests cover minimum-shape selection, multimodal coverage, effect authority, approval, sandboxing, progress, stop conditions, strict replay, case ordering, assessment reproduction, stored-report mutation, and deterministic serialization.

Five control-flow shapes

Use precise names:

Shape	Who chooses the next transition?	Iteration	Typical use
Deterministic workflow	Code	No	Parsing, formatting, exact transforms, known migrations
Model-enhanced workflow	Code; model fills one bounded semantic step	No	Extraction, summarization, classification with one route
Router workflow	A bounded rule or model selects one known branch	No	Ticket queues, specialist media pipelines, candidate stages
Bounded agent	Model selects among authorized actions using observations	Yes	Investigation, research, sandboxed repair
Human-led process	Human owns consequential decisions and effects	Optional advisory model	Payments, legal commitments, destructive or ambiguous actions

This taxonomy describes runtime control, not marketing. A workflow may contain ten model calls and still be a workflow if code predetermines their order. An agent may make only three model calls and still be an agent if the model adaptively chooses what to do next.

The ReAct paper demonstrates the useful core of an agentic loop: reasoning and environment actions are interleaved so observations can update the plan. That capability is valuable precisely when the required observation is not available before the action. It does not imply that interleaving should replace known control flow.

Tool use is also not synonymous with agency. A workflow can call a calculator, database, decoder, or renderer at a predetermined step. Toolformer studies a model learning when and how to invoke APIs, but production authority still belongs to the surrounding system. The fact that a model can propose a tool call does not grant it permission to execute one.

First principle: minimize delegated transition authority

Let:

S be the set of reachable runtime states;
A(s) be the authorized actions from state s;
O(a) be the possible observations after action a;
H be the maximum number of model-selected steps.

A crude upper bound on possible trajectories is:

product over h=1..H of |A(s_h)| × |O(a_h)|

The real system is not independent and the bound is usually loose, but its direction is correct. Adding a tool, retry, branch, or iteration multiplies paths. More paths mean more:

safety cases;
evaluator fixtures;
concurrency interleavings;
partial-effect states;
replay dependencies;
latency and cost tails;
observability requirements;
ways to stop incorrectly.

An agent is justified when that expanded state space buys necessary adaptation. It is waste when the valid transition graph was already known.

Uncertainty is not authority

A model may be uncertain about an invoice field. That justifies abstention or human review, not filesystem access.

A model may need repository evidence. That justifies read-only search, not merge permission.

A reviewer may need to compare refund evidence. That justifies an advisory summary, not payment execution.

Treat these as separate decisions:

capability: what the model can infer or propose
authority:  which effects the system may perform
autonomy:   which runtime transitions the model may select

A strong model can have low authority. A small specialist model can operate inside a high-impact workflow only if deterministic gates and human ownership contain the effect.

The design inputs

The reference case has eight material inputs.

1. Discovery scope

None means every required input is already present. A formatter does not need to discover what to read.

ClosedSet means the system may inspect a bounded, known collection: one invoice, one support ticket, or five generated candidates.

OpenWorld means the next relevant source or tool cannot be enumerated completely before the run: repository investigation, public-web research, or incident diagnosis.

Open-world discovery is evidence for an agent, but not sufficient by itself. An operator-led search can be safer when effects are consequential or the verifier is weak.

2. Transition knowledge

Deterministic means the valid next step is fully known from typed state.

KnownBranches means a finite transition table exists, though a classifier may choose the branch.

PartiallyKnown means new observations may require a previously unselected action sequence.

If the transition graph is known, encode it. A model can classify into a branch without owning the rest of the graph.

3. Effect risk

The casebook orders effects:

pure < read-only < reversible write < external write < irreversible

This ordering is about recovery, not emotional severity.

Pure computation changes no external state.
Read-only work can still leak protected data, so authorization remains necessary.
A reversible write needs isolation, idempotency, rollback, and often approval.
An external write crosses a system or organizational boundary.
An irreversible effect cannot be reliably compensated: a settled payment, published secret, or destructive physical action.

The recommended automated authority for a human-led process is at most read-only. The model may prepare evidence or a draft. The human executes the consequential action through a distinct identity and interface.

4. Verification

Exact verification compares deterministic properties: parse success, schema, digest, test result, or arithmetic invariant.

EvidenceBacked verification checks that claims cite real, relevant source regions and satisfy domain rules. It is stronger than stylistic judgment but may include calibrated uncertainty.

HumanJudgment is required when the acceptance function depends on values, ambiguity, novelty, or context that has not been operationalized reliably.

Weak verification should reduce autonomy, not motivate a longer self-critique loop. A model judging its own unobservable success is not independent evidence.

5. Modalities

Text, images, audio, video, and structured data change tools and evaluators, but not the fundamental architecture test.

A fixed image decode → OCR → schema validation pipeline is a workflow. A video funnel that sends five candidates through known low-cost and high-cost stages is a router workflow. A multimodal incident investigator that adaptively chooses which time range, log, frame, or transcript segment to inspect can be a bounded agent.

The reference requires exact declared modality coverage. Dropping Video from a continuity design is underpowered even if the text portion succeeds.

6. Budgets

Every model-using shape binds:

model steps;
tool calls;
parallel branches;
runtime deadline;
cost units.

An agent additionally needs no-progress and repeated-action stops. “Use reasonable effort” is not a budget. A provider timeout is not a total runtime deadline. A token cap is not a tool-effect cap.

7. Approval availability

Approval is a state transition:

proposed effect
    -> frozen payload and evidence
    -> authenticated reviewer decision
    -> current-policy revalidation
    -> execution

It is not a chat message saying “looks good.” Bind the exact payload, principal, policy release, expiry, and execution identity. If the payload changes after approval, request approval again.

8. Audit requirement

Strict replay is required when the case is audited, uses tools, or allows writes. A summary trace is enough only for simple no-effect flows where full causal reproduction is not a requirement.

Replay does not make an unsafe design safe. It makes its behavior inspectable and reproducible.

The recommendation function

The reference decision order is intentionally conservative:

if effect is irreversible:
    human-led
else if external write has no human approval:
    human-led
else if discovery is open-world
     or transitions are partially known
     or adaptive tools are required:
    bounded agent only for read-only work,
    or reversible writes with approval;
    otherwise human-led
else if transitions have known branches:
    router workflow
else if discovery is closed-set
     or verification is not exact:
    model-enhanced workflow
else:
    deterministic workflow

This is a default, not a universal theorem. Domain regulation may require a human earlier. A formally verified controller may automate a consequential but tightly bounded effect. The important property is that exceptions appear as explicit policy, evidence, and tests—not as an informal assumption hidden in a prompt.

Why the function does not begin with model quality

Suppose a frontier model and a small model both receive an invoice. The input set, output schema, effect, and verifier are unchanged. Both belong in the same model-enhanced workflow. Their measured quality changes which release qualifies, not who controls the transition graph.

Now suppose the task becomes “investigate why invoices from an unknown supplier family fail.” The discovery and transition structure changed. An agent may now be justified even with the same model.

Architecture follows the task-environment contract. Model selection follows measured capability inside that architecture.

Encoding the case in Rust

The case separates the problem from the proposed solution:

pub struct OrchestrationCase {
    pub case_id: String,
    pub objective: String,
    pub modalities: Vec<Modality>,
    pub discovery_scope: DiscoveryScope,
    pub transition_knowledge: TransitionKnowledge,
    pub effect_risk: EffectRisk,
    pub verification: VerificationKind,
    pub maximum_known_branches: u32,
    pub requires_adaptive_tool_use: bool,
    pub human_approval_available: bool,
    pub audit_required: bool,
    pub deadline_ms: u64,
    pub cost_budget_microunits: u64,
}

The candidate controls bind the exact case identity:

pub struct OrchestrationControls {
    pub case_sha256: String,
    pub shape: OrchestrationShape,
    pub accepted_modalities: Vec<Modality>,
    pub model_step_budget: u32,
    pub tool_call_budget: u32,
    pub parallel_branch_budget: u32,
    pub runtime_budget_ms: u64,
    pub cost_budget_microunits: u64,
    pub allowed_effect: EffectRisk,
    pub iterative: bool,
    pub sandboxed: bool,
    pub approval_gate: bool,
    pub progress_evidence_required: bool,
    pub stop_conditions: Vec<StopCondition>,
    pub verification: VerificationKind,
    pub replay: ReplayRequirement,
    pub fallback: FallbackAction,
}

The case digest prevents controls written for one problem from being reused after its modalities, risk, or deadline change. Closed enums, strict deserialization, unique ordering, and upper bounds make malformed policy fail before execution.

Assessment is more useful than a boolean

The compiler returns four dispositions.

Accepted means the candidate equals the minimum recommended shape and contains required controls.

OverAgentic means the design delegates more runtime decisions than the case requires. Example: turning exact transcript formatting into an eight-step tool-using loop.

Underpowered means the design cannot cover the declared case. Example: omitting video from a mixed-media continuity pipeline or using one deterministic branch where adaptive discovery is required.

Unsafe means authority or evidence controls are missing. Examples include automating an irreversible refund, allowing broader effects than the case, removing approval, disabling sandboxing, or downgrading strict replay.

Typed violations make the decision actionable:

more_autonomy_than_needed
human_control_required
effect_authority_too_broad
approval_gate_missing
sandbox_missing
progress_evidence_missing
no_progress_stop_missing
strict_replay_missing
modality_coverage_mismatch

The assessment itself is content-addressed and reproducible. A stored assessment is not trusted because it says “accepted”; the verifier recomputes it from the case and controls.

Case study 1: deterministic transcript formatting

Contract: convert a validated transcript into a fixed document schema.

Discovery: none.
Transitions: deterministic.
Effect: pure.
Verification: exact.
Shape: deterministic workflow.

Code should normalize line endings, validate timestamps, map speakers, serialize the schema, and compare exact invariants. A model adds variance without adding missing information.

An over-agentic version asks a model to choose whether to inspect, rewrite, critique, and retry. It creates new failure modes: omitted segments, paraphrases, repeated work, and cost variance. The casebook rejects that shape even if the model usually produces attractive formatting.

Case study 2: invoice extraction

Contract: extract a fixed schema from one bounded document and attach source regions.

Discovery: closed set.
Transitions: deterministic after one semantic inference.
Effect: pure.
Verification: evidence-backed.
Shape: model-enhanced workflow.

The program decodes the document, constructs the bounded request, calls one model, parses strict output, verifies every cited region, and either accepts or abstains. Repair can be another code-owned transition with a separate budget.

Do not give the model a general filesystem tool. It already has the complete invoice. If confidence is low, route to review; do not let it search unrelated documents until it feels confident.

Case study 3: support routing

Contract: choose one queue from a versioned twelve-label taxonomy and abstain when evidence is insufficient.

Discovery: closed set.
Transitions: known branches.
Effect: pure.
Verification: evidence-backed.
Shape: router workflow.

The model may produce a label and evidence. Code validates membership, confidence policy, and abstention. The branch then invokes a known workflow.

The router must not invent a thirteenth queue or decide what every queue does. Classification uncertainty belongs at the branch boundary.

Case study 4: mixed-media continuity triage

Contract: route a defect involving image, audio, or video to one of four specialist pipelines.

Discovery: closed set.
Transitions: known branches.
Effect: pure.
Verification: evidence-backed.
Shape: router workflow.

A model can inspect synchronized thumbnails, transcript segments, audio features, and metadata. It returns a typed defect class such as visual identity, timing, speech, or container integrity. Deterministic code routes the artifact.

This is multimodal but not agentic. The candidate set and next stages are known.

Case study 5: video candidate funnel

Contract: move candidates through fixed storyboard, cheap preview, continuity, policy, and final-render gates.

Discovery: closed set.
Transitions: known branches.
Effect: pure artifact creation.
Verification: evidence-backed.
Shape: router workflow.

Generation may be stochastic and expensive, yet control flow remains explicit:

storyboards
    -> cheap previews
    -> continuity and policy gates
    -> bounded finalist set
    -> expensive render
    -> final validation

A router chooses reject, repair, advance, or human review from a closed set. An autonomous agent deciding to keep generating until satisfied obscures the cost distribution and stopping rule.

Case study 6: repository incident investigation

Contract: follow read-only runtime and source evidence until one falsifiable hypothesis remains.

Discovery: open world inside an authorized repository and telemetry scope.
Transitions: partially known.
Effect: read-only.
Verification: evidence-backed.
Shape: bounded agent.

This is a legitimate agent case. A stack trace may point to a call site, which points to configuration, which requires a log query, which falsifies the initial hypothesis. The useful next action depends on the observation.

The agent still receives:

an allowlisted root and read-only telemetry queries;
eight model steps and sixteen tool calls in the reference policy;
a deadline and cost limit;
unique-action and evidence-progress tracking;
no-progress and repeated-action stops;
strict trace and replay;
human review for the conclusion.

The original SWE-bench paper illustrates why repository tasks are structurally difficult: the system must locate relevant code in a large repository and produce changes whose correctness interacts with multiple components. Benchmark success is not a permission model. Production code repair additionally needs isolation, effect control, and release gates.

Case study 7: sandboxed code repair

Contract: investigate, propose a patch, and run checks without merging.

Discovery: open world inside the repository.
Transitions: partially known.
Effect: reversible write.
Verification: tests plus evidence review.
Shape: bounded agent with approval.

The difference from incident investigation is effect. The agent receives a disposable worktree or microVM, not the developer’s live tree. Patch creation is reversible inside that boundary.

Completion requires:

requested behavior satisfied
AND focused regression passes
AND required broader checks pass
AND diff stays within scope
AND no protected file or secret crossed the boundary

Merge remains a separate human or release-system action. “Tests passed” does not mean “authorized to publish.”

Case study 8: schema migration

Contract: apply a precompiled migration after explicit approval.

Discovery: none.
Transitions: deterministic.
Effect: external write.
Verification: exact.
Shape: deterministic workflow with approval.

This case is consequential but not agentic. The migration order, preconditions, transaction semantics, checks, and rollback procedure are known. A model may help draft the migration during development, but runtime execution should be ordinary software.

The approval binds the migration digest and target. The runtime rechecks the schema version and fence before applying it. A deterministic high-impact workflow can be safer than a low-impact agent because its reachable states are smaller and testable.

Case study 9: public-web due diligence

Contract: adaptively gather public evidence and produce a cited, uncertainty-labeled memo.

Discovery: open world.
Transitions: partially known.
Effect: read-only.
Verification: human judgment supported by citation checks.
Shape: bounded agent.

The agent may choose new searches from contradictions or missing evidence. It cannot contact subjects, publish accusations, buy data, bypass access controls, or treat retrieved instructions as authority.

The memo separates:

directly supported claims;
inferred claims;
conflicting evidence;
unknowns;
source dates and retrieval identities.

Human review owns the final interpretation because citation existence does not settle relevance, fairness, or materiality.

Case study 10: irreversible refund

Contract: review evidence and prepare a refund decision without autonomously moving money.

Discovery: closed set.
Transitions: known branches.
Effect: irreversible.
Verification: human judgment.
Shape: human-led process.

The model can summarize the case, calculate policy facts with deterministic tools, and prepare a proposed action. Its automated effect ceiling is read-only.

The reviewer sees the original evidence, not only the model’s narrative. The payment system receives an authenticated human decision tied to the exact amount, recipient, reason, and expiry.

Moving the approval after the API call is not human-in-the-loop. It is incident review.

NIST’s AI RMF Core calls for defined roles and responsibilities in human-AI configurations and for the supported task and method to be specified. Its Generative AI Profile further connects risk level to review, tracking, documentation, and management oversight. The practical lesson is not “always add a human”; it is to assign ownership at the exact consequential transition.

Failure investigation 1: agentized extraction costs more and gets worse

Symptom: invoice latency and cost increase while field accuracy falls.

Trace: the agent repeatedly calls OCR on slightly different regions, then replaces supported values with plausible alternatives.

Root cause: the complete evidence set and transition graph were known. Iteration was introduced without a measurable progress function.

Repair: return to one bounded extraction step, region verification, typed abstention, and human-review routing.

Regression: submit an eight-step agent control for the extraction case. Expect more_autonomy_than_needed.

Failure investigation 2: an investigator loops without progress

Symptom: every step is syntactically valid, yet the run consumes its entire budget.

Trace: search timeout, read the same file, search timeout, read the same file.

Root cause: tool success was counted as task progress. No new evidence or state change was required.

Repair: progress means a new validated evidence identity, a falsified hypothesis, a narrowed candidate set, or a verifier state transition. Stop repeated action and no progress separately.

Regression: remove progress evidence and the no-progress stop. The casebook marks the design unsafe.

Failure investigation 3: router becomes a hidden agent

Symptom: a support router invents follow-up questions, calls customer tools, and moves tickets between queues repeatedly.

Root cause: “route” exposed general tools and a loop. The model owned downstream workflow transitions rather than one classification boundary.

Repair: one schema-constrained route decision, closed labels, abstention, and code-owned branch dispatch.

Regression: assert model_step_budget == 1, iterative == false, and branch budget equals the taxonomy size.

Failure investigation 4: approval is stale

Symptom: a reviewer approves a $25 refund, but the executed payload is $250.

Root cause: approval bound a conversation ID rather than exact effect arguments.

Repair: freeze and hash the invocation; bind principal, policy, expiry, and target; reauthorize immediately before effect execution; reject any argument drift.

Regression: mutate the approved payload and require an authorization failure before dispatch.

Failure investigation 5: multimodal success ignores one modality

Symptom: continuity evaluation passes on transcript and keyframes while the final video’s audio is shifted by two seconds.

Root cause: the controls silently omitted audio although the case declared it.

Repair: require exact modality coverage and modality-specific validators. The aggregate decision cannot pass when a required modality has no receipt.

Regression: remove one modality from accepted_modalities; expect modality_coverage_mismatch.

Human oversight without automation bias

A human gate can fail when:

the reviewer sees only the model summary;
hundreds of approvals make review ceremonial;
the UI emphasizes “recommended” and hides uncertainty;
the reviewer cannot inspect source evidence;
rejection requires more work than approval;
accountability is assigned to a person without time or authority.

Design the interface so the reviewer can:

see the exact proposed effect;
inspect original evidence and provenance;
see counterevidence, uncertainty, and policy checks;
edit or reject without executing;
understand expiry and reversibility;
know which identity will execute;
see whether the model or policy changed since evidence collection.

Measure disagreement, override, review duration, escalation, and post-effect correction. A 99.9% approval rate may mean excellent proposals, or it may mean the gate is decorative.

Security boundary

The architecture selector does not replace authorization. Every shape still needs:

tenant and principal identity;
data classification and provider boundary;
capability-scoped tools;
argument validation;
idempotency and effect receipts;
observation sanitization;
secret-safe trace capture;
release identity and replay;
cancellation, reconciliation, and rollback.

Agents make these controls more urgent because untrusted observations influence future action selection. Keep the instruction hierarchy outside retrieved content. A web page, image caption, audio transcript, compiler error, or tool response can supply evidence; it cannot grant a new capability.

Testing strategy

Test the selector as policy and the runtime as a state machine.

Selector properties

deterministic exact cases never receive a model budget;
open-world read-only cases receive bounded-agent controls;
irreversible effects always recommend human ownership;
model authority never exceeds case effect;
required modalities match exactly;
model, tool, branch, time, and cost bounds are finite;
approval appears for reversible or greater effects;
audited/tool/write cases require strict replay;
agent cases require progress, no-progress, and repeated-action rules.

Runtime mutations

duplicate the same action;
return a successful tool receipt with no new evidence;
exceed one budget while remaining under the others;
mutate an observation after receipt creation;
attempt an undeclared modality;
propose a broader effect than the grant;
change an approved argument;
replay with a missing dependency;
declare completion while the verifier rejects it;
route to a nonexistent branch.

Release comparison

Changing the recommendation compiler or any default control changes behavior. Version it as a harness component. Run held-out cases, strict historical replay, security mutations, performance comparison, and staged rollout exactly as Chapter 17 requires.

Exercises

Exercise 1: architecture card

For a product you know, write:

objective and completion condition;
discovery scope;
transition knowledge;
modalities;
effect risk;
verifier;
human owner;
budgets;
proposed shape.

Then write the smallest change to the case that would justify moving one level toward or away from an agent.

Exercise 2: remove an unnecessary agent

Find a loop whose possible next actions are all known. Replace it with a typed enum and explicit state transition. Compare:

reachable paths;
test cases;
latency distribution;
cost distribution;
failure classification.

Exercise 3: agent progress predicate

For repository investigation, implement:

fn is_progress(previous: &State, next: &State) -> bool

Count only new evidence, falsified hypotheses, reduced candidate sets, or verifier improvement. Prove that repeated tool success alone does not count.

Exercise 4: multimodal router

Add a case with text, image, audio, and video. Define one known branch per defect family. Mutate the controls to omit each modality in turn and assert rejection.

Exercise 5: approval binding

Create an effect proposal with amount, target, and reason. Hash it into an approval. Mutate one field and prove dispatch is impossible.

Exercise 6: autonomy budget

For a bounded agent with eight actions, four coarse observation outcomes, and six steps, calculate the loose trajectory bound. Then reduce the action set to three capabilities and compare the bound. Explain why empirical eval cases must still target likely and severe paths rather than enumerating the whole product.

Project milestone: finish the Harness phase

Extend the casebook with one domain-specific workflow and one justified agent.

Acceptance:

both cases have stable identities and sorted modalities;
the workflow contains no unnecessary iteration;
the agent has scoped tools, finite model/tool/time/cost budgets, progress evidence, deterministic stops, strict replay, and safe fallback;
any write has isolation and exact approval;
one over-agentic, one underpowered, and one unsafe mutation are rejected;
assessments reproduce from stored inputs;
the report is content-addressed;
the full workspace passes format, Clippy, tests, examples, dependency policy, and site build.

The Harness section is complete when Forge can demonstrate typed tasks, validation, budgets, tools, traces, replay, evaluation, release identity, and this architecture decision—not when it can conduct the longest possible conversation.