Clap Command Architecture and Terminal UX

A CLI is an API for humans, shell scripts, CI, and other programs. Its contract includes command names, arguments, stdout, stderr, exit codes, streaming, signals, configuration precedence, and artifact formats.

Forge already uses Clap derive for run, replay, eval, and compare. This chapter turns that vertical slice into a production command architecture.

Learning objectives

• model commands/options with Clap derive and validated domain conversion;
• keep main thin and reusable behavior in a library;
• separate human diagnostics from machine output;
• define stable exit-code categories;
• stream progress without corrupting final JSON;
• integrate cancellation/signals and bounded runtime ownership;
• test parsing and execution independently;
• design config precedence and secret handling with the next chapter;
• run every real Forge command.

Dependencies and architecture

Depends on workspace/API design, Tokio, cancellation, and Serde.

argv/env/files
   │ Clap parses syntax
   ▼
CLI DTO (`Cli`, `Command`)
   │ validate/map
   ▼
application command
   │ invoke harness/storage/evals
   ▼
structured result
   ├── stdout machine/human result
   └── stderr diagnostics/progress

Clap proves command-line syntax and basic value parsing. Domain validation still follows.

Derive a command tree

Conceptually:

#[derive(clap::Parser)]
struct Cli {
    #[command(subcommand)]
    command: Command,
}

#[derive(clap::Subcommand)]
enum Command {
    Run(RunArgs),
    Replay(ReplayArgs),
    Eval(EvalArgs),
    Compare(CompareArgs),
}

Prefer subcommands when operations have distinct required inputs/effects. Avoid dozens of booleans that create invalid combinations.

Use domain types/value parsers for bounded numbers and enums. File existence/canonicalization, cross-field budgets, and authorization belong after parse with typed application errors.

Thin binary, testable library

src/main.rs should:

1. parse CLI;
2. initialize configuration/telemetry;
3. build runtime/adapters;
4. call forge_cli::execute;
5. render result/error and exit.

Command implementation lives in src/lib.rs, so tests call functions without spawning a subprocess for every case. End-to-end tests still execute the binary to prove I/O/exit behavior.

Do not put provider clients, SQL, prompt assembly, and state-machine code inside match arms in main.

Stdout and stderr are separate protocols

For --format json:

• stdout contains exactly one documented JSON value/JSONL stream;
• stderr contains progress and diagnostics;
• logging never enters stdout;
• success JSON remains valid under verbose mode.

Forge currently prints eval reports to stdout and errors to stderr. Preserve that rule.

For streaming:

• JSONL events on stdout, one complete JSON object per line;
• final event includes terminal status;
• diagnostics on stderr;
• flushing/backpressure/cancellation are explicit;
• no ANSI color when output is not a terminal or --color never.

Human mode can use concise tables/progress, but machine mode must not depend on terminal width.

Exit codes

Define stable categories:

Code	Meaning example
0	requested command completed successfully
2	CLI syntax/usage error (Clap convention)
3	domain/task/config invalid
4	expected model output invalid/eval failed
5	provider/infrastructure unavailable
6	policy/permission denied
130	interrupted by Ctrl-C convention on Unix-like systems

Exact values are product policy. Document them and test them. Do not assign one code per internal error variant; preserve a small stable public classification while detailed diagnostics use a correlation ID/source chain.

An eval regression may be a successful command that found failure. CI usually benefits from nonzero when acceptance gates fail; report still goes to stdout.

Paths and stdin/stdout

Support conventional composition where useful:

• - means stdin/input or stdout/output only when unambiguous;
• PathBuf for paths;
• reject overwrite unless --force is explicit;
• write artifacts atomically through temp file then rename;
• never print binary media to terminal by accident;
• cap stdin bytes and processing time.

If input/output are both -, define stream framing. A command requiring random access cannot pretend arbitrary stdin is seekable.

Configuration precedence

Recommended:

safe defaults < config file < environment < CLI flags

Only explicitly provided CLI values should override lower layers. Clap defaults can erase provenance by always producing a value; use Option<T> in parse DTO then resolve centrally.

forge run --max-repairs must be cross-validated against max-model-calls after layering.

The next chapter’s mosaic-config records source per setting.

Secrets

• accept provider tokens primarily through secret manager/env/file descriptor;
• avoid CLI flags: argv may appear in process listings/shell history;
• never echo secrets in debug/config dumps;
• redact URLs/headers;
• pass a SecretString/provider adapter, not broad global config;
• do not serialize secrets into traces.

If an interactive secret prompt exists, require a TTY and disable echo.

Signals and cancellation

The binary owns OS signal handling:

tokio::select! {
    result = execute(command, cancellation.clone()) => result,
    _ = tokio::signal::ctrl_c() => {
        cancellation.cancel();
        // await bounded cleanup
    }
}

First interrupt requests graceful cancellation; a second may force faster exit under documented policy. Flush durable traces, but do not wait forever.

CLI process cancellation cannot prove remote provider work stopped.

Progress UX

Progress should answer:

• current stage;
• elapsed/remaining budget;
• model/tool activity summary;
• cancellation hint;
• artifact/trace location.

Avoid token-by-token animation by default in CI. Throttle updates. Terminal progress writes to stderr and detects TTY. Screen-reader and NO_COLOR behavior matter.

For long durable jobs, print run ID immediately and allow forge runs watch <id>/resume rather than binding all ownership to one terminal.

Shell safety

Use std::process::Command argument arrays for tools, never concatenate model/user text into a shell command. Validate executable allowlist, cwd, environment, timeout, output bytes, and capabilities.

Quoting is shell-specific and not a substitute for avoiding the shell.

Testing layers

Parser tests

Forge tests Cli::try_parse_from:

• subcommand and flags parse;
• required values fail;
• defaults/options are visible;
• aliases/conflicts are stable.

Execution tests

Call library command functions with scripted models/temp fixtures.

Binary tests

Spawn compiled binary and assert:

• exit code;
• stdout valid/exact;
• stderr redaction;
• signal handling;
• overwrite behavior;
• non-TTY/color.

Golden help

Help snapshots can catch accidental UX changes, but review intentionally; terminal wrapping/version can change.

Runnable Forge contract

cd rust/rust-ai-engineering

cargo run -q -p forge-cli --bin forge -- \
  run \
  --task fixtures/forge/task-valid.json \
  --responses fixtures/forge/responses-valid.json \
  --trace /tmp/forge-trace.jsonl

cargo run -q -p forge-cli --bin forge -- \
  replay --trace /tmp/forge-trace.jsonl

cargo run -q -p forge-cli --bin forge -- \
  eval --suite fixtures/forge/eval-suite.json

cargo test -p forge-cli

The eval must pass 4/4. run writes a real JSONL trace; replay validates it rather than calling a model.

Failure investigation: JSON output breaks in CI

Symptom: parser fails because log/progress lines precede JSON.

1. capture stdout/stderr separately;
2. find logger/progress target;
3. make stdout an explicit renderer only;
4. send tracing/progress to stderr;
5. test --format json with verbose and non-TTY;
6. validate output with JSON parser in integration test.

Common failure modes

• Clap defaults erasing layer provenance.
• main.rs containing application logic.
• logs/progress on stdout.
• secrets accepted/printed in argv/debug.
• one exit code for every failure.
• unchecked overwrite.
• shell interpolation.
• detached work after Ctrl-C.

Security and performance implications

Security:

• cap files/stdin before parse;
• canonicalize/confine tool paths;
• redact config/errors;
• treat response fixtures as untrusted;
• use explicit overwrite/side-effect flags;
• separate read-only and effectful commands.

Performance:

• CLI startup should not initialize every model/provider for metadata/help;
• lazy-load adapters after validation;
• stream large traces instead of collecting;
• terminal rendering should be throttled;
• release runtime/resources on bounded shutdown.

Exercises

1. Add --format human|json|jsonl.
2. Define/test exit-code mapping.
3. Add --max-model-calls and cross-field config validation.
4. Add atomic report output with overwrite protection.
5. Add Ctrl-C cancellation test around a scripted delayed model.

Solution guidance

Keep parser DTO optional; map to validated command after config resolution. Renderer owns stdout. Binary tests should inject scripted behavior rather than network. For overwrite, create-new or temp+atomic rename and handle races.

Check your understanding

• Does Clap parsing establish domain validity?
• Where should progress go in JSON mode?
• Why avoid secret flags?
• Who owns Tokio runtime/signals?
• Is eval regression a process crash or expected command outcome?

Completion checklist

• Thin binary delegates to a library.
• Stdout/stderr/exit codes are documented.
• Config precedence retains provenance.
• Secrets never enter argv/traces.
• Signals cancel and join owned tasks.
• Real Forge commands and parser tests pass.

Authoritative references

• Clap derive tutorial
• Clap API
• Cargo package layout
• std::process::ExitCode
• tokio::signal