Mosaic Architecture

Mosaic is the final system:

A Rust CLI and backend that turns mixed media into typed evidence, runs versioned AI programs, and proves how well each program works.

Signal Room and Cutroom become Mosaic programs. A new application supplies task definitions, tools, schemas, and graders without rewriting the runtime.

System map

                          ┌─────────────────────────────────┐
CLI / HTTP ─────────────▶ │ run service                     │
                          │ task · policy · budget · state   │
                          └──────────────┬──────────────────┘
                                         │
               ┌─────────────────────────┼──────────────────────────┐
               ▼                         ▼                          ▼
       context builder              tool runtime                 router
   retrieval · memory · media   sandbox · authorization   local · remote · cascade
               │                         │                          │
               └─────────────────────────┼──────────────────────────┘
                                         ▼
                                  verifier / grader
                                         │
                    ┌────────────────────┼────────────────────┐
                    ▼                    ▼                    ▼
                run store           trace store          artifact store
             SQL state machine      event stream       content-addressed media

Crate boundaries

mosaic-domain        no infrastructure dependencies
mosaic-program       task and program definitions
mosaic-harness       runtime state machine
mosaic-evidence      decoding, normalization, locators, provenance
mosaic-inference     local runtime adapters
mosaic-providers     remote model adapters
mosaic-tools         registry and effect boundary
mosaic-router        candidate models, policies, cascades
mosaic-evals         fixtures, runners, graders, comparisons
mosaic-storage       SQL and object storage adapters
mosaic-api           Axum/Tower transport
mosaic-cli           Clap transport

Keep the dependency graph acyclic and enforce it in workspace architecture.

A multimodal program

trait Program: Send + Sync {
    fn id(&self) -> ProgramId;
    fn task(&self, input: ProgramInput) -> Result<Task, ProgramError>;
    fn context_policy(&self) -> &dyn ContextPolicy;
    fn tool_set(&self) -> ToolSetId;
    fn output_contract(&self) -> OutputContract;
    fn graders(&self) -> Vec<Arc<dyn ErasedGrader>>;
}

Program versions are immutable. A change to instructions, schema, tools, retrieval, router, or verifier creates a new version.

Evidence and artifacts

Inputs and generated outputs use one provenance graph:

upload
 ├─ normalized audio
 │   └─ transcript
 ├─ sampled frame
 │   ├─ OCR spans
 │   └─ vision embedding
 └─ generated storyboard
     └─ selected video insert

Every edge records transform, implementation, parameters, and parent hashes. This supports caching, reproduction, deletion, and rights tracking.

Routing as policy

Routing considers:

• required capability and modality;
• data locality and provider approval;
• artifact size;
• measured slice performance;
• current latency and cost;
• local model residency;
• verifier feedback;
• risk.

The route is part of the trace. A task never selects a model by hidden global default.

Separate generation from publication

Text, image, audio, and video generators create candidate artifacts. Candidates pass:

1. format and safety checks;
2. semantic or multimodal graders;
3. provenance and rights checks;
4. human review where required;
5. an explicit publish action.

The harness may generate. The product authorizes publication.

Deployment shapes

Laptop

SQLite, filesystem blobs, local models, Forge-compatible CLI.

Team service

Postgres, object storage, workers, remote and local inference, Axum API, SSE events.

Accelerator pool

The harness service remains CPU-oriented; local inference servers own GPU batching and model residency.

The same domain and program definitions work across shapes.

Quality architecture

Every production program has:

• development and held-out eval suites;
• deterministic invariants;
• a model-grader calibration set if used;
• release thresholds;
• shadow or canary policy;
• rollback to a prior immutable version;
• slice dashboards;
• a failure inbox that can become curated data.

Mosaic’s most important screen is not a chat window. It is the comparison that shows which cases improved, which regressed, and why.