Provider-Neutral Capability Traits
A provider-neutral interface is not a function named call_model whose request and response are
untyped JSON.
That design merely moves provider coupling into string keys, undocumented defaults, error parsing, and conditionals. Nor is neutrality a giant trait with optional methods for text, embeddings, images, audio, video, tools, fine-tuning, and files. If most implementations return “unsupported” for most methods, the type system is no longer describing capability.
Neutrality means the product depends on a stable operation contract. Each adapter translates that contract to one concrete backend, preserves evidence about what ran, rejects unsupported options before I/O, and maps failures without hiding information needed for policy. A type implements only the capabilities it really has.
This chapter completes Part 3 with one executable vertical slice: the same bounded
TextGeneration request is dispatched through scripted, embedded-local, and remote adapters. All
three return one normalized result type with exact adapter/model release identity. They do not
produce identical words—and interchangeability never promises that they will.
Learning objectives
Section titled “Learning objectives”By the end you will be able to:
- distinguish provider neutrality from a lowest-common-denominator API;
- define traits around operations/capabilities rather than vendors;
- keep text generation, embedding, reranking, classification, image generation, speech, and video as separate contracts;
- decide when to use generic static dispatch and
dyn Traitruntime dispatch; - understand why this workspace uses
async_traitfor a dyn-dispatched async boundary; - validate global request bounds and adapter-specific advertised limits before backend work;
- represent seeded generation and JSON-object support as explicit capabilities;
- keep generated output and refusal as different enum variants;
- preserve stop reason, optional usage, backend request ID, topology, adapter ID, and immutable model release;
- normalize retry-relevant failures without putting retry policy in the provider adapter;
- build an exact scripted adapter whose mismatch does not consume the fixture;
- isolate local engine and remote transport SDK details behind ports;
- state what interchangeability guarantees and what it does not;
- design a reusable adapter conformance suite;
- add future multimodal capabilities without changing existing text consumers;
- threat-model provider responses, error messages, endpoint selection, and capability drift.
Prerequisites and dependency order
Section titled “Prerequisites and dependency order”This chapter composes the previous Part 3 contracts:
- Task-output contracts explains why generation, embeddings, reranking, and classification are not interchangeable.
- Model release identity defines requested versus resolved releases and promotion gates.
- Execution topology separates embedded, service, and provider boundaries.
- Quantization and device placement makes local runtime support/capacity a declared fact.
The adapter layer does not replace any of these. It carries their results into one operation boundary.
Completion artifact
Section titled “Completion artifact”Run:
cd rust/rust-ai-engineeringcargo run -q -p mosaic-harness --example capability_adaptersThe example creates one request:
prompt "Return a JSON object with one answer field."maximum output 32 tokensseed 42output mode JSON objectIt stores three different concrete adapters in Vec<Box<dyn TextGeneration>>:
| Adapter | Backend kind | Release evidence |
|---|---|---|
| exact fixture | scripted | script@sha256:1111 |
| fixture local engine | embedded | model@sha256:2222 |
| fixture HTTP transport | remote_api | provider/snapshot-2026-07-28 |
Each returns generated text, a normalized finish reason, optional usage, and its backend request ID. The local and remote engines are deliberately dependency-free fixtures: this chapter tests the adapter seam without credentials, network variance, model downloads, or native libraries. Later runtime chapters implement those ports against Candle, mistral.rs, ONNX Runtime, and selected provider APIs.
The example proves:
same operation request + same result envelope + three implementationsIt does not prove:
same text, quality, token count, latency, cost, safety behavior, or determinismThose are evaluated properties of adapter/model/harness releases.
Start from the operation, not the vendor
Section titled “Start from the operation, not the vendor”Ask what the application needs:
generate textembed textrerank candidatesclassify an inputgenerate an imagetranscribe audiosynthesize speechunderstand videogenerate videoEach operation has different input units, outputs, limits, streaming behavior, quality metrics, cost drivers, failure cases, and safety controls. Put them in separate traits:
trait TextGeneration { /* ... */ }trait TextEmbedding { /* ... */ }trait TextReranking { /* ... */ }trait ImageGeneration { /* ... */ }trait AudioTranscription { /* ... */ }trait SpeechGeneration { /* ... */ }An adapter can implement several traits when the evaluated release supports them. Consumers ask for only the trait they need. This gives two useful failure modes:
- a missing implementation fails during composition/compilation;
- a supported capability can still reject a request that exceeds its advertised limits.
A mega-trait tends to defer both failures until production.
Why not use one generic Value -> Value function?
Section titled “Why not use one generic Value -> Value function?”It cannot express:
- whether input is prompt text, token IDs, pixels, samples, or candidate pairs;
- whether output is text, vectors, ranked IDs, labels, bytes, or a media job handle;
- dimensions, sample rate, geometry, timestamps, or MIME/container constraints;
- whether a seed, schema, streaming, tools, or batch input is supported;
- the difference between refusal, truncation, transport error, invalid response, and task failure;
- which units usage/cost fields count;
- when an adapter change is semantically breaking.
Serde still belongs at network/config boundaries. serde_json::Value is not an application domain
model.
Define one honest text-generation contract
Section titled “Define one honest text-generation contract”The runnable trait is:
#[async_trait]pub trait TextGeneration: Send + Sync { fn identity(&self) -> &AdapterIdentity; fn limits(&self) -> TextGenerationLimits;
async fn generate( &self, request: TextGenerationRequest, ) -> Result<TextGenerationResult, CapabilityError>;}It contains one operation. It does not expose vendor SDK request types or transport errors.
Why asynchronous?
Section titled “Why asynchronous?”Remote calls wait on network I/O. Local calls may wait on a device scheduler, bounded blocking pool, or inference service. Returning an asynchronous operation gives the harness one composition shape. It does not make native blocking inference nonblocking: the local driver must still use the scheduling rules from the production chapters.
Rust’s async keyword documentation explains that an async function returns a future whose work progresses when polled/awaited. Cancellation is therefore not automatic backend cleanup. The caller must define what dropping the future means for an HTTP request, queued accelerator work, remote billing, or a native call that cannot be interrupted.
Why async_trait here?
Section titled “Why async_trait here?”Modern Rust supports async fn in traits for static dispatch, but an async method’s hidden future
prevents the direct trait from being dyn-compatible under the current
Rust Reference dyn-compatibility rules.
This example intentionally needs Box<dyn TextGeneration> for runtime adapter selection.
async_trait rewrites the method to return a boxed future so the trait can be used as a trait
object.
That allocation/indirection is a boundary cost, not a token-loop cost. If the adapter is selected at
compile time, use generic T: TextGeneration and benchmark whether monomorphization matters.
The Rust Reference on trait objects
describes the data pointer/vtable representation and runtime dispatch.
Request validity has two layers
Section titled “Request validity has two layers”The request constructor enforces hard library safety bounds:
pub struct TextGenerationRequest { prompt: String, max_output_tokens: u32, seed: Option<u64>, output_mode: TextOutputMode,}It rejects:
- empty/whitespace-only prompts;
- prompts above the library’s 16 MiB defensive bound;
- zero output tokens;
- output requests above the library’s one-million-token defensive bound.
These are not model context claims. They prevent obviously invalid/unbounded values from crossing the public capability API.
Each adapter then advertises narrower limits:
pub struct TextGenerationLimits { max_input_bytes: usize, max_output_tokens: u32, max_output_bytes: usize, supports_seed: bool, supports_json_object: bool,}validate_request runs before a scripted exchange is consumed, local engine is invoked, or remote
transport is called. Limit errors and unsupported options are therefore deterministic local
failures.
Bytes are not tokens
Section titled “Bytes are not tokens”Input bytes are an early allocation/transport bound. They are not a context-window calculation. The actual adapter must render roles/templates/tools, tokenize with the pinned processor, reserve output, and enforce its token/media context contract. The previous context chapter’s typed plan belongs between this preflight and backend execution.
Output bytes are also separate from output tokens. A provider-reported token count can be within budget while serialized Unicode/JSON output exceeds a product or storage bound. Check both.
Capabilities are release data
Section titled “Capabilities are release data”Do not ask a remote model “do you support seeds?” for every request or hard-code capabilities by brand. Build an evaluated capability manifest keyed by:
adapter implementation releaseresolved model releaseruntime/provider API releaseregion/topology where relevantWhen the capability set or a limit changes, rerun contract/evaluation gates. A mutable model alias can drift while your Rust binary is unchanged.
Identity travels with every successful result
Section titled “Identity travels with every successful result”The adapter identity contains:
pub struct AdapterIdentity { adapter_id: String, model_release: String, backend_kind: BackendKind,}adapter_id identifies implementation/configuration, not just vendor. model_release should be the
strongest resolved identity available: verified local artifact digest, immutable repository
revision, or provider snapshot/version. backend_kind records whether the result came from a
script, embedded runtime, inference service, or remote API.
All values are nonempty, trimmed, and bounded. The adapter constructors reject topology mismatch:
ScriptedTextAdapterrequiresScripted;LocalTextAdapterrequiresEmbedded;RemoteTextAdapterrequiresRemoteApi.
That seems strict, but it prevents traces saying “embedded” while data actually crossed an external provider boundary. A private inference-service adapter should be its own type with service authentication, transport, deadline, and topology semantics.
Backend request IDs are evidence, not identity
Section titled “Backend request IDs are evidence, not identity”A remote request ID helps provider support and cross-system tracing. It is not model release identity and should not determine replay. It is optional because local runtimes may not create one. Bound and sanitize it before logging; never place credentials or user content in it.
Generated output and refusal are different states
Section titled “Generated output and refusal are different states”The normalized outcome is an enum:
pub enum TextGenerationOutcome { Generated { text: String, finish: GenerationFinish, }, Refused { reason_code: String, },}Generated output has Stop or OutputLimit. Refusal has a bounded reason code, not a fake empty
answer. This avoids ambiguous combinations such as:
text = ""finish = stoperror = nullA content-policy block, safety refusal, unsupported request, and ordinary “no answer found” can have different product behavior. Extend the enum when the application has a measured need; do not collapse every non-answer into transport failure.
Provider APIs expose their own terminal structures. For example, Anthropic’s stop-reason guidance distinguishes a successful message’s stop reason from request failure. OpenAI’s current model catalog shows that endpoint/features vary by model. An adapter must translate documented fields and reject unknown critical variants until implemented—it should not assume every model under one provider has identical features.
JSON mode is not schema verification
Section titled “JSON mode is not schema verification”TextOutputMode::JsonObject means the adapter can request a provider/runtime JSON-object mode. It
does not prove:
- valid JSON was actually returned;
- the top-level value is an object;
- required fields/types/constraints hold;
- values are grounded or safe;
- unknown fields are absent.
The Forge JsonContractVerifier and later structured-output chapter still parse, validate, and
repair. Provider-native constraints reduce error rate; the harness owns acceptance.
Usage is optional rather than fabricated
Section titled “Usage is optional rather than fabricated”BackendTextOutput carries:
pub usage: Option<TextUsage>Some drivers know input/output token units. A local test engine may not. None is honest. Zero or
estimated numbers masquerading as billing tokens are not.
When usage is present for generated text, the adapter rejects zero output units and a reported output count above the request maximum. This is a protocol/integration invariant, not a complete provider billing audit. Different providers may count cached input, reasoning, images, audio, or tool items separately. Preserve those in a provider evidence extension rather than force them into two integers.
Failure taxonomy preserves policy inputs
Section titled “Failure taxonomy preserves policy inputs”The driver returns:
pub struct BackendFailure { pub kind: BackendFailureKind, pub message: String, pub retry_after_ms: Option<u64>,}Kinds are:
timeoutrate_limitedunavailablerejectedprotocolinternalThe taxonomy is intentionally not retryable: bool. Retry depends on:
- operation idempotency;
- whether the backend may have accepted/billed work;
- request deadline and remaining budget;
- attempt count and total cost;
Retry-Afteror service backoff;- traffic amplification/outage state;
- whether a different adapter/model preserves required capability;
- tenant and risk policy.
The adapter classifies facts; a bounded retry/router policy decides action.
HTTP mapping is not status-code-only
Section titled “HTTP mapping is not status-code-only”A remote adapter should distinguish:
- local connect/DNS/TLS/timeout failure;
- HTTP rate limit versus temporary service unavailable;
- authenticated rejection versus malformed request;
- success status with malformed/unknown JSON;
- streaming disconnect before/after a terminal event;
- provider refusal inside a successful protocol response.
Honor standardized semantics such as
HTTP Retry-After, while preserving
provider-specific documented codes in sanitized evidence. Never retry an arbitrary 4xx.
Sanitize errors at the boundary
Section titled “Sanitize errors at the boundary”SDK errors can contain endpoints, headers, prompt excerpts, response bodies, local paths, or credentials. Convert them into an internal typed failure and a safe message before tracing. Store restricted raw diagnostics only under explicit retention/access policy.
Scripted adapter: deterministic contract evidence
Section titled “Scripted adapter: deterministic contract evidence”A scripted exchange stores an exact expected request and either output or typed failure:
pub struct ScriptedExchange { pub expected_request: TextGenerationRequest, pub output: Result<BackendTextOutput, BackendFailure>,}The implementation:
- validates current limits;
- locks a FIFO queue;
- inspects the front exchange;
- returns
ScriptMismatchwithout consuming it if the request differs; - consumes only an exact match;
- normalizes the result through the same output checks used by real adapters.
This is stronger than a mock that returns the next string regardless of request. A prompt, seed, limit, or mode change breaks the fixture visibly. It is ideal for harness state-machine tests, goldens, failure injection, trace/replay, and CI without provider variance.
The mutex is asynchronous because calls can contend. Script ordering intentionally serializes the fixture; it is not a throughput model.
Local adapter: isolate runtime mechanics
Section titled “Local adapter: isolate runtime mechanics”The local adapter wraps:
#[async_trait]pub trait LocalTextEngine: Send + Sync { async fn generate( &self, request: &TextGenerationRequest, ) -> Result<BackendTextOutput, BackendFailure>;}A production implementation owns:
- tokenizer/template/model-bundle compatibility;
- device and compute/storage format;
- input/context/output planning;
- bounded scheduler/permits;
- blocking/native boundary;
- cancellation semantics;
- sampling configuration;
- cache lifecycle;
- conversion from runtime tensors/tokens into normalized output;
- local telemetry without content leakage.
Frameworks such as Candle provide tensor/model execution, not this whole product contract. The adapter makes the runtime replaceable while keeping its release/device facts visible.
Do not hold a global async mutex around a long native inference call. Use an owned scheduler/queue and permits; place truly blocking work off async executor threads.
Remote adapter: isolate transport and SDK mechanics
Section titled “Remote adapter: isolate transport and SDK mechanics”The remote adapter wraps:
#[async_trait]pub trait RemoteTextTransport: Send + Sync { async fn send( &self, request: &TextGenerationRequest, ) -> Result<BackendTextOutput, BackendFailure>;}A real implementation owns:
- URL/region/provider selection from trusted config;
- authentication without leaking secrets;
- provider request shape and role/template translation;
- connect/request/idle/overall deadlines;
- bounded response and error body reads;
- status/code parsing and safe error mapping;
- provider response variant handling;
- usage, finish/refusal, resolved-model and request-ID extraction;
- streaming event validation if it implements a streaming capability;
- cancellation/drain behavior and telemetry.
The harness never imports the provider SDK. Upgrading the SDK should affect the adapter crate and its conformance fixtures, not domain state machines.
SSRF and endpoint policy
Section titled “SSRF and endpoint policy”Do not accept arbitrary request-supplied base URLs. Resolve adapters from an allowlisted registry. Block link-local/cloud metadata addresses, loopback/private ranges as policy requires, redirects to wider trust boundaries, and untrusted proxy configuration. Pin TLS trust/configuration according to deployment policy.
Interchangeability is conditional
Section titled “Interchangeability is conditional”Two adapters are substitutable for one call only if all required conditions hold:
both implement TextGeneration∧ request passes both declared limits∧ required seed/JSON/stream/tool semantics exist∧ topology/data policy allows both∧ release has passed the task's quality/safety gates∧ latency/cost/capacity policy admits the routeTrait implementation proves only the first condition. Runtime validation proves some request conditions. Registry policy/evals prove the rest.
Do not route solely because both values fit Box<dyn TextGeneration>.
Avoid the lowest-common-denominator trap
Section titled “Avoid the lowest-common-denominator trap”Keep a small stable core, then model genuinely optional semantics explicitly:
- separate capability trait where behavior is a different operation;
- a request enum/extension whose unsupported form is preflight-rejected;
- a higher-level harness feature built over primitives;
- an adapter-specific escape hatch isolated from portable product logic.
If only one provider has a valuable feature, it can still be used. Mark the route nonportable and test its fallback. Neutrality should not erase capability.
Static versus runtime dispatch
Section titled “Static versus runtime dispatch”Use generics when:
- one adapter is wired for the whole binary/test;
- composition is known at compile time;
- you want concrete associated types or possible inlining;
- binary-size/monomorphization trade-offs are acceptable.
Use Arc<dyn TextGeneration> or Box<dyn TextGeneration> when:
- route selection happens from configuration or per task;
- several adapters coexist in a registry;
- plugins/feature flags choose implementations;
- uniform collections matter.
The dynamic dispatch overhead is negligible beside most inference/network work, but measure rather than repeat folklore. More important costs here are a boxed async future, request cloning, serialization, queueing, and backend execution.
Prefer Arc<dyn ...> for shared immutable adapter instances. The companion implements
TextGeneration for Arc<T> so generic and dynamic composition remain convenient.
Streaming needs its own terminal protocol
Section titled “Streaming needs its own terminal protocol”Do not change generate to return arbitrary byte chunks. A robust stream distinguishes:
started(identity, backend_request_id?)delta(sequence, typed content)usage_update(...)completed(finish, final usage)refused(reason)failed(classification)It must define:
- ordering and duplicate policy;
- maximum event/chunk/aggregate bytes;
- UTF-8/JSON boundary assembly;
- backpressure;
- heartbeat/idle timeout;
- disconnect cancellation;
- whether partial output is publishable;
- one terminal event;
- resumability/replay if offered.
A nonstreaming adapter may buffer a provider stream, but then it must enforce aggregate bounds and surface a missing terminal event as protocol failure. Later harness chapters add this as a separate capability rather than weakening the current result contract.
Build an adapter conformance suite
Section titled “Build an adapter conformance suite”Every TextGeneration implementation should run the same behavior cases:
- empty and oversized input fail before driver invocation;
- zero/excess output budget fails;
- unsupported seed and JSON mode are distinct;
- exact maximum boundaries pass;
- generated empty text is invalid;
- output byte bound is enforced;
- refusal is preserved, not turned into empty text/error;
- finish reason is preserved;
- known usage stays exact; unknown stays
None; - usage beyond request limit fails;
- backend request/release identity is present/bounded;
- timeout/rate limit/unavailable/rejected/protocol/internal remain distinct;
- no secret or content appears in normal logs/errors;
- cancellation respects the adapter’s documented cleanup contract;
- concurrency never corrupts response/request association.
Then add adapter-specific fixtures: real provider JSON variants, local tokenizer/runtime failures, stream truncation, SDK upgrades, proxy responses, and unknown enum fields.
The companion’s ten tests cover identity bounds, preflight, feature differences, topology mismatch, exact scripted consumption, failure classification, malformed success, usage overflow, and three-way dynamic dispatch.
Routing belongs above adapters
Section titled “Routing belongs above adapters”A router consumes:
- required capability and features;
- input/output/media/context shape;
- data-boundary and tenancy policy;
- exact evaluated releases;
- current capacity/circuit state;
- quality by task slice;
- latency/cost budget;
- fallback compatibility.
It returns a route decision and reason. The adapter executes that decision. If the adapter silently chooses another model/provider, trace identity and eval attribution become unreliable.
Fallback is a new decision, not an internal retry. Record:
original routefailure classfallback eligibility ruleselected fallback releasequality/cost implicationsNever downgrade a high-risk task to an unevaluated cheaper model because the preferred provider timed out.
Security implications
Section titled “Security implications”- Resolve adapters from trusted IDs; never deserialize arbitrary implementation types.
- Authorize model/capability/topology separately from authenticating the caller.
- Enforce request bounds before cloning, serialization, tokenization, or network I/O.
- Keep provider keys in secret types and out of
Debug, URLs, traces, and error messages. - Validate/sanitize remote response IDs, model names, error codes, text, usage, and finish variants.
- Bound remote bodies even on errors; an error response is untrusted input.
- Treat local model files/native runtimes as a supply-chain and memory-safety boundary.
- Prevent cross-tenant cache, trace, batch, and response association.
- Do not let model output select an unrestricted adapter, endpoint, account, or expensive feature.
- Preserve refusal/safety outcomes; do not auto-retry them into another provider unless explicit policy permits and evaluation covers that behavior.
- Separate retry budget from model-call/cost budget and make attempts observable.
- Pin egress, TLS, redirects, DNS/proxy behavior, and allowed regions.
- Audit capability/release drift before promotion.
Failure investigations
Section titled “Failure investigations”Failure: local adapter passes, remote adapter fails JSON verification
Section titled “Failure: local adapter passes, remote adapter fails JSON verification”Compare rendered prompt/roles, JSON-mode semantics, stop reason, truncation, output byte/token limit, and provider safety output. “Supports JSON” may mean syntactic JSON only. Keep the harness verifier constant and inspect adapter translation.
Failure: a seeded replay changes after provider migration
Section titled “Failure: a seeded replay changes after provider migration”The seed is one input, not a cross-runtime determinism guarantee. Compare resolved model, tokenizer, template, sampling algorithm/order, numerical kernels, parallelism, and provider version. Mark the scope of determinism in the adapter capability manifest.
Failure: rate limits cause a request storm
Section titled “Failure: rate limits cause a request storm”Confirm the adapter returns RateLimited and retry_after_ms. Inspect outer retry count, jitter,
deadline, concurrency, circuit breaker, and whether each application replica retries
independently. Retry policy should suppress synchronized amplification.
Failure: script tests pass while production response parsing breaks
Section titled “Failure: script tests pass while production response parsing breaks”Scripted outcomes test harness behavior, not provider serialization. Add captured/redacted provider protocol fixtures and adapter-level parser tests. Run a canary against a pinned provider release.
Failure: runtime-selected adapter violates data residency
Section titled “Failure: runtime-selected adapter violates data residency”Inspect route decision, registry topology metadata, endpoint/region resolution, redirects/proxy, and trace identity. Make data-boundary policy a precondition to route selection, not an adapter hint.
Failure: cancelling the future does not stop billed remote work
Section titled “Failure: cancelling the future does not stop billed remote work”Dropping an HTTP future may close the connection without cancelling accepted provider computation. Use provider cancellation when available, define idempotency/operation IDs, and charge budget according to possible acceptance. Do not claim cancellation beyond observed semantics.
What the companion intentionally does not implement yet
Section titled “What the companion intentionally does not implement yet”- a real provider HTTP client or credentials;
- a real Candle/mistral.rs/ONNX engine;
- roles/messages, tools, schemas beyond JSON-object mode, or conversation state;
- tokenization/context rendering;
- streaming;
- retry, routing, circuit breaking, or fallback;
- pricing;
- image/audio/video capabilities;
- a provider-specific evidence extension;
- cancellation/deadline wrappers.
Those exclusions prevent a fixture from being mislabeled as production inference. The ports and conformance rules are the completed Part 3 exit artifact; subsequent parts add real modalities and runtimes behind them.
Exercises
Section titled “Exercises”Recall
Section titled “Recall”- Why is a provider-neutral trait not necessarily a lowest-common-denominator API?
- Why should embedding not be an optional method on
TextGeneration? - What does implementing
TextGenerationprove about quality? - Why are input bytes and context tokens different bounds?
- Why is refusal not a backend error?
- Why is JSON-object mode insufficient for schema acceptance?
- What is the role of
AdapterIdentity? - Why does an HTTP request ID not identify a model release?
- Why is
BackendFailureKindnot aretryableboolean? - What does
async_traitchange for dyn dispatch?
Implementation
Section titled “Implementation”- Add a versioned
TextEmbeddingtrait with bounded batch input and fixed-width, finite vectors. - Add refusal and output-limit cases to the scripted example.
- Build a conformance helper generic over
T: TextGeneration. - Build a registry returning
Arc<dyn TextGeneration>by trusted adapter ID. - Add a deadline wrapper that returns typed timeout without hiding a possibly accepted operation.
- Add a bounded retry wrapper using full jitter, attempt/cost budget, and injected deterministic clock/randomness.
- Implement a private inference-service adapter distinct from remote-provider topology.
- Implement a streaming trait with sequenced typed events and one terminal event.
- Add redacted protocol fixtures for one provider without placing credentials/content in git.
- Connect a local runtime and record resolved artifact/runtime/device identity.
Design
Section titled “Design”- Design a router for generation plus embeddings without a universal model trait.
- Model provider-specific reasoning effort while keeping portable product logic testable.
- Decide how capability manifests are signed, cached, promoted, and rolled back.
- Design fallback for high-risk tasks where only one release passes the safety gate.
- Design cancellation semantics for queued local, running native, private service, and accepted remote work.
- Decide which provider evidence fields stay in the portable result and which belong in a typed extension.
Solution guidance
Section titled “Solution guidance”- Neutrality fixes a stable semantic core while allowing explicit extensions/separate traits.
- The input/output shape and quality contract differ; optional methods lie about compile-time support.
- It proves structural operation compatibility only; evaluation proves task suitability.
- Bytes bound memory/transport; the pinned tokenizer/template determines context units.
- The backend successfully returned a modeled non-generation outcome; policy may handle it differently.
- Syntax does not establish schema, semantics, grounding, or policy.
- It attributes behavior to adapter implementation, resolved release, and topology.
- It identifies one invocation, not the executable model composition.
- Retry is a caller policy combining failure fact with idempotency, deadline, attempts, cost, and risk.
- It produces a boxed future-compatible method so the trait can be used behind
dyn.
For the registry exercise, validate config into a closed enum, construct only approved adapter
types, store Arc<dyn TextGeneration>, and return a typed missing/unauthorized route error. Do not
construct URLs or SDK clients from untrusted request fields.
For retry, inject sleeper/clock/jitter sources; freeze failure sequences; assert attempt timing and total budget exactly. Never sleep in unit tests.
Check your understanding
Section titled “Check your understanding”- Can two adapters be trait-compatible but not safe substitutes for a task?
- Where should unsupported seed fail?
- Can an adapter report usage as
None? - Why should a scripted mismatch leave the next exchange intact?
- Which layer verifies final structured output?
- Which layer decides retry and fallback?
- What identity must accompany every result?
- How is a provider refusal different from HTTP rejection?
- When should you use generics instead of
dyn TextGeneration? - What must a new image-generation capability model that text does not?
- What happens when a provider adds an unknown terminal variant?
- How would you prove cancellation releases local KV/cache resources?
Completion checklist
Section titled “Completion checklist”- traits are named by operations/capabilities, not providers;
- unsupported operations are absent from types rather than optional methods;
- request/result types do not expose provider SDK structs;
- global and adapter-specific bounds fail before backend work;
- capabilities are keyed to evaluated adapter/model/runtime releases;
- storage/topology/release identity remains visible;
- generated, truncated, refused, and failed states cannot be confused;
- usage is exact or absent, never invented;
- backend request IDs are bounded/sanitized;
- failure categories preserve retry inputs;
- retry, routing, and fallback live above adapters;
- scripted requests match exactly and consume once;
- local blocking/device work is scheduled safely;
- remote endpoints, auth, deadlines, redirects, bodies, and errors are bounded;
- one conformance suite runs against every adapter;
- provider-specific parser fixtures cover unknown/malformed variants;
- streaming has typed sequence/terminal semantics if offered;
- cancellation semantics are documented and tested per topology;
- no secrets/user content leak through errors or telemetry;
- task quality/safety evals gate every route;
- rollback restores adapter, model, capability, and routing releases together.
Authoritative references
Section titled “Authoritative references”- The Rust Reference: traits
- The Rust Reference: trait object types
- The Rust Programming Language: futures and async syntax
- Rust
asynckeyword async-traitcrate documentation- HTTP Semantics RFC 9110:
Retry-After - OpenAI model feature and endpoint catalog
- Anthropic: handling stop reasons
- Hugging Face Candle repository
- ONNX Runtime architecture and execution providers
Part 3 is now executable end to end: verified model artifacts, transparent transformer/sampling mechanics, context/KV/latency arithmetic, behavioral calibration, distinct task outputs, immutable release audits, topology selection, deployment memory planning, and three interchangeable text-generation adapters behind an honest capability contract. Part 4 will carry this discipline into typed multimodal evidence.