The Rust AI Engineer’s Field Guide

Use this page during design and review.

Core patterns

Typed evidence

Every fact retains source, locator, hash, transform, and trust.

Capability traits

Model adapters implement narrow abilities—generation, embedding, transcription, vision—not one giant vendor interface.

Explicit run state

Preparing, acting, verifying, completed, failed, and awaiting approval are different states.

Deterministic shell, probabilistic center

Code owns parsing, policy, budgets, arithmetic, authorization, and observable verification. Models handle ambiguity.

Generate → check → repair → route

Repair exact violations, cap attempts, and escalate based on measured difficulty.

Durable trace

Every meaningful decision is explainable and replayable without logging secrets.

Eval before optimization

Freeze cases and metrics before changing prompts, models, routing, or weights.

Effects need receipts

Tool success is verified against environment state and protected with idempotency.

Anti-patterns

one giant prompt containing the entire world;
using String for every domain concept;
broad shell or browser tools with ambient credentials;
unbounded retries, queues, context, or agent loops;
trusting model confidence without calibration;
claiming JSON validity proves truth;
evaluating only the final prose;
using a frontier judge on every cheap call;
fine-tuning before diagnosing missing evidence;
hiding model or prompt changes behind a stable version;
publishing generated media directly from the generation step;
measuring cost per request instead of cost per verified completion.

Complete competency map

Rust language

values, expressions, structs, enums, pattern matching;
Option, Result, errors, newtypes;
ownership, borrowing, lifetimes, Cow;
collections, iterators, closures;
traits, generics, trait objects;
modules, crates, workspaces, features;
smart pointers, Send, Sync;
async, cancellation, channels, backpressure;
FFI and safe wrappers when native media or inference requires them.

Rust production

API boundaries and architecture;
Axum, Tower, SQLx, migrations;
streaming upload, SSE, durable jobs;
format, lint, unit, integration, property, fuzz, and golden tests;
profiling, benchmarking, memory, and concurrency limits;
structured tracing, metrics, configuration, secrets;
dependency, supply-chain, CI, release, and rollback discipline.

AI systems

model capabilities and limitations;
token, image, audio, and video representations;
embeddings and retrieval;
local and remote inference;
quantization and artifact management;
context selection and memory;
structured generation;
tool use, environments, and authorization;
workflows, agents, planning, and stopping;
multimodal ingestion and generation.

Harness optimization

task decomposition;
demonstrations and prompt/program optimization;
deterministic verifiers and model graders;
repair loops;
routing and cascades;
test-time compute and search;
synthetic data and distillation;
SFT, LoRA, QLoRA, and preference optimization.

Evaluation and operations

versioned cases and fixtures;
outcome, trajectory, and process grading;
calibration and human review;
quality/cost/latency/risk Pareto analysis;
shadow, canary, monitoring, drift, and incident response;
prompt injection, data boundaries, sandboxing, and provenance.

Design review

Ask:

What observable state proves success?
Which information is evidence and which is instruction?
Which steps are deterministic?
What is the narrowest tool set?
Who authorizes each effect?
What is bounded?
What is replayable?
Which eval slice represents the worst important failure?
Why is this model large enough—and no larger than needed?
Which change can be rolled back independently?

Final principle

The best Rust AI developer is not the person who knows the most model names or crate APIs. It is the person who can turn uncertain capability into a system whose boundaries, evidence, effects, and trade-offs are visible—and then prove that the next version is better.