Video Generation Funnels, Continuity, and Expensive-Stage Routing

Video generation multiplies image generation by time, but that description still understates the engineering problem. A video must preserve subjects, objects, camera, style, lighting, geometry, causal state, and timing while pixels change. Its cost grows across width, height, frame count, denoising steps, candidate count, temporal context, spatial and temporal super-resolution, frame interpolation, audio, safety analysis, and encoding.

The correct product abstraction is therefore not “send a prompt to the most expensive model.” It is a funnel:

intent → storyboards → cheap previews → measured selection
       → bounded final generation → continuity validation
       → optional enhancement/audio → safety → encoding → publication

This chapter implements that shape in Rust. The companion creates three deterministic storyboard candidates, gives each separate composition and continuity estimates, selects exactly one, checks that its expensive eight-frame stage fits budget, renders a moving subject under a one-pixel continuity constraint, writes a real two-second YUV4MPEG2 stream, validates its frame structure, and emits candidate, cost, frame, timing, and artifact receipts.

The candidate scorer and renderer are pedagogical procedural fixtures, not trained video models. Their purpose is to make routing and continuity observable. A latent video diffusion model can replace the final renderer while retaining the funnel.

Learning objectives

By the end you will be able to:

explain why a video release includes spatial, temporal, decoder, interpolation, and encoder components;
distinguish text-to-video, image-to-video, video-to-video, extension, interpolation, inpainting, and editing contracts;
reason about [batch, channels, frames, height, width] latent cost;
preserve rational frame rate, presentation time, duration, and audio synchronization;
design storyboards, shot plans, reference assets, and continuity state;
separate cheap candidate generation from expensive final rendering;
estimate and reserve cost before accelerator admission;
route only candidates that satisfy quality, safety, and budget floors;
detect identity drift, object disappearance, flicker, camera discontinuity, and physical-state regression;
understand spatial/temporal super-resolution and frame interpolation as distinct transforms;
bind seed, prompt, inputs, models, schedulers, adapters, and every enhancement stage;
validate a real frame stream before publication;
evaluate frame quality, motion, prompt adherence, continuity, diversity, and safety separately;
treat generated video as a derived artifact, never acquired evidence;
design cancellation, checkpointing, retries, and partial-artifact semantics for expensive jobs.

Prerequisites and dependency order

Complete video evidence processing for tracks, DTS/PTS, timescales, variable frame rate, motion, cuts, and coverage. Complete image generation for diffusion, latents, guidance, seeds, output validation, safety, and provenance. Complete timeline and geometry alignment before attaching generated audio or mapping source controls into model space.

Video is the first modality where an early cheap decision can save orders of magnitude of downstream compute. The funnel is not an optimization afterthought; it is part of correctness and abuse resistance.

Completion artifact

Run:

cd rust/rust-ai-engineering
cargo run -q -p mosaic-ml --example video_generation_funnel

The example writes:

target/generated-video/<y4m-sha256>.y4m

The pinned fixture reports:

canvas                 16 × 16 monochrome
rate                   4/1 frames per second
frames                 8
duration               2,000,000 µs
storyboard candidates  3
selected candidates    1
cheap stage units      6
expensive stage units  32
Y4M bytes              2,133
Y4M SHA-256            ab0e33efc8fb0456c790495c586807fb8bd6986edf3d4b4a9ef5fc347ec77cc6

Every frame has an exact presentation time, luma digest, object centroid, mean luma, and difference from the preceding frame. Run:

cargo test -p mosaic-ml --all-features video_generation
cargo test -p mosaic-ml --all-features --example video_generation_funnel

The implementation is crates/mosaic-ml/src/video_generation.rs; the executable is crates/mosaic-ml/examples/video_generation_funnel.rs.

Define the generation mode

“Generate video” hides materially different input and preservation contracts.

Text-to-video

Inputs include prompt, negative condition, duration, dimensions, aspect ratio, frame rate, motion intent, camera intent, seed, and perhaps storyboard. The model invents both appearance and motion, so identity and composition variance are high.

Image-to-video

A reference frame anchors appearance. The contract includes how it is decoded, oriented, cropped/padded, normalized, and positioned in time. Is it the first frame, a style reference, or a soft semantic condition? Does the generated result have to preserve identity or merely color and composition?

Video-to-video

A source video may contribute geometry, motion, depth, pose, optical flow, segmentation, camera, or appearance. Preserve its selected track, frame clock, temporal crop, resampling, spatial transform, and control strength. The output is a derived edit, not a new unrelated clip.

Extension

Forward/backward extension conditions on existing boundary frames. It needs overlap length, source timeline, direction, seam policy, and retained-state contract. The extended section and original must remain distinguishable in lineage.

Inpainting and object editing

Spatiotemporal masks identify regions over time. A static mask copied to every frame differs from a tracked mask. Specify polarity, feathering, interpolation, coordinate space, occlusion behavior, and whether protected pixels are copied or regenerated.

Interpolation

Interpolation creates frames between known endpoints. It should preserve endpoint pixels and timeline unless the contract says otherwise. A generative interpolator may hallucinate motion; do not treat it as a measurement of what happened between real frames.

Use separate typed capabilities. A source-preserving edit and open-ended text generation should not share one permissive request map.

Why video generation is expensive

An image latent might have shape:

[B, C, H, W]

A video latent adds time:

[B, C, T, H, W]

The raw number of latent values grows linearly with frames (T), but attention or state interaction can grow faster. Full attention across all spacetime positions has a quadratic term in:

[ N = T \times H \times W ]

Architectures factor spatial and temporal attention, use local windows, compress time, or operate over spacetime patches to control this cost. The Video Diffusion Models work extended image diffusion with spatial/temporal modeling and conditional extension. Make-A-Video separated spatial and temporal components and used interpolation and super-resolution stages.

A useful capacity estimate is:

resident component weights
+ text/reference encoders
+ latent batch × frames × channels × latent height × latent width
+ denoiser activations and attention workspace
+ CFG branch expansion
+ temporal control/state
+ decoder working set
+ frame-interpolation working set
+ spatial/temporal super-resolution working set
+ decoded frames
+ encoder buffers
+ safety/evaluation models
+ allocator cache
+ safety margin

Candidate count multiplies the expensive portion unless the funnel prevents it. Four final candidates at 96 frames, two CFG branches, and two enhancement passes are not “four generations” in the same sense as four thumbnail previews.

Cascades and spacetime representations

High-resolution models commonly use stages. Imagen Video describes a base generator followed by interleaved spatial and temporal super-resolution models. A general cascade can be:

prompt expansion or shot specification;
keyframe/storyboard generation;
low-resolution, low-frame-count video;
temporal extension;
temporal super-resolution or interpolation;
spatial super-resolution;
latent or pixel decoder;
audio generation and synchronization;
color, loudness, and encoding;
safety/provenance.

Each stage is a transformation with independent release identity and failure. If temporal interpolation changes from release A to B, the output release changed even if the base generator did not.

Another family compresses video spatially and temporally, then decomposes it into spacetime patches. The Sora technical report, Video generation models as world simulators, describes a diffusion transformer operating on variable-sized spacetime latent patches. That report also documents important limits: physical interactions, state changes, long-duration coherence, and spontaneous objects can fail. “World simulator” is a research framing, not evidence that generated physics is reliable.

Autoregressive video-token models instead predict codec or visual tokens. They inherit sampling, context, and error-accumulation concerns from sequence models. Diffusion, flow, and autoregressive architectures all require the same product boundary: exact space/time inputs, bounded compute, validated output, safety, and provenance.

Storyboards are an executable specification

A production storyboard should be structured, not merely a longer prompt:

project
  global style and palette
  character/object bible
  environment and lighting
  aspect ratio and delivery format
  shot 1
    duration and frame rate
    subject state at entry
    action and state transition
    camera/lens/motion
    composition and constraints
    dialogue/audio timing
    subject state at exit
  shot 2
    ...

Bind referenced images, masks, poses, depth maps, trajectories, logos, fonts, and audio by digest. Separate user-authored facts from model-expanded descriptive text. Prompt expansion must not invent a dangerous action, identity, endorsement, or factual claim outside the user’s authority.

For repeated characters, preserve an identity packet:

approved reference assets and crop/normalization maps;
appearance attributes and embeddings;
wardrobe and prop state;
pose/trajectory constraints;
allowed variation;
disallowed identity drift;
model/adapters and strengths;
last accepted frame/latent or state token;
release and authorization receipts.

The identity packet is sensitive. Reference faces and embeddings require tenant isolation, retention controls, and consent appropriate to the use.

Continuity is a set of invariants

Temporal consistency is not “adjacent frames look similar.” A static video has perfect similarity and no meaningful motion. Define the state that must persist or change.

Subject continuity

Track identity, shape, texture, clothing, accessories, and count. Test occlusion and re-entry. Subject embeddings can help, but may ignore hands, garments, or small props.

Object permanence and causal state

If a glass breaks, fragments should persist. If a door opens, its later state should be open unless acted upon. Keep a shot-state ledger:

before → action → expected after → observed after → confidence/reviewer

Generative video is especially prone to silently undoing state transitions.

Geometry and spatial relations

Track boxes, masks, keypoints, depth order, contact, and trajectories. A subject moving behind an object should occlude consistently. A hand holding a cup should maintain contact rather than merge or detach.

Motion

Measure optical flow or tracked displacement, acceleration, jerk, and residual after warping. Smoothness is not correctness: a smoothly sliding foot is still physically wrong. Compare motion to the shot specification.

Camera continuity

Separate camera pan/tilt/zoom/dolly from object motion. Preserve horizon, focal behavior, framing, and shot direction when required. Abrupt global motion might be an intentional cut; do not penalize it as flicker without shot boundaries.

Appearance and lighting

Track color, illumination direction, shadow state, texture frequency, and exposure. Per-frame image quality can be high while brightness pulses every other frame.

The companion uses a tiny but exact invariant: the selected square centroid may move at most one pixel along each axis between frames. This is not sufficient for natural video, but it demonstrates how continuity becomes a checked receipt rather than an adjective.

Frame clocks are rational

For frame index (i) at rational rate (n/d) frames per second:

[ t_i = \frac{i \times d}{n} ]

The companion stores:

[ t_{\mu,i} = \left\lfloor\frac{i \times d \times 1{,}000{,}000}{n}\right\rfloor ]

and total nominal duration:

[ D_\mu = \left\lfloor\frac{T \times d \times 1{,}000{,}000}{n}\right\rfloor ]

At 4/1 fps, eight frames cover two seconds and start at 0, 0.25, …, 1.75 seconds. The final frame’s presentation timestamp is not the duration endpoint.

Use rational rate and integer ticks, not a floating 29.97. Production output may be variable frame rate; then each frame needs an explicit half-open presentation interval. Preserve decode order separately when codecs reorder frames.

When adding audio, choose a canonical program clock and map both video PTS and audio sample ranges to it. “Starts at zero” is not synchronization. Encoder delay, resampling, trims, transitions, and generated pauses all need lineage-bearing maps.

The expensive-stage funnel

Stage 0: request and policy

Authenticate the caller, authorize identities/assets, classify safety, validate dimensions/rate/ duration, estimate cost, and reserve quota. Reject before acquiring model capacity.

Stage 1: structured candidates

Generate multiple storyboard or keyframe candidates cheaply. The companion creates three candidates from prompt/release/seed, each with composition score, continuity score, starting position, trajectory, palette, estimated final cost, and stable ID.

In production, candidates should be materially cheaper: text-only shot plans, low-resolution keyframes, sparse frames, or a distilled preview model.

Stage 2: deterministic filters

Reject invalid geometry, missing subjects, prohibited content, logo/text problems, unlicensed assets, identity mismatch, and predicted over-budget cases. Filters should return reasons.

Stage 3: ranking and diversity

Rank prompt adherence, composition, continuity feasibility, safety confidence, and expected cost. Do not select three near-duplicates. Use a diversity rule where exploration is valuable. Keep scores model/release-specific; do not compare arbitrary raw scores from different rankers.

Stage 4: expensive admission

The final candidate declares required units. In the companion:

8 frames × 4 expensive units/frame = 32 units

A request with 31 units fails before rendering. The system does not “try fewer frames” or silently lower quality. Product-controlled degradation should be an explicit alternative plan accepted by policy and caller.

Stage 5: generation and checkpoints

Generate under an owned durable job. Record stage start/end, model allocation, seed, scheduler, shape, checkpoint, attempts, and cancellation. A retry must not publish two artifacts under one idempotency key.

Stage 6: validation and publication

Decode frames, inspect timing, continuity, safety, audio synchronization, encoder output, and provenance. Only then atomically publish a completed artifact.

Walk through the Rust companion

Validate the full product

VideoGenerationPolicy independently bounds prompt bytes, width, height, pixels per frame, frame count, candidates, total luma bytes, expensive units, and output bytes. validate_request uses checked arithmetic for:

width × height
pixels × frames
frames × expensive cost
frame index × rational timestamp
header + per-frame markers + frame payloads

Width and height bounds do not replace total-pixel or total-frame-memory bounds.

Select exactly one candidate

Candidates are identified by a domain-separated digest of release identity, prompt digest, seed, and ordinal. Ranking sums bounded composition and continuity scores and breaks ties by stable ordinal. After ranking, exactly one selected flag is permitted. The receipt keeps rejected candidates because routing decisions matter to evaluation and cost accounting.

Real rankers need calibrated features and offline tests. A preview’s high aesthetic score can hide motion infeasibility. Include continuity feasibility and safety before expensive generation.

Render and measure frames

The procedural renderer creates a textured luma background and one three-by-three object. A triangular-wave trajectory bounces at frame boundaries without leaving the canvas. Each frame records:

exact index and presentation time;
SHA-256 of luma bytes;
object centroid in milli-pixels;
mean luma;
normalized absolute difference from the prior frame.

Full frame hashes prove identity, not quality. The metrics make simple regressions observable; trained systems need detectors, trackers, optical flow, segmentation, and human review.

Encode real YUV4MPEG2

The artifact begins:

YUV4MPEG2 W16 H16 F4:1 Ip A1:1 Cmono
FRAME
<256 luma bytes>
...

Y4M is a simple uncompressed frame stream useful for transparent fixtures and tool interchange. The companion validates exact header, per-frame markers, payload size, frame count, and total length. It does not pretend that opaque random bytes are H.264.

Production delivery normally uses MP4/WebM/MOV with H.264, HEVC, AV1, VP9, or another supported codec. Keep a high-quality master and treat every encode as a derived artifact. The encoder release, profile, pixel format, chroma subsampling, color primaries/transfer/matrix, range, GOP, bitrate/CRF, frame rate, timestamps, and audio settings are part of release identity.

Enhancement stages are not free postprocessing

Spatial super-resolution

Increases width/height while preserving time. It can invent texture, alter faces, break text, or change small objects. Evaluate temporal stability after upscaling, not only isolated frames.

Temporal super-resolution and interpolation

Adds frames. It can make motion smoother while hallucinating object paths or contacts. Preserve original frame positions and declare which frames are generated. Interpolation must not be presented as captured high-frame-rate evidence.

Frame-rate conversion

Dropping/duplicating frames is not learned interpolation. Both alter timing differently. Audio alignment must follow the selected output clock.

Color and tone transforms

HDR/SDR mapping, gamut conversion, and color grading can clip or shift output. Declare color metadata and test it in the encoded artifact.

Audio

Dialogue, ambience, effects, and music introduce their own generation releases, licenses, consent, timelines, loudness, and safety. Bind audio events to video state. Footsteps should align to contact, not merely share duration.

Re-run validation after every stage that can change pixels, timing, audio, safety, watermark, or provenance.

Long-form generation and temporal chunking

The requested duration can exceed the model’s temporal context. Generating consecutive chunks independently usually causes a seam: subject appearance shifts, camera velocity resets, background geometry changes, or the next action forgets the previous state.

A chunk contract should record:

source shot and canonical frame interval;
context frames supplied before and after the target interval;
overlap frames regenerated by both neighboring chunks;
latent/state checkpoint and compatible release;
subject, object, camera, and causal-state snapshot at the boundary;
seed derivation per shot and chunk;
overlap selection or blending rule;
rejected alternatives and seam scores;
final map from chunk-local frames to published PTS.

There are several strategies:

overlapping extension conditions each new segment on accepted trailing frames;
sliding-window generation keeps a bounded recent context while advancing time;
hierarchical generation creates sparse long-range keyframes before filling intervals;
global low-resolution first establishes the entire motion/story arc before enhancement;
shot decomposition accepts intentional cuts and enforces continuity only across declared project state;
bidirectional bridging conditions on both an earlier and later accepted endpoint.

No strategy guarantees long-range causal consistency. Evaluate recurrence after long occlusion, state changes across many seconds, and repeated chunk boundaries. A seam detector based only on pixel difference will reject intentional motion and accept smoothly morphing identity. Combine motion-compensated appearance, tracked identity/geometry, camera estimates, and state assertions.

When two chunks generate the same overlap, do not average arbitrary RGB frames; ghosting is likely. Select a seam at a low-residual point, use a trained transition, or regenerate the bridge under both boundary conditions. Preserve which frames came from which chunk and transform.

Retries must keep chunk identity stable. If chunk 4 fails, rerunning chunks 1–3 wastes compute and may change accepted history. If a retry changes seed or upstream state, it is a new branch whose downstream chunks must be invalidated. Durable lineage should make that dependency explicit.

Evaluation must be temporal

Prompt and storyboard adherence

Annotate requested subjects, attributes, counts, actions, relations, camera movement, duration, shot boundaries, and state transitions. Score each separately. A video containing all nouns but the wrong action is not prompt-faithful.

Frame quality

Measure blur, exposure, compression, text/OCR, faces, hands, and aesthetic preference, but report them separately from temporal metrics.

Temporal quality

Evaluate:

subject and background consistency;
object permanence and count;
motion smoothness and intended dynamic degree;
optical-flow consistency and warp residual;
flicker after compensating for motion;
trajectory and contact correctness;
camera path;
action completion;
causal state before/after;
long-range recurrence after occlusion;
cut and transition quality;
loop seam, if a loop is required.

VBench is useful as an example of decomposing evaluation into multiple quality and semantic dimensions rather than one score.

Distribution metrics

Fréchet Video Distance compares feature distributions of real and generated video sets. It is not a per-prompt correctness score. Results depend on feature extractor, clip preprocessing, frame count, sample count, and reference distribution. Later work, Beyond FVD, analyzes non-Gaussian features, temporal insensitivity, and sample requirements. Do not use one FVD number as a complete release gate.

Human evaluation

Use blinded paired comparisons with replayable prompt/seed/release manifests. Ask separate questions for:

visual quality;
prompt adherence;
motion plausibility;
subject/scene consistency;
physical/state correctness;
camera behavior;
safety and deceptive potential;
preference.

Randomize order and side, stratify prompts, record reviewer language/domain, measure agreement, and report uncertainty. Inspect full clips at normal speed and frame-by-frame; each reveals different defects.

Safety and provenance

Video can combine identity impersonation, non-consensual intimate imagery, harassment, fraud, political deception, violent/sexual content, minors, copyrighted characters, logos, and misleading factual scenes. Controls must cover inputs, intermediate candidates, final frames, audio, and publication context.

A funnel can stop prohibited content before expensive work:

authorize source media and depicted identities;
classify prompt and reference assets;
reject unsafe storyboard candidates;
run selected final generation in isolation;
sample frames adaptively for safety and identity checks;
inspect audio/transcript;
evaluate temporal evasion, not one thumbnail;
attach provenance/watermark;
publish only after current authorization.

Sampling only the first, middle, and last frames can miss brief prohibited content. Combine fixed coverage, cuts, motion, semantic changes, and uncertainty-triggered sampling; decode all frames for high-risk use cases.

C2PA manifests can record inputs, AI generation, edits, and transcodes under the C2PA specification. Preserve ingredient relationships for source images/video/audio. A valid manifest establishes tamper-evident provenance under its trust model, not that a generated event happened.

In Mosaic, every generated frame stream is derived:

prompt + authorized reference ingredients
        ↓ storyboard candidates and routing receipt
selected candidate + exact generation bundle
        ↓ base video
enhancement + audio + encoding transforms
        ↓
published artifact + validation + provenance

Never feed generated video back into research retrieval as independent support for a factual claim.

Durable jobs, cancellation, retries, and cost

Final video generation is long-running and expensive. Use durable state:

accepted
→ candidates_ready
→ candidate_selected
→ expensive_admitted
→ base_generating
→ enhancing
→ validating
→ publishing
→ completed | failed | cancelled

Persist stage receipts and content-addressed checkpoints only where the model/runtime supports safe resumption. A checkpoint is compatible with an exact release and shape; do not resume it under changed weights or scheduler.

Cancellation must propagate to model processes, decoder, encoder, and upload. Fence stale workers with lease generations. Account actual cost even when validation rejects the artifact. Do not mark a job complete when model sampling ends; completion requires validated publication.

Retry taxonomy:

transient capacity/network/storage errors may retry with the same idempotency identity;
deterministic invalid request/model incompatibility should not retry;
OOM may choose a predeclared lower-cost plan, never silent degradation;
safety rejection is terminal unless policy explicitly allows reviewed appeal;
encoder failure can retry from a validated master without rerunning generation.

Cost records should distinguish cheap candidates, final sampling, enhancement, safety, encoding, storage, and egress. Otherwise the funnel cannot prove it saves anything.

Failure investigation

Subject changes appearance after occlusion

Compare identity references, crop transforms, embeddings, adapters, conditioning strength, context window, last accepted frame/state, and shot boundary. Add occlusion/re-entry evaluation, not just adjacent-frame similarity.

Object disappears or count changes

Inspect storyboard count, detector/tracker confidence, attention/control maps, temporal chunk seam, and enhancement stages. Check whether the object survives low-resolution generation but vanishes in super-resolution.

Smooth motion is physically wrong

Evaluate contacts, depth order, acceleration, support, and state transitions. Optical-flow smoothness alone can reward sliding or melting.

Flicker appears only after upscaling

Compare base and super-resolved frames with motion compensation. Check independent per-frame upscaling, temporal conditioning, tile seams, precision, and color transforms. The enhancement release may be the regression even when the base model is stable.

Output duration or A/V sync is wrong

Inspect rational rates, PTS intervals, frame interpolation, trims, audio sample clock, encoder delay, and mux timebase conversion. Count decoded frames and samples from the final file rather than trusting request metadata.

Cost exceeds estimate

Compare admitted versus actual candidates, steps, CFG branches, frame count, spatial/temporal enhancement, retries, decode, validation, and encoding. Ensure rejected candidates never reach the expensive stage and retries reuse valid masters.

A partial artifact was published after cancellation

Quarantine it, inspect state transition and atomic-publish boundary, fence stale workers, make incomplete object keys unreachable, and add a cancellation race regression at every stage.

Alternatives and trade-offs

Approach	Strength	Cost or limitation
hosted video API	fastest access to managed models	provider drift, data/identity policy, limited internals
local latent video model	privacy and release control	accelerator capacity and operations
image animation	strong appearance anchor	limited motion/3D state
procedural/3D renderer	exact geometry, camera, physics	asset creation and narrower visual domain
stock/approved footage retrieval	factual/licensed source potential	limited novelty and indexing burden
motion graphics/templates	exact text/brand/layout	constrained visual grammar
human production	intentional continuity and performance	time and cost
hybrid storyboard + human edit	controllable quality funnel	workflow/tooling complexity

For charts, UI demos, product explainers, and training procedures, deterministic rendering is often better than generative video. Use a model where synthesis uncertainty is acceptable and evaluated.

Exercises

Recall

Why can video activation cost grow faster than frame count?
What is the difference between temporal super-resolution and frame-rate conversion by duplication?
Why is adjacent-frame similarity not a sufficient continuity score?
At 24/1 fps, what are the timestamp and duration endpoint for frame index 47 in a 48-frame clip?
Why must enhancement models be in release identity?
What work remains after final denoising before a job is complete?

Implementation

Add two shots and an intentional cut. Make continuity validation reset motion at the cut while preserving project-level palette.
Add a required keyframe digest and prove the selected first frame satisfies a declared pixel or embedding tolerance.
Add an optional mono PCM audio track manifest with a rational map to frame time. Reject duration mismatch above a stated tolerance.
Implement a lower-cost fallback plan that requires explicit request permission and produces a different generation identity.
Add checkpoint states and a test proving a stale lease generation cannot publish.
Add adaptive safety-frame sampling using first/last, intervals, cuts, high motion, and semantic uncertainty.

Design

Design a six-stage commercial video funnel with per-stage model, budget, quality, safety, retry, and rollback gates.
Design a continuity evaluation for a two-character scene involving exchanged objects, occlusion, a camera cut, and re-entry.
Capacity-plan ten concurrent 720p, six-second requests with three cheap candidates and one final candidate. State latent geometry, frames, steps, CFG, enhancements, memory, and queue assumptions.
Write an incident runbook for a generated political impersonation published with stripped provenance.

Solution guidance

Frames multiply spacetime positions; global attention can be quadratic in all positions and activation/checkpoint patterns add non-linear pressure.
A temporal model synthesizes intermediate content; duplication repeats samples without new motion and has different timing/artifact behavior.
Static or smoothly wrong frames can score well while identity, action, physics, or causal state fails.
Frame 47 starts at (47/24) seconds; the 48-frame duration endpoint is (48/24=2) seconds.
They change pixels and may alter identity, safety, text, continuity, and numerics.
Decode, continuity/safety/identity checks, audio, encoding, artifact verification, provenance, authorization recheck, and atomic publication.
Add shot ID/cut markers; enforce one-pixel motion only within a shot and declare cross-cut invariants.
Bind decoded/normalized keyframe plus transform; compare at the exact model/output coordinate space.
Map half-open audio sample ranges and frame PTS into one canonical microsecond clock.
Represent fallback as a distinct typed plan with caller consent; bind it to receipt and eval it independently.
Persist lease generation with checkpoint/publication write; reject all stale completions.
Union reasons, deduplicate frames, preserve complete coverage receipt, and fail when the bound cannot cover required risk.
Every stage needs input/output identity, estimated/actual cost, quality/safety floor, retry class, and rollback artifact.
Track both identities, object ownership, masks/boxes, occlusion intervals, shot cut, re-entry identity, and post-exchange causal state.
Calculate per-stage residency and activation peaks, not pixels alone; candidate previews and final/enhancement workloads need separate queues.
Quarantine/take down, preserve restricted evidence, assess distribution and identity harm, notify according to policy/law, revoke route/release, repair provenance durability, and add adversarial transcode/metadata-stripping regressions.

Check your understanding

Is the generation capability typed by mode and source-preservation promise?
Can every input asset be restored by digest and transform?
Are frame rate, PTS, duration, and audio mapping rational and exact?
Does the storyboard preserve state before and after every action?
Are cheap and expensive stage costs independently visible?
Can a rejected candidate accidentally consume final-generation capacity?
Is exactly one selected candidate recorded?
Are base, temporal, spatial, interpolation, decoder, audio, and encoder releases pinned?
Can continuity checks distinguish motion from cuts and camera movement?
Is final encoded media decoded and verified?
Can cancellation prevent stale or partial publication?
Are generated scenes always labeled as derived rather than factual evidence?

Practical completion checklist

Authoritative references

Ho et al., Video Diffusion Models.
Singer et al., Make-A-Video.
Ho et al., Imagen Video.
Blattmann et al., Stable Video Diffusion.
Kondratyuk et al., VideoPoet.
OpenAI, Video generation models as world simulators.
Unterthiner et al., Towards Accurate Generative Models of Video: A New Metric & Challenges.
Huang et al., VBench.
Luo et al., Beyond FVD.
Coalition for Content Provenance and Authenticity, C2PA Specification 2.2.