Day 16 — Build the Evidence Timeline

Today separate specialist outputs become one segment timeline.

Define the join product

pub struct SegmentEvidence {
    pub segment_id: String,
    pub time: TimeRange,
    pub keyframes: Vec<EvidenceId>,
    pub speech: Vec<SpeechSpanId>,
    pub on_screen_text: Vec<TextRegionId>,
    pub audio_events: Vec<AudioEventId>,
}

The struct stores references, not copied payloads. Large pixels and audio remain in the artifact store; the timeline stays cheap to serialize and inspect.

Join by overlap, not nearest guess

An observation belongs to a segment when time intervals overlap according to a named rule. Decide how you handle boundary equality, observations spanning multiple segments, uncertain timestamps, and gaps. Write table-driven tests before implementing the interval join.

segment A: [0, 5)
segment B: [5, 9)
speech:    [4, 7)

Does speech attach to both segments, split at the boundary, or remain a parent observation referenced by both? Any can be valid if downstream citation remains honest.

Make the timeline inspectable

Render a compact table with segment time, thumbnail ID, transcript excerpt, OCR strings, and audio events. Before asking a model for a critique, a human should be able to spot missing or misaligned evidence.

Break it deliberately

Shuffle specialist output order, duplicate an ID, insert a zero-length segment, and shift one clock. The serialized timeline must remain deterministic or fail with a named invariant.

Completion proof

For one generated claim, follow its citation to segment, specialist observation, transformation, and source media timestamp. If the journey relies on array position rather than stable identity, fix it.

Deep reference: Timeline and geometry alignment.

Next: Day 17 — Critique and Route →.