Benchmarks

Benchmark the runtime, not just the model.

A model can classify a clip correctly and still fail in a continuous product: the queue grows, the same event fires four times, or the cost of watching one stream for an hour makes the feature unshippable. These tracks measure the runtime that sits between the stream and the judge.

No results are published yet. The measurement contract below is pre-registered: the bars are declared before the numbers exist, so a passing run cannot be chosen after the fact. Silence is never a pass.

01 · Tracks

What each track asks.

Three questions, each with its own pass condition. A track is reported whole or not at all.

Queues, memory, stale work

Does the runtime remain bounded under overload?

Drive input faster than the judge. Pass only when pending depth and oldest age remain bounded, supersession is explicit, memory is flat, and stale completions cannot mutate the gate.

oldest pending agememory slopestale completionsdelivery isolation
Recall, precision, cost

Can the watch stay useful and affordable?

Compare judgments per hour and cost only at a held event-recall target. Include fan-out across every active watch so a gate cannot win by silently dropping work.

event recallprecisionjudgments/hourcost/stream-hour
Decision under load

Does reflex latency hold under target concurrency?

Measure actionable latency from decision eligibility to event emission, then decompose admission, queue, provider, gate, and delivery overhead.

p50 / p95 / p99actionable latencyconcurrencyprovider decomposition
02 · Provenance

What a published number has to carry.

A benchmark without its configuration is an anecdote. Every result on this page will ship with all four.

01Locked configuration

Attention profile, episode policy, judge model and version, and delivery policy: pinned, not described.

02Reproducible artifact

The trace bundle the run produced, so the numbers can be recomputed rather than trusted.

03Held target

Cost and judgment-rate claims are only meaningful at a stated event-recall target. The target is declared first.

04Full fan-out

Every active watch counts. A gate cannot win by silently dropping work it was supposed to see.

Results

Numbers land here.

Until then, the honest thing to look at is the runtime itself: run a watch against your own media in the Playground and read the trace it produces.