Scoring

The composite score, translation metrics, the grounding gate, generalization, pose scoring, and leadership stability.

Composite score

S_total = 0.60·S_translation + 0.25·S_generalization + 0.15·S_pose

A submission that fails the grounding gate receives zero, regardless of the composite score.

Translation score

S_translation = 0.50·BLEURT_norm + 0.30·chrF_norm + 0.20·BLEU_norm

Computed over the studio, broadcast, and conversational slices only. The canary slice is deliberately excluded so that in-domain strength cannot hide a generalization collapse.

Grounding gate

The problem

A fluent system can earn non-zero translation credit while completely ignoring the video. Unrelated English sentences from the same domain share vocabulary, register, and sentence shape, so a good language prior scores above zero on its own.

The test

Signet scores every hypothesis against two reference sets: the true references, and a deterministic derangement of those same references. A derangement is a shuffle in which no sentence stays matched with its own clip.

G = M(hypotheses, true references) / M(hypotheses, deranged references)

If a model is reading the video, it matches the true references much better than the shuffled ones and G rises above 1. If it is producing generic domain-shaped English, it scores about the same against both and G sits near 1.

The V1 gate

The threshold is calibrated against the private corpus before economic enforcement.

Generalization

S_generalization = min(1.0, (BLEURT_canary / BLEURT_main) / 0.85)

This component is graded, not a hard zero. A broadly trained model may translate correctly while using a different English register from the reference interpreter, and that should cost points rather than disqualify.

The division of labour is precise:

• the grounding gate decides whether the video was genuinely read;
• the generalization component prices how well that ability transfers to the target operating conditions.

Pose score

Pose is scored with normalized, inverted DTW-PA-MPJPE over body and hand landmarks.

• Procrustes alignment (PA) removes camera translation, scale, and rotation.
• Dynamic time warping (DTW) removes harmless timing differences.

Pose is not the primary anti-cheat mechanism. It exists to create a common visual representation, provide a weak input-processing signal, and accumulate the data foundation for a later text-to-sign product line.

Leadership stability

The leaderboard is ranked by composite score. The leader changes only when the challenger's advantage is significant under a paired bootstrap at p < 0.05.

This exists so that the commercially deployed champion does not change because of sampling noise.

Summary of what each component defends