Risks & FAQ
Security and failure model, key project risks, and common questions.
Security & failure model
The mechanism publishes explicit rejection reasons rather than hiding failures inside one number. Core rejection classes:
| Class | Trigger |
|---|---|
| Integrity | Weight hash mismatch |
| Availability | Unreachable or gated model revision (R02); a runtime profile the network does not offer, or a model whose entry point fails to load in it (R03) |
| Capacity | Parameter cap violation; manifest and actual parameter count disagree |
| Behaviour | Non-determinism |
| Isolation | Attempted egress or runtime download dependency |
| Budget | Runtime budget violation |
| Format | Runner output that does not parse: a line that is not a JSON object (R09); a line without a string clip_id and translation, or with a pose that is not a list (R10); a pose that does not match the 543-landmark layout or the clip's frame count (R11); an output that does not have exactly one parsed line per clip (R12). The runner writes every line and coerces the translation; only a pose that is not a list can fail R10 from the return type |
| Execution | The network's runtime container failed to start or crashed (R13). Classified as infrastructure: the run is retried before the code is published, because a host that ran out of disk has not shown the submission to fail anything |
| Grounding | Failed canary grounding gate (R14) |
Infrastructure failure is separated from miner failure. A validator that cannot execute the round honestly abstains instead of fabricating or reusing a stale score.
Key risks
Accuracy risk
The largest risk is that automated translation does not reach production quality in any near-term vertical. Containment: measure private-domain performance before committing to a broad commercial claim, and enter reviewable asynchronous workflows first.
Data risk
Private corpus acquisition and rotation are recurring requirements. They are funded as core infrastructure, not treated as a one-time benchmark purchase.
Gaming risk
Memorization and language-prior bluffing are addressed by private data, frozen offline execution, deranged-reference grounding, canary conditions, and artifact pinning.
Competitive risk
Existing accessibility companies can improve their own AI. Signet's defensibility rests on continuous open research competition plus a private longitudinal evaluation and data asset. It does not rest on a belief that incumbents will remain static.
Community risk
Deaf community participation is required in corpus creation and product evaluation. The project targets unserved content volume and does not position automated output as a replacement for certified human interpretation in high-stakes contexts.
FAQ
Is Signet an inference marketplace? No. Miners do not host endpoints and validators do not call miner-operated services. Miners publish frozen artifacts that validators run themselves.
Can I submit a closed model? No. Weights and the loader class must be in one public, ungated Hugging Face repository pinned to an exact commit SHA, with a SHA-256 hash declared for every file the validator will open.
Can I point at a main branch or a latest tag? No. The revision must be a commit SHA; a branch name is rejected with R02 because it can be moved after the round freezes. There is no image tag to pin: you name a runtime profile, currently torch2.4-transformers4.44, and never a digest. The network publishes that profile as an image pinned by digest when it releases the runtime.
Can my model call an API at inference time? No. Evaluation runs with no network interface, so the call cannot be made. Any egress requirement is a rejection, not a penalty.
Why is there a 2B parameter cap? To reward data quality, representation, and training efficiency instead of raw scale, and to keep validation inside one-GPU economics.
Do public benchmarks affect my score? Never. How2Sign, YouTube-ASL, OpenASL, and PHOENIX-2014T are training substrate. Only the private rotating corpus sets weights.
Will the private clips ever be released? Once the private corpus exists and footage retires from active scoring, that retired footage would be released as a public self-evaluation archive. Active clips are never published.
Why is the canary slice excluded from the translation score? So that strong in-domain performance cannot mask a generalization collapse. The canary slice feeds the grounding gate and the generalization component instead.
What happens if my model is fluent but ignores the video? The grounding ratio stays near 1, the gate fails, and the score is zero under rejection code R14.
Does the champion change every round? No. The leader changes only when the challenger is significantly ahead under a paired bootstrap at p < 0.05.
What if fewer than 10 miners qualify? Unused Arena weight is burned, not redistributed.
Does Signet replace human interpreters? No. Signet does not target regulated, medical, legal, emergency, or Video Relay Service contexts. It targets asynchronous content that today typically receives no sign-language access at all.