FallForge Gate — proof-of-play for small language models
A tuned SLM is worth exactly what it can prove. This harness measures a candidate model against a baseline on a named use-case — deterministic scorers, real latencies, a verdict that can and does say LOSES — and seals the numbers into a tamper-evident receipt. Every claim on a receipt is a measurement, never an assertion.
The shipped receipt — re-verified in your browser, right now
This page carries the gate's own kernel. On load it fetches the repo's receipt.json — a real run of two real local models — and re-verifies the hash in front of you.
loading receipt…
Verify any receipt
Score a run yourself (advanced) — eval set + outputs, scored by the same kernel
Paste an eval set and one outputs array per side. The kernel scores, compares, and hands you the verdict — all client-side, nothing leaves this page.
How the gate works
Deterministic scorers only — exact match, required substrings, numeric tolerance, JSON field checks. No LLM judge: correlated checkers give false confidence, and a judge model grading its cousin is theatre.
A verdict that can refuse — fewer than probes is not evidence, so the gate will not certify on it. An equal score is not a win. A gate that cannot say LOSES is not a gate.
Receipts are scoped — a receipt says "on THIS probe set, measured", never "better in general". The scope line is inside the hash.
Tamper shows — the receipt's facts are canonically serialized and SHA-256 hashed; flip a verdict, inflate a margin, sandbag the baseline — the hash breaks, and this page shows it.
Honest limits (v1)
Receipts are hash-self-verified — tamper-evident, not yet signed. Ed25519 signing with the-wallet identity is v2; today a receipt proves internal consistency, not who issued it.
Deterministic scorers can't grade open-ended prose. The gate covers extraction, classification, structured output — the use-cases SLMs are actually minted for. Fuzzy judging is deliberately out.
Latency is measured on whatever machine ran the eval — comparable within a receipt, not across machines.
The mutation witness gates the kernel in CI (survivors need a written equivalence argument); the shipped receipt is re-verified against the shipped kernel on every push.