Held-back data proves a model only if the answers were fixed first, every prediction came before the answers were opened, and no failed run was quietly dropped. Blind Test Proof seals all three into Bitcoin and publishes the scores, every attempt included.
Fingerprint the held-back data before anyone predicts it, or set a closing time for a future window. The scoring rule is sealed with it.
Every prediction run numbered 1, 2, 3 in the order it was sealed, candidate or established method. Nothing can be withdrawn.
The answers must match the fingerprint sealed at the start, or the reveal is refused. Every attempt is scored, failures included.
A public study page: every attempt, when it was sealed, its score, best against the established method, and how many tries it took.
Prove a benchmark result was not trained on the test set or picked from fifty quiet runs.
Demand, energy, markets, weather: predictions sealed before the period they forecast.
Pre-registered predictions against held-back observations, failures published with the wins.
Show a regulator the model was validated on data it never saw, on a record nobody can rewrite.
Predicted outcomes fixed before the trial data is unblinded.
Open studies: anyone with a key enters, every entry timed and scored on the same sealed answers.
# 1. Seal the study: the held-back answers are fingerprinted in memory, never stored
curl -X POST https://sebbi.pro/x/blindtest/create -H "Authorization: Bearer $SEBBI_KEY" -d '{
"title":"Solar wind arrival times, held-back windows","metric":"mae",
"answers":{"w-001":412.5,"w-002":388.1, ...}}'
# -> study BT-..., holdout_commitment 9f2c..., public page
# 2. Every prediction run, the established method included
curl -X POST https://sebbi.pro/x/blindtest/predict -H "Authorization: Bearer $SEBBI_KEY" -d '{
"study":"BT-...","label":"network model v3","role":"candidate","predictions":{"w-001":409.0, ...}}'
# -> attempt 7, sealed
# 3. Reveal once: must match the sealed fingerprint, every attempt is scored
curl -X POST https://sebbi.pro/x/blindtest/reveal -H "Authorization: Bearer $SEBBI_KEY" -d '{
"study":"BT-...","answers":{"w-001":412.5, ...}}'
# -> BEATS_BASELINE, best candidate attempt 7 of 9, sealed
The held-back answers are fingerprinted in memory and never stored. For data that does not exist yet, set a closing time instead: attempts stop at that moment and the answers can only be revealed after it. Scoring rules: exact match, within a tolerance, mean absolute error, root mean square error. Anyone can check a study page without a key. 50p per study per month after 7 days free.