Skip to content

Require secret leaderboard pass for public scores#495

Merged
msaroufim merged 1 commit into
gpu-mode:mainfrom
SinatrasC:strict-secret-leaderboard-validation
Jun 18, 2026
Merged

Require secret leaderboard pass for public scores#495
msaroufim merged 1 commit into
gpu-mode:mainfrom
SinatrasC:strict-secret-leaderboard-validation

Conversation

@SinatrasC

Copy link
Copy Markdown
Contributor

Summary

  • Require a matching passed secret leaderboard run on the same submission and runner before a public score is eligible for leaderboard ranking/count surfaces.
  • Keep failed secret rows exclusion intact, including failures from non-leaderboard secret modes.
  • Apply the same validity rule to HF export rows so public-only scores are not exported as successful leaderboard results.
  • Add regression coverage for public scores with missing secret leaderboard validation.

Testing

  • DOCKER_CONFIG=/tmp/docker-config-anon uv run --with pytest pytest -q tests/test_leaderboard_db.py tests/test_hf_export.py
    • 79 passed
  • git diff --check

@msaroufim msaroufim merged commit 2fbe079 into gpu-mode:main Jun 18, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants