Three questions — happy to be pointed at documentation I have missed.
1. docs/source/leaderboard.rst links to SUBMISSION.md, which returns 404. Is there a current submission path?
2. Which instance set do the published numbers cover? There appear to be three different populations described:
- the Hugging Face release is 1,136 tasks (Verified 313 / Multi 461 / Poly 219 / Pro 143)
docs/source/leaderboard.rst describes Verified 500 / Pro 2,294 / Poly 640 / Multi 1,000+sel500.csv in this repo is 500 instances spanning all four benches (Verified 174 / Multi 156 / Poly 116 / Pro 54), while the main board is labelled the Verified split
3. How is Context F1 computed? I cannot find an F1 in the scorer — metrics/compute.py returns coverage and precision per granularity, but no combined figure.
Three questions — happy to be pointed at documentation I have missed.
1.
docs/source/leaderboard.rstlinks toSUBMISSION.md, which returns 404. Is there a current submission path?2. Which instance set do the published numbers cover? There appear to be three different populations described:
docs/source/leaderboard.rstdescribes Verified 500 / Pro 2,294 / Poly 640 / Multi 1,000+sel500.csvin this repo is 500 instances spanning all four benches (Verified 174 / Multi 156 / Poly 116 / Pro 54), while the main board is labelled the Verified split3. How is Context F1 computed? I cannot find an F1 in the scorer —
metrics/compute.pyreturns coverage and precision per granularity, but no combined figure.