Add challenge prequalification fairness guard#423
Conversation
|
Hardening update pushed in 25af825: conflicted reviewer scores are now excluded from weighted threshold evidence, while the conflict finding remains auditable. I added a regression that failed before the fix with 75 == 50 and now passes. Validation refreshed locally: npm run check, npm test (6 tests), node --check on index/demo/test, ffprobe on demo.mp4, git diff --check, and sensitive-term scan returned no matches. |
|
Follow-up hardening pass for the challenge prequalification fairness guard. What changed:
Validation:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed:
Why this matters:
Validation:
|
|
Hardening update pushed in 2ff5d97: individual rubric weight values are now validated, not just the total. This closes the fairness gap where a malformed rubric could still sum to 100 while containing a negative criterion and an over-100 criterion, distorting weighted prequalification outcomes. Verification refreshed:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in 7bb2370:
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Why this matters:
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Why this matters:
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Why this matters:
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Why this matters:
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Why this matters:
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Why this matters:
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Why this matters:
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Why this matters:
Validation refreshed locally:
|
|
Hardening update pushed in What changed:
Validation refreshed locally:
PR state after push: open, |
|
Hardening update pushed in What changed:
Validation refreshed locally:
|
|
Follow-up competitive hardening pass for the challenge prequalification fairness guard. What changed in
Validation refreshed locally:
|
|
Hardening update pushed in What changed:
Fresh validation:
|
|
Hardening update pushed in Validation refreshed locally:
|
|
Hardening update pushed in What changed:
Fresh validation:
|
|
Pushed a focused hardening commit for malformed published criteria evidence: b2c3a66. A prequalification packet with criteria: [null] previously crashed at publicCriteriaIds; it now emits a high-priority malformed-published-criterion-entry fairness hold and routes remediation to publish-complete-screening-criteria before sponsor decisions can take effect. Verified npm test (30), npm run check, node --check, 15 JSON packet parses, ffprobe on the H.264 1280x720 demo video, diff checks, and a focused restricted-string scan. |
|
Fresh hardening pushed in Verification passed: red regression first reproduced the blank challenge ID release, then |
|
Pushed focused hardening commit A Verification passed: red regression captured first, |
/claim #18
Summary
Adds a distinct
challenge-prequalification-fairness-guard/slice for Scientific Bounty System issue #18. The guard evaluates sponsor-side prequalification rounds before solver teams are accepted or rejected.It checks well-formed top-level prequalification packets, challenge identity, published screening criteria, criteria-list evidence, criterion identifiers after trimming, valid weights and totals, valid thresholds and reviewer quorum, sponsor accept/reject decisions, applicant identities, applicant-list evidence, review-list evidence, reviewer identities, duplicate reviewer score evidence, valid finite 0-100 score values, anonymous-screening requirements, reviewer conflicts, rejection reasons, appeal windows, and deterministic audit evidence.
Conflicted reviewer scores are excluded from threshold scoring while the conflict remains auditable. Repeated reviewer identities are deduplicated before quorum or threshold scoring. Malformed top-level packets, missing challenge identities, missing applicant lists, malformed applicant rows, missing or duplicate applicant identities, missing criteria lists, malformed criteria entries, missing review lists, malformed review entries, missing reviewer identities, invalid sponsor decisions, and invalid score values are held before solver access can change.
Latest Hardening
5e5c287: malformed top-level prequalification packets such asnullnow produce a deterministic reviewer packet instead of crashing atround.challengeIdbefore evidence generation.unidentified-challengeandunidentified-applicant, emitsmalformed-prequalification-roundwith missing list/identity evidence, and routes high-prioritycomplete-prequalification-evidenceremediation.reports/malformed-prequalification-round-packet.jsonand refreshed the generated Markdown, SVG, and MP4 evidence.Prior hardening also covers malformed criteria entries, missing/blank challenge identities, malformed review entries, malformed applicant entries, duplicate applicant identities, missing applicant lists, missing criteria lists, missing review lists, missing or blank reviewer identities, duplicate reviewer score evidence, incomplete reviewer scores, invalid score values, invalid quorum/threshold/sponsor decisions, appeal-window errors, hidden criteria, reviewer conflicts, and malformed rejection-reason evidence.
Non-overlap
This is scoped to prequalification fairness before solver acceptance or rejection. It does not duplicate broad bounty-system modules, intake compliance, workspace privacy, clarification freeze, arbitration/scoring, payout eligibility, sponsor data-room access, benchmark leakage, evaluator calibration, reviewer workload SLA, award transparency, appeals, escrow settlement, sponsor reliability, submission quarantine/security, deadline fairness, COI recusal, withdrawal reimbursement, localization, communication parity, onboarding clocks, regulatory attestation, or payout-routing slices.
Validation
TypeError: Cannot read properties of null (reading 'challengeId')atchallengeIdFor.npm testfromchallenge-prequalification-fairness-guard-> 32 challenge prequalification fairness tests passed.node --checkpassed forindex.js,test.js, anddemo.js.npm run demoregenerated 17 JSON packets plus Markdown and SVG evidence.npm run videoregeneratedreports/demo.mp4.npm run checkpassed the package test, demo, and video sequence.reports/malformed-prequalification-round-packet.json.ffprobeverifiedreports/demo.mp4as H.264, 1280x720, 30 fps, 4.0s, 120 frames, 46,164 bytes.git diff --checkandgit diff --cached --checkpassed; only Windows line-ending normalization warnings appeared.challenge-prequalification-fairness-guard/.OPEN, merge stateCLEAN, head5e5c287; no checks are reported for this branch.Demo Artifacts
challenge-prequalification-fairness-guard/reports/prequalification-fairness-packet.jsonchallenge-prequalification-fairness-guard/reports/missing-criterion-id-packet.jsonchallenge-prequalification-fairness-guard/reports/normalized-criterion-id-packet.jsonchallenge-prequalification-fairness-guard/reports/invalid-reviewer-score-packet.jsonchallenge-prequalification-fairness-guard/reports/invalid-reviewer-quorum-packet.jsonchallenge-prequalification-fairness-guard/reports/invalid-sponsor-decision-packet.jsonchallenge-prequalification-fairness-guard/reports/missing-applicant-identity-packet.jsonchallenge-prequalification-fairness-guard/reports/duplicate-applicant-identity-packet.jsonchallenge-prequalification-fairness-guard/reports/missing-review-list-packet.jsonchallenge-prequalification-fairness-guard/reports/malformed-review-entry-packet.jsonchallenge-prequalification-fairness-guard/reports/missing-criteria-list-packet.jsonchallenge-prequalification-fairness-guard/reports/malformed-criterion-entry-packet.jsonchallenge-prequalification-fairness-guard/reports/missing-applicant-list-packet.jsonchallenge-prequalification-fairness-guard/reports/malformed-prequalification-round-packet.jsonchallenge-prequalification-fairness-guard/reports/missing-challenge-identity-packet.jsonchallenge-prequalification-fairness-guard/reports/malformed-applicant-entry-packet.jsonchallenge-prequalification-fairness-guard/reports/blank-rejection-reason-packet.jsonchallenge-prequalification-fairness-guard/reports/prequalification-fairness-report.mdchallenge-prequalification-fairness-guard/reports/summary.svgchallenge-prequalification-fairness-guard/reports/demo.mp4Synthetic data only. No credentials, payment processors, identity providers, private workspaces, sponsor systems, solver accounts, payout systems, or external APIs are used.
AI-assisted with OpenAI Codex; I reviewed and locally verified the diff before submitting.