Skip to content

Add challenge prequalification fairness guard#423

Open
KoiosSG wants to merge 28 commits into
SCIBASE-AI:mainfrom
KoiosSG:challenge-prequalification-fairness-18
Open

Add challenge prequalification fairness guard#423
KoiosSG wants to merge 28 commits into
SCIBASE-AI:mainfrom
KoiosSG:challenge-prequalification-fairness-18

Conversation

@KoiosSG
Copy link
Copy Markdown

@KoiosSG KoiosSG commented May 28, 2026

/claim #18

Summary

Adds a distinct challenge-prequalification-fairness-guard/ slice for Scientific Bounty System issue #18. The guard evaluates sponsor-side prequalification rounds before solver teams are accepted or rejected.

It checks well-formed top-level prequalification packets, challenge identity, published screening criteria, criteria-list evidence, criterion identifiers after trimming, valid weights and totals, valid thresholds and reviewer quorum, sponsor accept/reject decisions, applicant identities, applicant-list evidence, review-list evidence, reviewer identities, duplicate reviewer score evidence, valid finite 0-100 score values, anonymous-screening requirements, reviewer conflicts, rejection reasons, appeal windows, and deterministic audit evidence.

Conflicted reviewer scores are excluded from threshold scoring while the conflict remains auditable. Repeated reviewer identities are deduplicated before quorum or threshold scoring. Malformed top-level packets, missing challenge identities, missing applicant lists, malformed applicant rows, missing or duplicate applicant identities, missing criteria lists, malformed criteria entries, missing review lists, malformed review entries, missing reviewer identities, invalid sponsor decisions, and invalid score values are held before solver access can change.

Latest Hardening

  • 5e5c287: malformed top-level prequalification packets such as null now produce a deterministic reviewer packet instead of crashing at round.challengeId before evidence generation.
  • The packet uses unidentified-challenge and unidentified-applicant, emits malformed-prequalification-round with missing list/identity evidence, and routes high-priority complete-prequalification-evidence remediation.
  • Added reports/malformed-prequalification-round-packet.json and refreshed the generated Markdown, SVG, and MP4 evidence.

Prior hardening also covers malformed criteria entries, missing/blank challenge identities, malformed review entries, malformed applicant entries, duplicate applicant identities, missing applicant lists, missing criteria lists, missing review lists, missing or blank reviewer identities, duplicate reviewer score evidence, incomplete reviewer scores, invalid score values, invalid quorum/threshold/sponsor decisions, appeal-window errors, hidden criteria, reviewer conflicts, and malformed rejection-reason evidence.

Non-overlap

This is scoped to prequalification fairness before solver acceptance or rejection. It does not duplicate broad bounty-system modules, intake compliance, workspace privacy, clarification freeze, arbitration/scoring, payout eligibility, sponsor data-room access, benchmark leakage, evaluator calibration, reviewer workload SLA, award transparency, appeals, escrow settlement, sponsor reliability, submission quarantine/security, deadline fairness, COI recusal, withdrawal reimbursement, localization, communication parity, onboarding clocks, regulatory attestation, or payout-routing slices.

Validation

  • Red regression first reproduced TypeError: Cannot read properties of null (reading 'challengeId') at challengeIdFor.
  • npm test from challenge-prequalification-fairness-guard -> 32 challenge prequalification fairness tests passed.
  • node --check passed for index.js, test.js, and demo.js.
  • npm run demo regenerated 17 JSON packets plus Markdown and SVG evidence.
  • npm run video regenerated reports/demo.mp4.
  • npm run check passed the package test, demo, and video sequence.
  • All 17 generated JSON packets parsed successfully, including reports/malformed-prequalification-round-packet.json.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 120 frames, 46,164 bytes.
  • git diff --check and git diff --cached --check passed; only Windows line-ending normalization warnings appeared.
  • Staged allowlist check confirmed all staged paths are under challenge-prequalification-fairness-guard/.
  • Focused payout/credential scan returned no matches.
  • GitHub PR state after push: OPEN, merge state CLEAN, head 5e5c287; no checks are reported for this branch.

Demo Artifacts

  • challenge-prequalification-fairness-guard/reports/prequalification-fairness-packet.json
  • challenge-prequalification-fairness-guard/reports/missing-criterion-id-packet.json
  • challenge-prequalification-fairness-guard/reports/normalized-criterion-id-packet.json
  • challenge-prequalification-fairness-guard/reports/invalid-reviewer-score-packet.json
  • challenge-prequalification-fairness-guard/reports/invalid-reviewer-quorum-packet.json
  • challenge-prequalification-fairness-guard/reports/invalid-sponsor-decision-packet.json
  • challenge-prequalification-fairness-guard/reports/missing-applicant-identity-packet.json
  • challenge-prequalification-fairness-guard/reports/duplicate-applicant-identity-packet.json
  • challenge-prequalification-fairness-guard/reports/missing-review-list-packet.json
  • challenge-prequalification-fairness-guard/reports/malformed-review-entry-packet.json
  • challenge-prequalification-fairness-guard/reports/missing-criteria-list-packet.json
  • challenge-prequalification-fairness-guard/reports/malformed-criterion-entry-packet.json
  • challenge-prequalification-fairness-guard/reports/missing-applicant-list-packet.json
  • challenge-prequalification-fairness-guard/reports/malformed-prequalification-round-packet.json
  • challenge-prequalification-fairness-guard/reports/missing-challenge-identity-packet.json
  • challenge-prequalification-fairness-guard/reports/malformed-applicant-entry-packet.json
  • challenge-prequalification-fairness-guard/reports/blank-rejection-reason-packet.json
  • challenge-prequalification-fairness-guard/reports/prequalification-fairness-report.md
  • challenge-prequalification-fairness-guard/reports/summary.svg
  • challenge-prequalification-fairness-guard/reports/demo.mp4

Synthetic data only. No credentials, payment processors, identity providers, private workspaces, sponsor systems, solver accounts, payout systems, or external APIs are used.

AI-assisted with OpenAI Codex; I reviewed and locally verified the diff before submitting.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 28, 2026

Hardening update pushed in 25af825: conflicted reviewer scores are now excluded from weighted threshold evidence, while the conflict finding remains auditable. I added a regression that failed before the fix with 75 == 50 and now passes. Validation refreshed locally: npm run check, npm test (6 tests), node --check on index/demo/test, ffprobe on demo.mp4, git diff --check, and sensitive-term scan returned no matches.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 28, 2026

Follow-up hardening pass for the challenge prequalification fairness guard.

What changed:

  • Added an explicit regression for rejected applicants whose appeal deadline has already expired before the prequalification packet is generated.
  • Treat expired appeal windows as expired-appeal-window fairness holds rather than allowing them through as reject-with-audit.
  • Route the remediation to publish-rejection-reasons-and-appeal-window, matching the applicant-facing fairness action for missing appeal windows.

Validation:

  • Confirmed the new regression failed before the implementation with reject-with-audit instead of hold-for-fairness-review.
  • npm test -> 7 challenge prequalification fairness tests passed.
  • npm run check -> test, demo, and demo video generation passed.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s.
  • git diff --check and git diff --cached --check passed, with only Git line-ending normalization warnings on Windows.
  • Sensitive-term scan found only expected local hashing/test assertions, no payout or credential strings.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 28, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed:

  • Added a regression for malformed published rubric weights, where criteria totaled 145 instead of 100.
  • Hold affected applicants for fairness review when the published criterion weights do not sum to 100, before sponsor accept/reject decisions can take effect.
  • Added a high-priority remediation action: publish-valid-weighted-scoring-rubric.
  • Updated the README so the documented guard explicitly covers valid criterion weight totals.

Why this matters:

Validation:

  • Confirmed the new regression failed before the implementation: applicant was incorrectly accept-prequalified with invalid 145-total criterion weights.
  • npm test -> 8 challenge prequalification fairness tests passed.
  • npm run check -> test, demo, and video generation passed.
  • npm run demo -> generated prequalification packet/report/SVG with expected accepted/held counts.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 45,643 bytes.
  • git diff --check and git diff --cached --check passed; the only messages were Git line-ending normalization warnings on Windows.
  • Sensitive-term scan of the code/test/docs patch found no payout or credential strings.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 29, 2026

Hardening update pushed in 2ff5d97: individual rubric weight values are now validated, not just the total. This closes the fairness gap where a malformed rubric could still sum to 100 while containing a negative criterion and an over-100 criterion, distorting weighted prequalification outcomes.

Verification refreshed:

  • Red regression first: npm test failed on the invalid individual weight case (accept-prequalified vs hold-for-fairness-review).
  • Green: npm test passes with 9 challenge prequalification fairness tests.
  • npm run check passes: tests, demo packet/report/SVG, and demo MP4 generation.
  • ffprobe confirms reports/demo.mp4 is H.264, 1280x720, 30fps, 4s, 45,643 bytes.
  • git diff --check and git diff --cached --check pass.
  • Credential/payout-focused scan across changed code/docs/reports returned no matches.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 29, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in 7bb2370:

  • Added a regression for incomplete reviewer score evidence where a review record has no scores map and another only scores part of the published rubric.
  • The guard now treats missing score maps as missing criterion evidence instead of crashing during hidden-criterion or weighted-score evaluation.
  • Affected applicants are held for fairness review with missing-published-criterion-score, and remediation now prioritizes complete-prequalification-evidence before threshold reconciliation.
  • README, requirements map, and acceptance notes now explicitly cover complete reviewer score evidence.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with TypeError: Cannot convert undefined or null to object.
  • npm test -> 10 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated prequalification packet/report/SVG with expected accepted/held counts.
  • npm run video -> regenerated demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index, demo, and test.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 45,643 bytes.
  • git diff --check and git diff --cached --check passed; only Git line-ending normalization warnings appeared on Windows.
  • Sensitive-term scan returned no payout or credential strings.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 29, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in d70a8c8:

  • Added a regression for duplicate non-conflicted reviewer score evidence where the same reviewer identity appeared twice and incorrectly satisfied quorum.
  • Repeated reviewer identities are now deduplicated before reviewer quorum and weighted threshold scoring.
  • Duplicate reviewer score evidence now holds the applicant for fairness review with duplicate-reviewer-score-evidence and routes remediation to deduplicate-reviewer-score-evidence.
  • README, requirements map, acceptance notes, and the demo summary visual now explicitly cover distinct reviewer quorum.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with accept-prequalified instead of hold-for-fairness-review.
  • npm test -> 11 challenge prequalification fairness tests passed.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index, demo, and test.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 45,643 bytes.
  • git diff --check and git diff --cached --check passed; only Git line-ending normalization warnings appeared on Windows before staging.
  • Sensitive-term scan returned no payout or credential strings.
  • GitHub PR merge state after push: CLEAN.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 29, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in 40955b6:

  • Added a regression for rejected applicants whose appeal deadline is malformed or unparseable before the prequalification packet is generated.
  • Appeal-window evaluation now parses both the applicant appeal deadline and packet generation timestamp; invalid timestamp evidence yields invalid-appeal-window instead of being treated as open by string comparison.
  • Rejected applicants with invalid appeal windows are held for fairness review and routed to publish-rejection-reasons-and-appeal-window before rejection packets or challenge-access decisions are published.
  • README, requirements map, and acceptance notes now explicitly cover parseable appeal-window evidence.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with reject-with-audit instead of hold-for-fairness-review.
  • npm test -> 12 challenge prequalification fairness tests passed.
  • npm run check -> test, demo, and video generation passed.
  • npm run demo -> regenerated prequalification packet/report/SVG artifacts with expected accepted/held counts.
  • npm run video -> regenerated reports/demo.mp4.
  • node --check passed for index, demo, and test.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 45,643 bytes.
  • git diff --check and git diff --cached --check passed; only Git line-ending normalization warnings appeared on Windows.
  • Sensitive-term scan returned no payout or credential strings.
  • GitHub PR merge state after push: CLEAN.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 29, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in 2a6c286:

  • Added a regression for malformed sponsor pass thresholds, specifically a negative threshold that previously allowed acceptance by construction.
  • Prequalification rounds now hold affected applicants with pass-threshold-invalid when the threshold is missing, non-finite, below 0, or above 100.
  • Added high-priority remediation via publish-valid-prequalification-threshold before sponsor accept/reject decisions can take effect.
  • README, requirements map, and acceptance notes now explicitly cover valid 0-100 pass thresholds.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with accept-prequalified instead of hold-for-fairness-review.
  • npm test -> 13 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated prequalification packet/report/SVG with expected accepted/held counts.
  • npm run video -> regenerated reports/demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index, demo, and test.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 45,643 bytes.
  • git diff --check and git diff --cached --check passed; only Git line-ending normalization warnings appeared on Windows.
  • Sensitive-term scan returned no payout or credential strings.
  • GitHub PR merge state after push: CLEAN.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 29, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in 3d7c8d7:

  • Added a regression for rejected applicants that omit the rejectionReasons list entirely.
  • Missing applicant rejection-reason lists now normalize to [] instead of crashing prequalification packet generation.
  • The applicant is held for fairness review with missing-rejection-reason, preserving the existing applicant-facing remediation path: publish-rejection-reasons-and-appeal-window.
  • README, requirements map, and acceptance notes now explicitly cover omitted rejection-reason lists, not just empty lists.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with TypeError: Cannot read properties of undefined (reading 'length').
  • npm test -> 14 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated prequalification packet/report/SVG with expected accepted/held counts.
  • npm run video -> regenerated reports/demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index, demo, and test.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 45,643 bytes.
  • git diff --check and git diff --cached --check passed; only Git line-ending normalization warnings appeared on Windows.
  • Sensitive-term scan returned no payout or credential strings.
  • GitHub PR merge state after push: CLEAN.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 29, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in 9be11bc:

  • Added a regression for duplicate published criterion IDs, where a sponsor rubric reused domain-fit under valid-looking weights and previously accepted the applicant.
  • Duplicate published criterion identifiers now hold the applicant for fairness review with duplicate-published-criterion before ambiguous rubric evidence can drive accept/reject decisions.
  • Remediation now routes to publish-unique-screening-criteria with high priority.
  • README, requirements map, acceptance notes, and the demo summary visual now explicitly cover unique criterion identifiers.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with accept-prequalified instead of hold-for-fairness-review.
  • npm test -> 15 challenge prequalification fairness tests passed.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index, demo, and test.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 45,643 bytes.
  • git diff --check and git diff --cached --check passed; only Git line-ending normalization warnings appeared on Windows before staging.
  • Sensitive-term scan returned no payout or credential strings.
  • GitHub PR merge state after push: CLEAN.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in ef304da:

  • Added a regression for high-scoring prequalification reviews whose reviewer identity evidence is blank or missing.
  • Missing reviewer identities now produce missing-reviewer-identity, are excluded from distinct reviewer quorum, and are excluded from weighted threshold scoring until evidence is completed.
  • The affected applicant is held for fairness review and routed to complete-prequalification-evidence before solver access changes.
  • The demo packet/report/SVG/MP4 evidence now includes the missing-reviewer-identity hold, with accepted applicants still 1 and held applicants now 3.
  • README, requirements map, and acceptance notes now explicitly cover missing reviewer identity evidence.

Why this matters:

  • A prequalification round can claim anonymous screening while still requiring internal auditable reviewer identity evidence. Without that evidence, blank reviewer rows could satisfy quorum and approve/reject teams unfairly.
  • This keeps PR Add challenge prequalification fairness guard #423 focused on sponsor-side prequalification fairness while strengthening a core reviewer-quorum invariant.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with accept-prequalified instead of hold-for-fairness-review.
  • npm test -> 16 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated prequalification packet/report/SVG with missing-reviewer-identity included.
  • npm run check -> test, demo, and video generation passed.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 44,989 bytes.
  • git diff --check and git diff --cached --check passed; only Git line-ending normalization warnings appeared on Windows before staging.
  • Sensitive-term scan returned no payout or credential strings.
  • GitHub PR merge state after push: CLEAN; no checks are reported for this branch.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in b803d03:

  • Added a regression for sponsor rubrics with a missing or blank published criterion identifier.
  • Such rubrics now hold affected applicants for fairness review with missing-published-criterion-id instead of allowing a blank criterion key to drive threshold scoring.
  • Added high-priority remediation through publish-complete-screening-criteria before solver access changes.
  • Added reports/missing-criterion-id-packet.json and refreshed the report/SVG/MP4 reviewer evidence so the new gate is visible in the PR artifacts.
  • README, requirements map, acceptance notes, and the PR body now explicitly cover complete criterion identifiers.

Why this matters:

  • Prequalification fairness depends on auditable, published screening criteria. A blank rubric identifier can make sponsor scoring unauditable even when reviewer score evidence appears complete and passes the threshold.
  • This keeps PR Add challenge prequalification fairness guard #423 focused on prequalification fairness while strengthening a core rubric-integrity invariant in the active issue Scientific Bounty System #18 competition.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with accept-prequalified instead of hold-for-fairness-review.
  • npm test -> 17 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated prequalification packet/report/SVG and missing-criterion-id-packet.json.
  • npm run video -> regenerated reports/demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index, test, and demo.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 44,989 bytes.
  • git diff --check and git diff --cached --check passed; only Windows line-ending normalization warnings appeared before staging.
  • Sensitive-term scan returned no payout or credential strings.
  • GitHub PR merge state after push: CLEAN; no checks are reported for this branch.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in 1cb4160:

  • Added a regression for malformed reviewer score evidence where a reviewer supplied an over-100 score (140) and the applicant was previously accepted.
  • Reviewer score values now must be finite numbers in the 0-100 range before they count as threshold evidence.
  • Invalid score evidence now holds the applicant with reviewer-score-value-invalid, uses high-priority remediation publish-valid-reviewer-score-evidence, and excludes invalid values from weighted scoring.
  • Added reports/invalid-reviewer-score-packet.json and refreshed the report/SVG/MP4 reviewer evidence so the new gate is visible.

Why this matters:

  • Prequalification fairness depends on valid reviewer score evidence, not just complete score keys. A single malformed score can inflate threshold evidence and allow solver access changes that should be held for remediation.
  • This keeps PR Add challenge prequalification fairness guard #423 focused on sponsor-side prequalification fairness while strengthening a core scoring-integrity invariant in the active issue Scientific Bounty System #18 competition.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with accept-prequalified instead of hold-for-fairness-review.
  • npm test -> 18 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated prequalification packet/report/SVG and invalid-reviewer-score-packet.json.
  • npm run video -> regenerated reports/demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index, test, and demo.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 44,989 bytes.
  • git diff --check and git diff --cached --check passed; only Windows line-ending normalization warnings appeared before staging.
  • Focused sensitive scan returned no payout or credential strings.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in a4d96c0:

  • Added a regression for malformed reviewer quorum requirements where minReviewers = 0 previously allowed a one-review applicant to be accepted.
  • Reviewer quorum requirements now have to be positive integers before sponsor accept/reject decisions can take effect.
  • Invalid quorum configuration produces reviewer-quorum-invalid, holds the applicant for fairness review, and routes high-priority remediation to publish-valid-reviewer-quorum.
  • Added reports/invalid-reviewer-quorum-packet.json and refreshed the Markdown reviewer evidence so the new gate is visible in the PR artifacts.

Why this matters:

  • Distinct reviewer quorum is a core prequalification fairness invariant. If the quorum value can be zero or malformed, a sponsor can accidentally or deliberately bypass independent review before solver access changes.
  • This keeps PR Add challenge prequalification fairness guard #423 focused on sponsor-side prequalification fairness while strengthening the reviewer-quorum contract in a crowded issue Scientific Bounty System #18 field.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with accept-prequalified instead of hold-for-fairness-review.
  • npm test -> 19 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated prequalification packet/report/SVG and invalid-reviewer-quorum-packet.json.
  • npm run video -> regenerated reports/demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index, test, and demo.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 44,989 bytes.
  • git diff --check and git diff --cached --check passed; only Windows line-ending normalization warnings appeared before staging.
  • Focused sensitive scan returned no payout or credential strings.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in 4b0283e:

  • Added a regression for sponsor rubrics whose published criterion identifiers only differ by surrounding whitespace, e.g. domain-fit and domain-fit.
  • Published criterion duplicate detection now trims identifiers before comparing them, so whitespace-variant duplicates hold the applicant with duplicate-published-criterion instead of allowing ambiguous rubric evidence to drive acceptance.
  • Added reports/normalized-criterion-id-packet.json and refreshed the Markdown reviewer evidence so the new gate is visible in the PR artifacts.
  • README, requirements map, acceptance notes, and the PR body now explicitly cover normalized criterion identifier uniqueness.

Why this matters:

  • Prequalification scoring must be tied to auditable, unique published criteria. Whitespace variants can make a rubric look distinct to code while representing the same sponsor criterion to reviewers.
  • This keeps PR Add challenge prequalification fairness guard #423 focused on sponsor-side prequalification fairness while tightening another core rubric-integrity invariant in the issue Scientific Bounty System #18 competition.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with accept-prequalified instead of hold-for-fairness-review.
  • npm test -> 20 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated prequalification, missing-criterion, normalized-criterion, invalid-score, invalid-quorum, Markdown, and SVG artifacts.
  • npm run video -> regenerated reports/demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index.js, test.js, and demo.js.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 44,989 bytes.
  • git diff --check and git diff --cached --check passed; only Windows line-ending normalization warnings appeared.
  • Focused sensitive scan returned no payout, credential, or token strings.
  • Expanded private-term scan only matched safety-boundary wording in README/requirements/acceptance/report; generated packets did not expose private data.
  • GitHub PR merge state after push: CLEAN; no checks are reported for this branch.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in cd790ec:

  • Added a regression for rejected applicants whose rejectionReasons list contains only blank/whitespace text.
  • Applicant rejection reasons are now normalized to trimmed non-empty strings before they count as applicant-facing explanation evidence.
  • Blank rejection reasons now hold the applicant with missing-rejection-reason and route remediation to publish-rejection-reasons-and-appeal-window.
  • Added reports/blank-rejection-reason-packet.json and refreshed README, requirements map, acceptance notes, Markdown/SVG/MP4 reviewer evidence.

Why this matters:

  • A non-empty list is not the same as a meaningful rejection reason. Without this, a sponsor could publish a reject-with-audit packet with no applicant-facing explanation beyond whitespace.
  • This keeps PR Add challenge prequalification fairness guard #423 focused on sponsor-side prequalification fairness and closes a concrete applicant-remediation gap in the active issue Scientific Bounty System #18 competition.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with reject-with-audit instead of hold-for-fairness-review.
  • npm test -> 21 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated prequalification/missing-criterion/normalized-criterion/invalid-score/invalid-quorum/blank-rejection-reason packets plus Markdown/SVG evidence.
  • npm run video -> regenerated reports/demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 44,989 bytes.
  • git diff --check and git diff --cached --check passed; Git only reported Windows line-ending normalization warnings.
  • Focused sensitive scan returned no payout, credential, or token strings.
  • GitHub PR merge state after push: CLEAN; no checks are reported for this branch.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in 71109b6:

  • Added a regression for malformed sponsor decision evidence where a high-scoring applicant had sponsorDecision: "waitlist" and was previously accepted.
  • Sponsor decisions now have to be explicit accept or reject values before they can affect solver access.
  • Invalid sponsor decisions hold the applicant with sponsor-decision-invalid, route high-priority remediation to publish-valid-sponsor-decision, and are represented in reports/invalid-sponsor-decision-packet.json.
  • README, requirements map, acceptance notes, Markdown/SVG evidence, and demo video text now surface the new guard.

Why this matters:

  • Prequalification fairness requires auditable sponsor decision evidence, not only high reviewer scores. A malformed decision value should never silently become an access-changing accept/reject outcome.
  • This keeps PR Add challenge prequalification fairness guard #423 focused on its distinct sponsor-side prequalification fairness slice while closing a concrete malformed-decision gap.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with accept-prequalified instead of hold-for-fairness-review.
  • npm test -> 22 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated prequalification/missing-criterion/normalized-criterion/invalid-score/invalid-quorum/invalid-sponsor-decision/blank-rejection-reason packets plus Markdown/SVG evidence.
  • npm run video -> regenerated reports/demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 46,713 bytes.
  • All generated JSON packets parsed successfully.
  • git diff --check and git diff --cached --check passed; Git only reported Windows line-ending normalization warnings before staging.
  • Focused sensitive scan returned no payout, credential, or token strings.
  • GitHub PR merge state after push: CLEAN; no checks are reported for this branch.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in 3647ad0:

  • Added a regression for prequalification applicants whose identity evidence is blank or missing.
  • Missing applicant identities now hold the applicant for fairness review with missing-applicant-identity before any sponsor access decision can take effect.
  • Generated output now uses a stable unidentified-applicant identifier instead of leaking blank identity strings into decision IDs, audit digests, or remediation action IDs.
  • Remediation routes to complete-prequalification-evidence, and the new reports/missing-applicant-identity-packet.json artifact makes the hold visible to reviewers.
  • README, requirements map, acceptance notes, Markdown/SVG evidence, and demo video text now explicitly cover complete applicant identity evidence.

Why this matters:

  • Prequalification fairness depends on every accept/reject decision being tied to an auditable applicant identity. Without this, malformed sponsor input could approve or reject an unidentified applicant while producing unusable remediation/audit records.
  • This keeps PR Add challenge prequalification fairness guard #423 focused on its distinct sponsor-side prequalification fairness slice and closes another concrete evidence-integrity gap in the active issue Scientific Bounty System #18 competition.

Validation refreshed locally:

  • Confirmed the first new regression failed before implementation with accept-prequalified instead of hold-for-fairness-review.
  • Confirmed the output-normalization regression failed before implementation when the generated packet did not contain unidentified-applicant.
  • npm test -> 23 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated 8 JSON packets plus Markdown/SVG evidence, including missing-applicant-identity-packet.json.
  • npm run check -> test, demo, and video generation passed.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 45,050 bytes.
  • All generated JSON packets parsed successfully.
  • git diff --check and git diff --cached --check passed; only Windows line-ending normalization warnings appeared before staging.
  • Focused sensitive scan returned no payout, credential, or token strings.
  • GitHub PR merge state after push: CLEAN; no checks are reported for this branch.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Hardening update pushed in f24fb2e for the review-list evidence path.

What changed:

  • Added a regression for sparse prequalification packets where round.reviews is missing.
  • The guard now holds the affected applicant for missing-review-list instead of crashing during review grouping.
  • Missing review-list evidence routes to complete-prequalification-evidence with high priority before sponsor acceptance can take effect.
  • The demo now emits reports/missing-review-list-packet.json and includes it in the Markdown report.

Validation refreshed locally:

  • npm test -> 24 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated 9 JSON packets plus Markdown/SVG evidence.
  • npm run video -> regenerated reports/demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • ffprobe -> H.264, 1280x720, 30 fps, 4.0s, 44,663 bytes.
  • All 9 generated JSON packets parsed successfully.
  • git diff --check and git diff --cached --check passed; only Windows line-ending normalization warnings appeared before staging.
  • Focused sensitive scan returned no payout, credential, or token strings.

PR state after push: open, CLEAN, head f24fb2e; no checks are reported for this branch.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Hardening update pushed in 4cafdff for sparse published-criteria evidence.

What changed:

  • Added a regression for prequalification packets where round.criteria is missing.
  • The guard now holds the affected applicant for missing-published-criteria-list instead of crashing during criteria/scoring/digest evaluation.
  • Missing criteria-list evidence routes to publish-complete-screening-criteria with high priority before sponsor acceptance can take effect.
  • The demo now emits reports/missing-criteria-list-packet.json and includes it in the Markdown report.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with TypeError: Cannot read properties of undefined (reading 'map') at publicCriteriaIds(round).
  • npm test -> 25 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated 10 JSON packets plus Markdown/SVG evidence.
  • npm run video -> regenerated reports/demo.mp4.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index, test, and demo.
  • ffprobe -> H.264, 1280x720, 30 fps, 4.0s, 120 frames.
  • All 10 generated JSON packets parsed successfully.
  • git diff --check and git diff --cached --check passed.
  • Focused restricted-string scan returned no matches.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Follow-up competitive hardening pass for the challenge prequalification fairness guard.

What changed in 92ba303:

  • Added a regression for sparse prequalification packets where round.applicants is missing.
  • The guard now emits a high-priority missing-applicant-list fairness hold instead of crashing at applicant iteration.
  • Added reports/missing-applicant-list-packet.json and refreshed the Markdown/README/requirements evidence so reviewers can inspect the new path.

Validation refreshed locally:

  • Confirmed the new regression failed before implementation with TypeError: Cannot read properties of undefined (reading 'map').
  • npm test -> 26 challenge prequalification fairness tests passed.
  • npm run demo -> regenerated 11 JSON packets plus Markdown/SVG evidence.
  • npm run check -> test, demo, and video generation passed.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 44,663 bytes.
  • All generated JSON packets parsed successfully.
  • git diff --check and git diff --cached --check passed; focused restricted-string scan returned no matches.
  • GitHub PR state after push: OPEN, merge state CLEAN, head 92ba303; no checks are reported for this branch.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 30, 2026

Hardening update pushed in 303f18f for duplicate applicant identity evidence.

What changed:

  • Added a red regression for two prequalification rows that normalize to the same applicant ID (applicant-duplicate and applicant-duplicate).
  • The guard now holds both rows with duplicate-applicant-identity before sponsor accept/reject decisions can change solver access.
  • Added reports/duplicate-applicant-identity-packet.json and refreshed Markdown evidence.

Fresh validation:

  • npm test -> 27 challenge prequalification fairness tests passed
  • npm run demo -> regenerated 12 JSON packets plus Markdown/SVG evidence
  • npm run video -> regenerated reports/demo.mp4
  • npm run check -> test, demo, and video generation passed
  • all 12 generated JSON packets parsed successfully
  • ffprobe -> H.264, 1280x720, 30 fps, 4.0s, 120 frames
  • git diff --check and git diff --cached --check passed
  • focused scan found only existing benign payout-scope wording in docs/demo text, no secrets or tokens
  • GitHub PR state after push: OPEN, merge state CLEAN, head 303f18f; no checks are reported for this branch

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented May 31, 2026

Hardening update pushed in d154a75: malformed applicant entries such as applicants: [null] now produce a high-priority malformed-applicant-entry fairness hold for unidentified-applicant instead of crashing prequalification packet generation.

Validation refreshed locally:

  • red regression first reproduced TypeError: Cannot read properties of null (reading 'id')
  • npm test -> challenge prequalification fairness tests passed (28)
  • npm run demo, npm run video, and npm run check
  • parsed all 13 generated JSON packets, including malformed-applicant-entry-packet.json
  • ffprobe verified reports/demo.mp4 as H.264 1280x720, 30fps, 4.0s, 44,663 bytes
  • git diff --check, git diff --cached --check, staged allowlist check, and restricted-string scan passed

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented Jun 1, 2026

Hardening update pushed in 530f27f for malformed reviewer evidence.

What changed:

  • Added a regression for sparse prequalification packets with reviews: [null].
  • Malformed review entries now emit a high-priority malformed-review-entry fairness hold instead of crashing before packet generation.
  • Valid reviewer rows remain countable, so the packet preserves usable quorum evidence while blocking malformed evidence from taking effect.
  • Added reports/malformed-review-entry-packet.json and refreshed the generated Markdown report/body evidence.

Fresh validation:

  • Red regression first reproduced TypeError: Cannot read properties of null (reading 'applicantId') at reviewApplicantIdFor.
  • npm test -> 29 challenge prequalification fairness tests passed.
  • npm run check -> test, demo, and video generation passed.
  • node --check passed for index.js, test.js, and demo.js.
  • Parsed all 14 generated JSON packets.
  • ffprobe verified reports/demo.mp4 as H.264, 1280x720, 30 fps, 4.0s, 44,663 bytes.
  • git diff --check, git diff --cached --check, staged allowlist check, and focused sensitive scan passed.
  • GitHub PR state after push: OPEN / CLEAN; no checks are reported for this branch.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented Jun 1, 2026

Pushed a focused hardening commit for malformed published criteria evidence: b2c3a66. A prequalification packet with criteria: [null] previously crashed at publicCriteriaIds; it now emits a high-priority malformed-published-criterion-entry fairness hold and routes remediation to publish-complete-screening-criteria before sponsor decisions can take effect. Verified npm test (30), npm run check, node --check, 15 JSON packet parses, ffprobe on the H.264 1280x720 demo video, diff checks, and a focused restricted-string scan.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented Jun 1, 2026

Fresh hardening pushed in 4c74a9a: missing or blank challenge IDs now hold prequalification decisions with missing-challenge-identity, use unidentified-challenge in reviewer/audit packets, and route high-priority complete-challenge-identity-evidence before solver access can change.

Verification passed: red regression first reproduced the blank challenge ID release, then npm test (31), npm run check, node --check, 16 JSON parses including reports/missing-challenge-identity-packet.json, ffprobe H.264 1280x720 30fps 4.0s MP4 metadata, diff checks, staged allowlist check, and focused restricted-string scans.

@KoiosSG
Copy link
Copy Markdown
Author

KoiosSG commented Jun 2, 2026

Pushed focused hardening commit 5e5c287 for malformed top-level prequalification packets.

A null prequalification round previously crashed at challengeIdFor before reviewer packet generation. It now holds with malformed-prequalification-round, uses unidentified-challenge / unidentified-applicant, routes high-priority complete-prequalification-evidence, and emits reports/malformed-prequalification-round-packet.json.

Verification passed: red regression captured first, npm test (32), node --check, npm run demo, npm run video, npm run check, 17 JSON packet parses, ffprobe H.264 1280x720 30fps 4.0s / 120 frames / 46,164 bytes, diff checks, staged allowlist check, and focused payout/credential scan. PR is OPEN / CLEAN at head 5e5c287; no checks are reported for this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant