Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
5c855a4
Add challenge prequalification fairness guard
KoiosSG May 28, 2026
25af825
Harden prequalification conflict scoring
KoiosSG May 28, 2026
9033fef
Harden prequalification appeal windows
KoiosSG May 28, 2026
950e3e8
Validate prequalification rubric weights
KoiosSG May 28, 2026
2ff5d97
Harden prequalification rubric weights
KoiosSG May 29, 2026
7bb2370
Handle incomplete prequalification scores
KoiosSG May 29, 2026
d70a8c8
Deduplicate reviewer score evidence
KoiosSG May 29, 2026
40955b6
Validate prequalification appeal timestamps
KoiosSG May 29, 2026
2a6c286
Validate prequalification pass thresholds
KoiosSG May 29, 2026
3d7c8d7
Handle missing rejection reason lists
KoiosSG May 29, 2026
9be11bc
Validate unique prequalification criteria
KoiosSG May 29, 2026
ef304da
Require reviewer identity for quorum
KoiosSG May 30, 2026
b803d03
Validate complete prequalification criteria identifiers
KoiosSG May 30, 2026
1cb4160
Validate reviewer score evidence range
KoiosSG May 30, 2026
a4d96c0
Validate prequalification reviewer quorum
KoiosSG May 30, 2026
4b0283e
Normalize prequalification criterion identifiers
KoiosSG May 30, 2026
cd790ec
Reject blank rejection reasons
KoiosSG May 30, 2026
71109b6
Harden prequalification sponsor decisions
KoiosSG May 30, 2026
3647ad0
Harden prequalification applicant identity
KoiosSG May 30, 2026
f24fb2e
Harden prequalification review list evidence
KoiosSG May 30, 2026
4cafdff
Harden missing prequalification criteria evidence
KoiosSG May 30, 2026
92ba303
Harden missing applicant list prequalification packets
KoiosSG May 30, 2026
303f18f
Harden prequalification applicant identity guard
KoiosSG May 30, 2026
d154a75
Harden malformed applicant entries
KoiosSG May 31, 2026
530f27f
Harden malformed review entries
KoiosSG Jun 1, 2026
b2c3a66
Harden malformed criteria entries
KoiosSG Jun 1, 2026
4c74a9a
Harden missing challenge identity holds
KoiosSG Jun 1, 2026
5e5c287
Harden malformed prequalification round packets
KoiosSG Jun 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions challenge-prequalification-fairness-guard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Challenge Prequalification Fairness Guard

This module adds a focused Scientific Bounty System slice for SCIBASE issue #18. It evaluates sponsor-side prequalification rounds before solver teams are accepted or rejected from a challenge.

The guard checks well-formed prequalification packets, challenge identity, published screening criteria, complete criteria-list evidence, well-formed criteria entries, complete and unique criterion identifiers after trimming, valid criterion weight values and totals, valid pass thresholds, valid reviewer quorum requirements, valid sponsor accept/reject decisions, complete applicant-list and unique applicant identity evidence after trimming, well-formed applicant entries, complete round-level review-list evidence, well-formed review entries, complete reviewer score evidence, valid finite 0-100 reviewer score values, weighted threshold consistency, anonymous-screening requirements, reviewer conflicts, distinct reviewer quorum, missing reviewer identity evidence, duplicate reviewer score evidence, missing, empty, or blank rejection reason evidence, parseable appeal windows, and audit evidence. Conflicted reviewer scores are excluded from threshold scoring while the conflict remains auditable, repeated reviewer identities are deduplicated before quorum or threshold scoring, and malformed top-level packets, missing challenge identities, missing applicant lists, malformed applicant entries, missing or duplicate applicant identities, missing criteria lists, malformed criteria entries, missing review lists, malformed review entries, missing reviewer identities, invalid sponsor decisions, or invalid score values are excluded from taking effect until the evidence is completed. Unfair or incomplete screening decisions are held for remediation before challenge access changes.

## Run

```bash
npm test
npm run demo
npm run video
npm run check
```

## Outputs

- `reports/prequalification-fairness-packet.json`
- `reports/missing-criterion-id-packet.json`
- `reports/normalized-criterion-id-packet.json`
- `reports/invalid-reviewer-score-packet.json`
- `reports/invalid-reviewer-quorum-packet.json`
- `reports/invalid-sponsor-decision-packet.json`
- `reports/missing-applicant-identity-packet.json`
- `reports/duplicate-applicant-identity-packet.json`
- `reports/missing-review-list-packet.json`
- `reports/malformed-review-entry-packet.json`
- `reports/missing-criteria-list-packet.json`
- `reports/malformed-criterion-entry-packet.json`
- `reports/missing-applicant-list-packet.json`
- `reports/malformed-prequalification-round-packet.json`
- `reports/missing-challenge-identity-packet.json`
- `reports/malformed-applicant-entry-packet.json`
- `reports/blank-rejection-reason-packet.json`
- `reports/prequalification-fairness-report.md`
- `reports/summary.svg`
- `reports/demo.mp4`

All data is synthetic. The module does not call payment processors, identity providers, private workspaces, sponsor systems, solver accounts, or external APIs.
45 changes: 45 additions & 0 deletions challenge-prequalification-fairness-guard/acceptance-notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Acceptance Notes

This #18 slice focuses specifically on fair sponsor-side prequalification before solvers enter or are rejected from a scientific challenge.

It is not:

- a broad scientific bounty marketplace module
- a general challenge intake compliance gate
- a submission workspace privacy or data-room access guard
- an arbitration scoring or payout eligibility ledger
- a clarification freeze, benchmark leakage, evaluator calibration, or reviewer workload guard

Validation coverage:

- eligible applicants are accepted when published criteria, quorum, and weighted thresholds are satisfied
- anonymous-screening leaks hold a candidate for fairness review
- inconsistent threshold decisions are held before rejection is published
- conflicted reviewer participation and missing rejection reasons remain auditable
- conflicted reviewer scores are excluded from weighted threshold evidence
- unpublished screening criteria are blocked before results are published
- invalid appeal-window timestamps hold rejected applicants before rejection packets are published
- invalid individual criterion weights are held even when the total still sums to 100
- duplicate published criterion IDs are held before ambiguous rubric evidence can drive acceptance or rejection
- whitespace-variant published criterion IDs such as `domain-fit` and ` domain-fit ` are treated as duplicates before ambiguous rubric evidence can drive acceptance or rejection
- missing or blank published criterion IDs are held before unauditable rubric evidence can drive acceptance or rejection
- invalid pass thresholds are held before sponsor accept/reject decisions can take effect
- invalid reviewer quorum requirements are held before sponsor accept/reject decisions can take effect
- invalid sponsor decision values are held before malformed accept/reject evidence can change solver access
- missing or blank applicant identities are held before anonymous or malformed applicant rows can change solver access
- duplicate applicant identities after trimming are held before conflicting prequalification rows can change solver access
- malformed applicant entries such as `null` are held for evidence completion instead of crashing sparse prequalification packets
- invalid reviewer score values outside the finite 0-100 range are held before malformed scoring evidence can drive acceptance or rejection
- missing rejection reason lists are normalized to an auditable fairness hold instead of crashing the prequalification packet
- blank rejection reason text is normalized away and held as missing applicant-facing rejection evidence
- incomplete reviewer score evidence is held for completion without crashing the prequalification packet
- missing review lists are held for evidence completion instead of crashing sparse prequalification packets
- malformed review entries such as `null` are held for evidence completion instead of crashing sparse prequalification packets
- missing published criteria lists are held for evidence completion instead of crashing sparse prequalification packets
- malformed published criteria entries such as `null` are held for evidence completion instead of crashing sparse prequalification packets
- missing applicant lists are held for evidence completion instead of crashing sparse prequalification packets
- malformed top-level prequalification packets such as `null` are held for evidence completion instead of crashing before reviewer packets can be generated
- missing or blank challenge identities are held before solver access decisions can detach from a specific challenge audit trail
- duplicate reviewer score evidence is held and deduplicated before quorum or weighted threshold scoring
- missing or blank reviewer identities are held and excluded from reviewer quorum until evidence is completed
- audit digests are deterministic and private-data free
Loading