diff --git a/challenge-prequalification-fairness-guard/README.md b/challenge-prequalification-fairness-guard/README.md new file mode 100644 index 00000000..82682f87 --- /dev/null +++ b/challenge-prequalification-fairness-guard/README.md @@ -0,0 +1,39 @@ +# Challenge Prequalification Fairness Guard + +This module adds a focused Scientific Bounty System slice for SCIBASE issue #18. It evaluates sponsor-side prequalification rounds before solver teams are accepted or rejected from a challenge. + +The guard checks well-formed prequalification packets, challenge identity, published screening criteria, complete criteria-list evidence, well-formed criteria entries, complete and unique criterion identifiers after trimming, valid criterion weight values and totals, valid pass thresholds, valid reviewer quorum requirements, valid sponsor accept/reject decisions, complete applicant-list and unique applicant identity evidence after trimming, well-formed applicant entries, complete round-level review-list evidence, well-formed review entries, complete reviewer score evidence, valid finite 0-100 reviewer score values, weighted threshold consistency, anonymous-screening requirements, reviewer conflicts, distinct reviewer quorum, missing reviewer identity evidence, duplicate reviewer score evidence, missing, empty, or blank rejection reason evidence, parseable appeal windows, and audit evidence. Conflicted reviewer scores are excluded from threshold scoring while the conflict remains auditable, repeated reviewer identities are deduplicated before quorum or threshold scoring, and malformed top-level packets, missing challenge identities, missing applicant lists, malformed applicant entries, missing or duplicate applicant identities, missing criteria lists, malformed criteria entries, missing review lists, malformed review entries, missing reviewer identities, invalid sponsor decisions, or invalid score values are excluded from taking effect until the evidence is completed. Unfair or incomplete screening decisions are held for remediation before challenge access changes. + +## Run + +```bash +npm test +npm run demo +npm run video +npm run check +``` + +## Outputs + +- `reports/prequalification-fairness-packet.json` +- `reports/missing-criterion-id-packet.json` +- `reports/normalized-criterion-id-packet.json` +- `reports/invalid-reviewer-score-packet.json` +- `reports/invalid-reviewer-quorum-packet.json` +- `reports/invalid-sponsor-decision-packet.json` +- `reports/missing-applicant-identity-packet.json` +- `reports/duplicate-applicant-identity-packet.json` +- `reports/missing-review-list-packet.json` +- `reports/malformed-review-entry-packet.json` +- `reports/missing-criteria-list-packet.json` +- `reports/malformed-criterion-entry-packet.json` +- `reports/missing-applicant-list-packet.json` +- `reports/malformed-prequalification-round-packet.json` +- `reports/missing-challenge-identity-packet.json` +- `reports/malformed-applicant-entry-packet.json` +- `reports/blank-rejection-reason-packet.json` +- `reports/prequalification-fairness-report.md` +- `reports/summary.svg` +- `reports/demo.mp4` + +All data is synthetic. The module does not call payment processors, identity providers, private workspaces, sponsor systems, solver accounts, or external APIs. diff --git a/challenge-prequalification-fairness-guard/acceptance-notes.md b/challenge-prequalification-fairness-guard/acceptance-notes.md new file mode 100644 index 00000000..1f77aa11 --- /dev/null +++ b/challenge-prequalification-fairness-guard/acceptance-notes.md @@ -0,0 +1,45 @@ +# Acceptance Notes + +This #18 slice focuses specifically on fair sponsor-side prequalification before solvers enter or are rejected from a scientific challenge. + +It is not: + +- a broad scientific bounty marketplace module +- a general challenge intake compliance gate +- a submission workspace privacy or data-room access guard +- an arbitration scoring or payout eligibility ledger +- a clarification freeze, benchmark leakage, evaluator calibration, or reviewer workload guard + +Validation coverage: + +- eligible applicants are accepted when published criteria, quorum, and weighted thresholds are satisfied +- anonymous-screening leaks hold a candidate for fairness review +- inconsistent threshold decisions are held before rejection is published +- conflicted reviewer participation and missing rejection reasons remain auditable +- conflicted reviewer scores are excluded from weighted threshold evidence +- unpublished screening criteria are blocked before results are published +- invalid appeal-window timestamps hold rejected applicants before rejection packets are published +- invalid individual criterion weights are held even when the total still sums to 100 +- duplicate published criterion IDs are held before ambiguous rubric evidence can drive acceptance or rejection +- whitespace-variant published criterion IDs such as `domain-fit` and ` domain-fit ` are treated as duplicates before ambiguous rubric evidence can drive acceptance or rejection +- missing or blank published criterion IDs are held before unauditable rubric evidence can drive acceptance or rejection +- invalid pass thresholds are held before sponsor accept/reject decisions can take effect +- invalid reviewer quorum requirements are held before sponsor accept/reject decisions can take effect +- invalid sponsor decision values are held before malformed accept/reject evidence can change solver access +- missing or blank applicant identities are held before anonymous or malformed applicant rows can change solver access +- duplicate applicant identities after trimming are held before conflicting prequalification rows can change solver access +- malformed applicant entries such as `null` are held for evidence completion instead of crashing sparse prequalification packets +- invalid reviewer score values outside the finite 0-100 range are held before malformed scoring evidence can drive acceptance or rejection +- missing rejection reason lists are normalized to an auditable fairness hold instead of crashing the prequalification packet +- blank rejection reason text is normalized away and held as missing applicant-facing rejection evidence +- incomplete reviewer score evidence is held for completion without crashing the prequalification packet +- missing review lists are held for evidence completion instead of crashing sparse prequalification packets +- malformed review entries such as `null` are held for evidence completion instead of crashing sparse prequalification packets +- missing published criteria lists are held for evidence completion instead of crashing sparse prequalification packets +- malformed published criteria entries such as `null` are held for evidence completion instead of crashing sparse prequalification packets +- missing applicant lists are held for evidence completion instead of crashing sparse prequalification packets +- malformed top-level prequalification packets such as `null` are held for evidence completion instead of crashing before reviewer packets can be generated +- missing or blank challenge identities are held before solver access decisions can detach from a specific challenge audit trail +- duplicate reviewer score evidence is held and deduplicated before quorum or weighted threshold scoring +- missing or blank reviewer identities are held and excluded from reviewer quorum until evidence is completed +- audit digests are deterministic and private-data free diff --git a/challenge-prequalification-fairness-guard/demo.js b/challenge-prequalification-fairness-guard/demo.js new file mode 100644 index 00000000..0e1e7a40 --- /dev/null +++ b/challenge-prequalification-fairness-guard/demo.js @@ -0,0 +1,850 @@ +const fs = require('fs'); +const path = require('path'); +const { evaluatePrequalificationRound, buildSampleRound } = require('./index'); + +const reportsDir = path.join(__dirname, 'reports'); +fs.mkdirSync(reportsDir, { recursive: true }); + +const result = evaluatePrequalificationRound(buildSampleRound()); +const missingCriterionResult = evaluatePrequalificationRound(buildMissingCriterionIdRound()); +const normalizedCriterionResult = evaluatePrequalificationRound(buildNormalizedCriterionIdRound()); +const invalidScoreResult = evaluatePrequalificationRound(buildInvalidReviewerScoreRound()); +const invalidQuorumResult = evaluatePrequalificationRound(buildInvalidReviewerQuorumRound()); +const invalidSponsorDecisionResult = evaluatePrequalificationRound(buildInvalidSponsorDecisionRound()); +const missingApplicantIdentityResult = evaluatePrequalificationRound(buildMissingApplicantIdentityRound()); +const duplicateApplicantIdentityResult = evaluatePrequalificationRound(buildDuplicateApplicantIdentityRound()); +const missingReviewListResult = evaluatePrequalificationRound(buildMissingReviewListRound()); +const malformedReviewEntryResult = evaluatePrequalificationRound(buildMalformedReviewEntryRound()); +const missingCriteriaListResult = evaluatePrequalificationRound(buildMissingCriteriaListRound()); +const malformedCriterionEntryResult = evaluatePrequalificationRound(buildMalformedCriterionEntryRound()); +const missingApplicantListResult = evaluatePrequalificationRound(buildMissingApplicantListRound()); +const malformedPrequalificationRoundResult = evaluatePrequalificationRound(null); +const missingChallengeIdentityResult = evaluatePrequalificationRound(buildMissingChallengeIdentityRound()); +const malformedApplicantEntryResult = evaluatePrequalificationRound(buildMalformedApplicantEntryRound()); +const blankRejectionReasonResult = evaluatePrequalificationRound(buildBlankRejectionReasonRound()); + +const packetPath = path.join(reportsDir, 'prequalification-fairness-packet.json'); +const missingCriterionPacketPath = path.join(reportsDir, 'missing-criterion-id-packet.json'); +const normalizedCriterionPacketPath = path.join(reportsDir, 'normalized-criterion-id-packet.json'); +const invalidScorePacketPath = path.join(reportsDir, 'invalid-reviewer-score-packet.json'); +const invalidQuorumPacketPath = path.join(reportsDir, 'invalid-reviewer-quorum-packet.json'); +const invalidSponsorDecisionPacketPath = path.join( + reportsDir, + 'invalid-sponsor-decision-packet.json' +); +const missingApplicantIdentityPacketPath = path.join( + reportsDir, + 'missing-applicant-identity-packet.json' +); +const duplicateApplicantIdentityPacketPath = path.join( + reportsDir, + 'duplicate-applicant-identity-packet.json' +); +const missingReviewListPacketPath = path.join(reportsDir, 'missing-review-list-packet.json'); +const malformedReviewEntryPacketPath = path.join( + reportsDir, + 'malformed-review-entry-packet.json' +); +const missingCriteriaListPacketPath = path.join(reportsDir, 'missing-criteria-list-packet.json'); +const malformedCriterionEntryPacketPath = path.join( + reportsDir, + 'malformed-criterion-entry-packet.json' +); +const missingApplicantListPacketPath = path.join(reportsDir, 'missing-applicant-list-packet.json'); +const malformedPrequalificationRoundPacketPath = path.join( + reportsDir, + 'malformed-prequalification-round-packet.json' +); +const missingChallengeIdentityPacketPath = path.join( + reportsDir, + 'missing-challenge-identity-packet.json' +); +const malformedApplicantEntryPacketPath = path.join( + reportsDir, + 'malformed-applicant-entry-packet.json' +); +const blankRejectionReasonPacketPath = path.join(reportsDir, 'blank-rejection-reason-packet.json'); +const reportPath = path.join(reportsDir, 'prequalification-fairness-report.md'); +const svgPath = path.join(reportsDir, 'summary.svg'); + +fs.writeFileSync(packetPath, `${JSON.stringify(result, null, 2)}\n`); +fs.writeFileSync(missingCriterionPacketPath, `${JSON.stringify(missingCriterionResult, null, 2)}\n`); +fs.writeFileSync(normalizedCriterionPacketPath, `${JSON.stringify(normalizedCriterionResult, null, 2)}\n`); +fs.writeFileSync(invalidScorePacketPath, `${JSON.stringify(invalidScoreResult, null, 2)}\n`); +fs.writeFileSync(invalidQuorumPacketPath, `${JSON.stringify(invalidQuorumResult, null, 2)}\n`); +fs.writeFileSync( + invalidSponsorDecisionPacketPath, + `${JSON.stringify(invalidSponsorDecisionResult, null, 2)}\n` +); +fs.writeFileSync( + missingApplicantIdentityPacketPath, + `${JSON.stringify(missingApplicantIdentityResult, null, 2)}\n` +); +fs.writeFileSync( + duplicateApplicantIdentityPacketPath, + `${JSON.stringify(duplicateApplicantIdentityResult, null, 2)}\n` +); +fs.writeFileSync(missingReviewListPacketPath, `${JSON.stringify(missingReviewListResult, null, 2)}\n`); +fs.writeFileSync( + malformedReviewEntryPacketPath, + `${JSON.stringify(malformedReviewEntryResult, null, 2)}\n` +); +fs.writeFileSync( + missingCriteriaListPacketPath, + `${JSON.stringify(missingCriteriaListResult, null, 2)}\n` +); +fs.writeFileSync( + malformedCriterionEntryPacketPath, + `${JSON.stringify(malformedCriterionEntryResult, null, 2)}\n` +); +fs.writeFileSync( + missingApplicantListPacketPath, + `${JSON.stringify(missingApplicantListResult, null, 2)}\n` +); +fs.writeFileSync( + malformedPrequalificationRoundPacketPath, + `${JSON.stringify(malformedPrequalificationRoundResult, null, 2)}\n` +); +fs.writeFileSync( + missingChallengeIdentityPacketPath, + `${JSON.stringify(missingChallengeIdentityResult, null, 2)}\n` +); +fs.writeFileSync( + malformedApplicantEntryPacketPath, + `${JSON.stringify(malformedApplicantEntryResult, null, 2)}\n` +); +fs.writeFileSync(blankRejectionReasonPacketPath, `${JSON.stringify(blankRejectionReasonResult, null, 2)}\n`); + +const decisions = result.decisions + .map( + (decision) => + `- ${decision.applicantId}: ${decision.decision}, score ${decision.weightedScore}, reasons: ${ + decision.reasons.length > 0 ? decision.reasons.join(', ') : 'none' + }` + ) + .join('\n'); + +const actions = result.remediationActions + .map((action) => `- ${action.id}: ${action.action} (${action.priority})`) + .join('\n'); + +const markdown = `# Challenge Prequalification Fairness Guard + +Challenge: ${result.challengeId} +Generated: ${result.generatedAt} + +## Summary + +- Accepted applicants: ${result.summary.accepted} +- Held for fairness review: ${result.summary.held} +- Rejected with audit trail: ${result.summary.rejectedWithAudit} +- Remediation actions: ${result.summary.remediationActions} +- Criteria digest: ${result.criteriaDigest} +- Audit digest: ${result.auditDigest} + +## Decisions + +${decisions} + +## Remediation Actions + +${actions} + +## Missing Criterion Identifier Packet + +- Applicant: ${missingCriterionResult.decisions[0].applicantId} +- Decision: ${missingCriterionResult.decisions[0].decision} +- Reasons: ${missingCriterionResult.decisions[0].reasons.join(', ')} +- Remediation: ${missingCriterionResult.remediationActions[0].action} +- Audit digest: ${missingCriterionResult.auditDigest} + +## Normalized Criterion Identifier Packet + +- Applicant: ${normalizedCriterionResult.decisions[0].applicantId} +- Decision: ${normalizedCriterionResult.decisions[0].decision} +- Reasons: ${normalizedCriterionResult.decisions[0].reasons.join(', ')} +- Remediation: ${normalizedCriterionResult.remediationActions[0].action} +- Audit digest: ${normalizedCriterionResult.auditDigest} + +## Invalid Reviewer Score Packet + +- Applicant: ${invalidScoreResult.decisions[0].applicantId} +- Decision: ${invalidScoreResult.decisions[0].decision} +- Reasons: ${invalidScoreResult.decisions[0].reasons.join(', ')} +- Remediation: ${invalidScoreResult.remediationActions[0].action} +- Audit digest: ${invalidScoreResult.auditDigest} + +## Invalid Reviewer Quorum Packet + +- Applicant: ${invalidQuorumResult.decisions[0].applicantId} +- Decision: ${invalidQuorumResult.decisions[0].decision} +- Reasons: ${invalidQuorumResult.decisions[0].reasons.join(', ')} +- Remediation: ${invalidQuorumResult.remediationActions[0].action} +- Audit digest: ${invalidQuorumResult.auditDigest} + +## Invalid Sponsor Decision Packet + +- Applicant: ${invalidSponsorDecisionResult.decisions[0].applicantId} +- Decision: ${invalidSponsorDecisionResult.decisions[0].decision} +- Reasons: ${invalidSponsorDecisionResult.decisions[0].reasons.join(', ')} +- Remediation: ${invalidSponsorDecisionResult.remediationActions[0].action} +- Audit digest: ${invalidSponsorDecisionResult.auditDigest} + +## Missing Applicant Identity Packet + +- Applicant: ${JSON.stringify(missingApplicantIdentityResult.decisions[0].applicantId)} +- Decision: ${missingApplicantIdentityResult.decisions[0].decision} +- Reasons: ${missingApplicantIdentityResult.decisions[0].reasons.join(', ')} +- Remediation: ${missingApplicantIdentityResult.remediationActions[0].action} +- Audit digest: ${missingApplicantIdentityResult.auditDigest} + +## Duplicate Applicant Identity Packet + +- Applicant: ${duplicateApplicantIdentityResult.decisions[0].applicantId} +- Decision: ${duplicateApplicantIdentityResult.decisions[0].decision} +- Reasons: ${duplicateApplicantIdentityResult.decisions[0].reasons.join(', ')} +- Remediation: ${duplicateApplicantIdentityResult.remediationActions[0].action} +- Audit digest: ${duplicateApplicantIdentityResult.auditDigest} + +## Missing Review List Packet + +- Applicant: ${missingReviewListResult.decisions[0].applicantId} +- Decision: ${missingReviewListResult.decisions[0].decision} +- Reasons: ${missingReviewListResult.decisions[0].reasons.join(', ')} +- Remediation: ${missingReviewListResult.remediationActions[0].action} +- Audit digest: ${missingReviewListResult.auditDigest} + +## Malformed Review Entry Packet + +- Applicant: ${malformedReviewEntryResult.decisions[0].applicantId} +- Decision: ${malformedReviewEntryResult.decisions[0].decision} +- Reviewers counted: ${malformedReviewEntryResult.decisions[0].reviewersCounted} +- Reasons: ${malformedReviewEntryResult.decisions[0].reasons.join(', ')} +- Remediation: ${malformedReviewEntryResult.remediationActions[0].action} +- Audit digest: ${malformedReviewEntryResult.auditDigest} + +## Missing Criteria List Packet + +- Applicant: ${missingCriteriaListResult.decisions[0].applicantId} +- Decision: ${missingCriteriaListResult.decisions[0].decision} +- Reasons: ${missingCriteriaListResult.decisions[0].reasons.join(', ')} +- Remediation: ${missingCriteriaListResult.remediationActions[0].action} +- Audit digest: ${missingCriteriaListResult.auditDigest} + +## Malformed Criterion Entry Packet + +- Applicant: ${malformedCriterionEntryResult.decisions[0].applicantId} +- Decision: ${malformedCriterionEntryResult.decisions[0].decision} +- Reasons: ${malformedCriterionEntryResult.decisions[0].reasons.join(', ')} +- Remediation: ${malformedCriterionEntryResult.remediationActions[0].action} +- Audit digest: ${malformedCriterionEntryResult.auditDigest} + +## Missing Applicant List Packet + +- Applicant: ${missingApplicantListResult.decisions[0].applicantId} +- Decision: ${missingApplicantListResult.decisions[0].decision} +- Reasons: ${missingApplicantListResult.decisions[0].reasons.join(', ')} +- Remediation: ${missingApplicantListResult.remediationActions[0].action} +- Audit digest: ${missingApplicantListResult.auditDigest} + +## Malformed Prequalification Round Packet + +- Challenge: ${malformedPrequalificationRoundResult.challengeId} +- Applicant: ${malformedPrequalificationRoundResult.decisions[0].applicantId} +- Decision: ${malformedPrequalificationRoundResult.decisions[0].decision} +- Reasons: ${malformedPrequalificationRoundResult.decisions[0].reasons.join(', ')} +- Remediation: ${malformedPrequalificationRoundResult.remediationActions[0].action} +- Audit digest: ${malformedPrequalificationRoundResult.auditDigest} + +## Missing Challenge Identity Packet + +- Challenge: ${missingChallengeIdentityResult.challengeId} +- Applicant: ${missingChallengeIdentityResult.decisions[0].applicantId} +- Decision: ${missingChallengeIdentityResult.decisions[0].decision} +- Reasons: ${missingChallengeIdentityResult.decisions[0].reasons.join(', ')} +- Remediation: ${missingChallengeIdentityResult.remediationActions[0].action} +- Audit digest: ${missingChallengeIdentityResult.auditDigest} + +## Malformed Applicant Entry Packet + +- Applicant: ${malformedApplicantEntryResult.decisions[0].applicantId} +- Decision: ${malformedApplicantEntryResult.decisions[0].decision} +- Reasons: ${malformedApplicantEntryResult.decisions[0].reasons.join(', ')} +- Remediation: ${malformedApplicantEntryResult.remediationActions[0].action} +- Audit digest: ${malformedApplicantEntryResult.auditDigest} + +## Blank Rejection Reason Packet + +- Applicant: ${blankRejectionReasonResult.decisions[0].applicantId} +- Decision: ${blankRejectionReasonResult.decisions[0].decision} +- Reasons: ${blankRejectionReasonResult.decisions[0].reasons.join(', ')} +- Remediation: ${blankRejectionReasonResult.remediationActions[0].action} +- Audit digest: ${blankRejectionReasonResult.auditDigest} + +## Safety + +All fixtures are synthetic. The guard does not call payment processors, identity providers, private workspaces, sponsor systems, or external APIs. +`; + +fs.writeFileSync(reportPath, markdown); + +const svg = ` + + + Challenge Prequalification Fairness Guard + Accepted applicants: ${result.summary.accepted} + Held for fairness review: ${result.summary.held} + Remediation actions: ${result.summary.remediationActions} + Checks: criteria, applicant identities, sponsor decisions, valid scores + thresholds, anonymity, conflicts, appeals + Unfair screening decisions are held before applicants are accepted or rejected. + ${result.auditDigest} + +`; + +fs.writeFileSync(svgPath, svg); + +console.log(`Wrote ${path.relative(__dirname, packetPath)}`); +console.log(`Wrote ${path.relative(__dirname, missingCriterionPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, normalizedCriterionPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, invalidScorePacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, invalidQuorumPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, invalidSponsorDecisionPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, missingApplicantIdentityPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, duplicateApplicantIdentityPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, missingReviewListPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, malformedReviewEntryPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, missingCriteriaListPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, malformedCriterionEntryPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, missingApplicantListPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, malformedPrequalificationRoundPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, missingChallengeIdentityPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, malformedApplicantEntryPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, blankRejectionReasonPacketPath)}`); +console.log(`Wrote ${path.relative(__dirname, reportPath)}`); +console.log(`Wrote ${path.relative(__dirname, svgPath)}`); +console.log(`Accepted applicants: ${result.summary.accepted}`); +console.log(`Held applicants: ${result.summary.held}`); + +function buildMissingCriterionIdRound() { + const round = buildSampleRound(); + round.criteria = [ + { + id: 'domain-fit', + label: 'Domain fit for the scientific challenge', + weight: 50 + }, + { + id: ' ', + label: 'Blank sponsor rubric identifier', + weight: 25 + }, + { + id: 'safety-plan', + label: 'Risk, NDA, and responsible-use plan', + weight: 25 + } + ]; + round.applicants = [ + { + id: 'applicant-missing-criterion-id', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-missing-criterion-id', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + ' ': 95, + 'safety-plan': 92 + } + }, + { + applicantId: 'applicant-missing-criterion-id', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 96, + ' ': 94, + 'safety-plan': 94 + } + } + ]; + return round; +} + +function buildNormalizedCriterionIdRound() { + const round = buildSampleRound(); + round.criteria = [ + { + id: 'domain-fit', + label: 'Domain fit for the scientific challenge', + weight: 50 + }, + { + id: ' domain-fit ', + label: 'Whitespace-padded duplicate sponsor rubric identifier', + weight: 25 + }, + { + id: 'safety-plan', + label: 'Risk, NDA, and responsible-use plan', + weight: 25 + } + ]; + round.applicants = [ + { + id: 'applicant-normalized-criterion-id', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-normalized-criterion-id', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + ' domain-fit ': 95, + 'safety-plan': 92 + } + }, + { + applicantId: 'applicant-normalized-criterion-id', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 96, + ' domain-fit ': 94, + 'safety-plan': 94 + } + } + ]; + return round; +} + +function buildInvalidReviewerScoreRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-invalid-reviewer-score', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-invalid-reviewer-score', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 140, + 'data-readiness': 96, + 'safety-plan': 94 + } + }, + { + applicantId: 'applicant-invalid-reviewer-score', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 93 + } + } + ]; + return round; +} + +function buildInvalidReviewerQuorumRound() { + const round = buildSampleRound(); + round.minReviewers = 0; + round.applicants = [ + { + id: 'applicant-invalid-reviewer-quorum', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-invalid-reviewer-quorum', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 93 + } + } + ]; + return round; +} + +function buildInvalidSponsorDecisionRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-invalid-sponsor-decision', + sponsorDecision: 'waitlist', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-invalid-sponsor-decision', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 94 + } + }, + { + applicantId: 'applicant-invalid-sponsor-decision', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 93 + } + } + ]; + return round; +} + +function buildMissingApplicantIdentityRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: ' ', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: ' ', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 94 + } + }, + { + applicantId: ' ', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 93 + } + } + ]; + return round; +} + +function buildDuplicateApplicantIdentityRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: ' applicant-duplicate ', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + }, + { + id: 'applicant-duplicate', + sponsorDecision: 'reject', + rejectionReasons: ['duplicate entry should be resolved before screening'], + appealDueAt: '2026-06-04T08:00:00Z' + } + ]; + round.reviews = [ + { + applicantId: 'applicant-duplicate', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 94 + } + }, + { + applicantId: 'applicant-duplicate', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 93 + } + } + ]; + return round; +} + +function buildMissingReviewListRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-missing-review-list', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + delete round.reviews; + return round; +} + +function buildMalformedReviewEntryRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-malformed-review-entry', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + null, + { + applicantId: 'applicant-malformed-review-entry', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 93 + } + }, + { + applicantId: 'applicant-malformed-review-entry', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 91 + } + } + ]; + return round; +} + +function buildMissingCriteriaListRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-missing-criteria-list', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-missing-criteria-list', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 93 + } + }, + { + applicantId: 'applicant-missing-criteria-list', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 91 + } + } + ]; + delete round.criteria; + return round; +} + +function buildMalformedCriterionEntryRound() { + const round = buildSampleRound(); + round.criteria = [null]; + round.applicants = [ + { + id: 'applicant-malformed-criterion-entry', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + return round; +} + +function buildMissingApplicantListRound() { + const round = buildSampleRound(); + delete round.applicants; + return round; +} + +function buildMissingChallengeIdentityRound() { + const round = buildSampleRound(); + round.challengeId = ' '; + round.applicants = [ + { + id: 'applicant-missing-challenge-identity', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-missing-challenge-identity', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 93 + } + }, + { + applicantId: 'applicant-missing-challenge-identity', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 91 + } + } + ]; + return round; +} + +function buildMalformedApplicantEntryRound() { + const round = buildSampleRound(); + round.applicants = [null]; + round.reviews = []; + return round; +} + +function buildBlankRejectionReasonRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-blank-rejection-reason', + sponsorDecision: 'reject', + rejectionReasons: [' '], + appealDueAt: '2026-06-04T08:00:00Z' + } + ]; + round.reviews = [ + { + applicantId: 'applicant-blank-rejection-reason', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['insufficient validation plan'], + scores: { + 'domain-fit': 58, + 'data-readiness': 60, + 'safety-plan': 62 + } + }, + { + applicantId: 'applicant-blank-rejection-reason', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['insufficient validation plan'], + scores: { + 'domain-fit': 59, + 'data-readiness': 61, + 'safety-plan': 60 + } + } + ]; + return round; +} diff --git a/challenge-prequalification-fairness-guard/index.js b/challenge-prequalification-fairness-guard/index.js new file mode 100644 index 00000000..d2b273dc --- /dev/null +++ b/challenge-prequalification-fairness-guard/index.js @@ -0,0 +1,921 @@ +const crypto = require('crypto'); + +function stableStringify(value) { + if (Array.isArray(value)) { + return `[${value.map(stableStringify).join(',')}]`; + } + + if (value && typeof value === 'object') { + return `{${Object.keys(value) + .sort() + .map((key) => `${JSON.stringify(key)}:${stableStringify(value[key])}`) + .join(',')}}`; + } + + return JSON.stringify(value); +} + +function digest(value) { + return `sha256:${crypto.createHash('sha256').update(stableStringify(value)).digest('hex')}`; +} + +function groupBy(items, getKey) { + return items.reduce((groups, item) => { + const key = getKey(item); + if (!groups[key]) { + groups[key] = []; + } + groups[key].push(item); + return groups; + }, {}); +} + +function scoreForCriterion(reviews, criterionId) { + const scores = reviews + .map((review) => reviewScores(review)[criterionId]) + .filter((score) => isValidReviewerScore(score)); + + if (scores.length === 0) { + return null; + } + + return scores.reduce((total, score) => total + score, 0) / scores.length; +} + +function weightedScore(criteria, reviews) { + const weighted = criteria.reduce((total, criterion) => { + const average = scoreForCriterion(reviews, criterion.id); + if (average === null) { + return total; + } + return total + average * (criterion.weight / 100); + }, 0); + + return Math.round(weighted); +} + +function uniqueSorted(values) { + return Array.from(new Set(values)).sort(); +} + +function challengeIdFor(round) { + return typeof round.challengeId === 'string' ? round.challengeId.trim() : ''; +} + +function prequalificationRoundIsRecord(round) { + return Boolean(round && typeof round === 'object' && !Array.isArray(round)); +} + +function prequalificationRoundRecordFor(round) { + if (prequalificationRoundIsRecord(round)) { + return round; + } + + return { + challengeId: '', + generatedAt: null, + anonymousScreeningRequired: false, + minReviewers: null, + passThreshold: null, + criteria: null, + applicants: null, + reviews: null, + malformedPrequalificationRound: true + }; +} + +function outputChallengeIdFor(round) { + return challengeIdFor(round) || 'unidentified-challenge'; +} + +function challengeIdentityIsMissing(round) { + return !challengeIdFor(round); +} + +function criterionIdFor(criterion) { + return typeof criterion.id === 'string' ? criterion.id.trim() : criterion.id; +} + +function criterionIsRecord(criterion) { + return Boolean(criterion && typeof criterion === 'object' && !Array.isArray(criterion)); +} + +function criterionRecordFor(criterion) { + if (criterionIsRecord(criterion)) { + return criterion; + } + + return { + id: '', + weight: null, + malformedCriterionEntry: true + }; +} + +function duplicatePublishedCriterionIds(round) { + const criterionCounts = criteriaListFor(round).reduce((counts, criterion) => { + const criterionId = criterionIdFor(criterion); + counts[criterionId] = (counts[criterionId] || 0) + 1; + return counts; + }, Object.create(null)); + + return uniqueSorted( + Object.entries(criterionCounts) + .filter(([, count]) => count > 1) + .map(([criterionId]) => criterionId) + ); +} + +function hasMissingPublishedCriterionIds(round) { + return criteriaListFor(round).some( + (criterion) => typeof criterion.id !== 'string' || criterion.id.trim().length === 0 + ); +} + +function reviewIsRecord(review) { + return Boolean(review && typeof review === 'object' && !Array.isArray(review)); +} + +function reviewerIdFor(review) { + if (!reviewIsRecord(review)) { + return ''; + } + + return typeof review.reviewerId === 'string' ? review.reviewerId.trim() : ''; +} + +function applicantIsRecord(applicant) { + return Boolean(applicant && typeof applicant === 'object' && !Array.isArray(applicant)); +} + +function applicantRecordFor(applicant) { + if (applicantIsRecord(applicant)) { + return applicant; + } + + return { + id: '', + sponsorDecision: null, + rejectionReasons: [], + appealDueAt: null, + malformedApplicantEntry: true + }; +} + +function applicantIdFor(applicant) { + if (!applicantIsRecord(applicant)) { + return ''; + } + + return typeof applicant.id === 'string' ? applicant.id.trim() : ''; +} + +function applicantIdentityIsMissing(applicant) { + return !applicantIdFor(applicant); +} + +function duplicateApplicantIds(round) { + const applicantCounts = applicantListFor(round).reduce((counts, applicant) => { + const applicantId = applicantIdFor(applicant); + if (!applicantId) { + return counts; + } + counts[applicantId] = (counts[applicantId] || 0) + 1; + return counts; + }, Object.create(null)); + + return uniqueSorted( + Object.entries(applicantCounts) + .filter(([, count]) => count > 1) + .map(([applicantId]) => applicantId) + ); +} + +function reviewApplicantIdFor(review) { + if (!reviewIsRecord(review)) { + return ''; + } + + return typeof review.applicantId === 'string' ? review.applicantId.trim() : review.applicantId; +} + +function reviewListFor(round) { + return Array.isArray(round.reviews) ? round.reviews.filter(reviewIsRecord) : []; +} + +function reviewListIsMissing(round) { + return !Array.isArray(round.reviews); +} + +function reviewListHasMalformedEntries(round) { + return Array.isArray(round.reviews) && round.reviews.some((review) => !reviewIsRecord(review)); +} + +function criteriaListFor(round) { + return Array.isArray(round.criteria) ? round.criteria.map(criterionRecordFor) : []; +} + +function criteriaListIsMissing(round) { + return !Array.isArray(round.criteria); +} + +function criteriaListHasMalformedEntries(round) { + return Array.isArray(round.criteria) && round.criteria.some((criterion) => !criterionIsRecord(criterion)); +} + +function applicantListFor(round) { + return Array.isArray(round.applicants) ? round.applicants : []; +} + +function applicantListIsMissing(round) { + return !Array.isArray(round.applicants); +} + +function outputApplicantIdFor(applicant) { + return applicantIdFor(applicant) || 'unidentified-applicant'; +} + +function duplicateNonConflictedReviewerIds(reviews) { + const counts = reviews + .filter((review) => !review.conflict) + .reduce((reviewerCounts, review) => { + const reviewerId = reviewerIdFor(review); + if (!reviewerId) { + return reviewerCounts; + } + reviewerCounts[reviewerId] = (reviewerCounts[reviewerId] || 0) + 1; + return reviewerCounts; + }, {}); + + return uniqueSorted( + Object.entries(counts) + .filter(([, count]) => count > 1) + .map(([reviewerId]) => reviewerId) + ); +} + +function hasMissingReviewerIdentity(reviews) { + return reviews.some((review) => !reviewerIdFor(review)); +} + +function countableNonConflictedReviews(reviews) { + const seenReviewerIds = new Set(); + + return reviews.filter((review) => { + if (review.conflict) { + return false; + } + + const reviewerId = reviewerIdFor(review); + if (!reviewerId) { + return false; + } + + if (seenReviewerIds.has(reviewerId)) { + return false; + } + + seenReviewerIds.add(reviewerId); + return true; + }); +} + +function publicCriteriaIds(round) { + return criteriaListFor(round).map((criterion) => criterion.id); +} + +function reviewScores(review) { + return reviewIsRecord(review) && review.scores && typeof review.scores === 'object' ? review.scores : {}; +} + +function isValidReviewerScore(score) { + return typeof score === 'number' && Number.isFinite(score) && score >= 0 && score <= 100; +} + +function applicantRejectionReasons(applicant) { + if (!Array.isArray(applicant.rejectionReasons)) { + return []; + } + + return applicant.rejectionReasons + .filter((reason) => typeof reason === 'string') + .map((reason) => reason.trim()) + .filter(Boolean); +} + +function criteriaWeightTotal(round) { + return criteriaListFor(round).reduce((total, criterion) => total + criterion.weight, 0); +} + +function criteriaWeightsHaveInvalidValues(round) { + return criteriaListFor(round).some( + (criterion) => + typeof criterion.weight !== 'number' || + !Number.isFinite(criterion.weight) || + criterion.weight < 0 || + criterion.weight > 100 + ); +} + +function passThresholdIsInvalid(round) { + return ( + typeof round.passThreshold !== 'number' || + !Number.isFinite(round.passThreshold) || + round.passThreshold < 0 || + round.passThreshold > 100 + ); +} + +function reviewerQuorumIsInvalid(round) { + return !Number.isInteger(round.minReviewers) || round.minReviewers < 1; +} + +function sponsorDecisionIsInvalid(applicant) { + return !['accept', 'reject'].includes(applicant.sponsorDecision); +} + +function reviewUsesHiddenCriteria(review, criteriaIds) { + return Object.keys(reviewScores(review)).some((criterionId) => !criteriaIds.includes(criterionId)); +} + +function reviewHasInvalidPublishedScoreValues(review, criteriaIds) { + const scores = reviewScores(review); + return criteriaIds.some( + (criterionId) => + Object.prototype.hasOwnProperty.call(scores, criterionId) && + !isValidReviewerScore(scores[criterionId]) + ); +} + +function appealStatus(applicant, round) { + if (applicant.sponsorDecision !== 'reject') { + return 'not-required'; + } + + if (!applicant.appealDueAt) { + return 'missing'; + } + + const appealDueAt = Date.parse(applicant.appealDueAt); + const generatedAt = Date.parse(round.generatedAt); + if (!Number.isFinite(appealDueAt) || !Number.isFinite(generatedAt)) { + return 'invalid'; + } + + return appealDueAt >= generatedAt ? 'open' : 'expired'; +} + +function reasonsForApplicant(applicant, reviews, round) { + const criteriaIds = publicCriteriaIds(round); + const duplicateCriterionIds = duplicatePublishedCriterionIds(round); + const duplicateApplicantIdentities = duplicateApplicantIds(round); + const nonConflictedReviews = countableNonConflictedReviews(reviews); + const duplicateReviewerIds = duplicateNonConflictedReviewerIds(reviews); + const applicantAppealStatus = appealStatus(applicant, round); + const reasons = []; + + if (applicant.missingApplicantList) { + const missingApplicantReasons = ['missing-applicant-list']; + + if (round.malformedPrequalificationRound) { + missingApplicantReasons.push('malformed-prequalification-round'); + } + + if (challengeIdentityIsMissing(round)) { + missingApplicantReasons.push('missing-challenge-identity'); + } + + if (criteriaListIsMissing(round)) { + missingApplicantReasons.push('missing-published-criteria-list'); + } + + if (reviewListIsMissing(round)) { + missingApplicantReasons.push('missing-review-list'); + } + + return uniqueSorted(missingApplicantReasons); + } + + if (applicant.malformedApplicantEntry) { + return ['malformed-applicant-entry']; + } + + if (round.malformedPrequalificationRound) { + reasons.push('malformed-prequalification-round'); + } + + if (challengeIdentityIsMissing(round)) { + reasons.push('missing-challenge-identity'); + } + + if (round.anonymousScreeningRequired && reviews.some((review) => !review.anonymousScreeningObserved)) { + reasons.push('anonymous-screening-leak'); + } + + if (reviews.some((review) => review.conflict)) { + reasons.push('reviewer-conflict'); + } + + if (duplicateCriterionIds.length > 0) { + reasons.push('duplicate-published-criterion'); + } + + if (hasMissingPublishedCriterionIds(round)) { + reasons.push('missing-published-criterion-id'); + } + + if (criteriaListIsMissing(round)) { + reasons.push('missing-published-criteria-list'); + } + + if (criteriaListHasMalformedEntries(round)) { + reasons.push('malformed-published-criterion-entry'); + } + + if (duplicateReviewerIds.length > 0) { + reasons.push('duplicate-reviewer-score-evidence'); + } + + if (hasMissingReviewerIdentity(reviews)) { + reasons.push('missing-reviewer-identity'); + } + + if (reviewListHasMalformedEntries(round)) { + reasons.push('malformed-review-entry'); + } + + if (reviewListIsMissing(round)) { + reasons.push('missing-review-list'); + } + + if (applicantIdentityIsMissing(applicant)) { + reasons.push('missing-applicant-identity'); + } + + if (duplicateApplicantIdentities.includes(applicantIdFor(applicant))) { + reasons.push('duplicate-applicant-identity'); + } + + if (criteriaWeightTotal(round) !== 100) { + reasons.push('criteria-weight-total-invalid'); + } + + if (criteriaWeightsHaveInvalidValues(round)) { + reasons.push('criteria-weight-value-invalid'); + } + + if (passThresholdIsInvalid(round)) { + reasons.push('pass-threshold-invalid'); + } + + if (reviewerQuorumIsInvalid(round)) { + reasons.push('reviewer-quorum-invalid'); + } + + if (nonConflictedReviews.length < round.minReviewers) { + reasons.push('reviewer-quorum-shortfall'); + } + + if (sponsorDecisionIsInvalid(applicant)) { + reasons.push('sponsor-decision-invalid'); + } + + if (reviews.some((review) => reviewUsesHiddenCriteria(review, criteriaIds))) { + reasons.push('unpublished-screening-criterion'); + } + + if ( + criteriaIds.some((criterionId) => + reviews.some((review) => typeof reviewScores(review)[criterionId] !== 'number') + ) + ) { + reasons.push('missing-published-criterion-score'); + } + + if (reviews.some((review) => reviewHasInvalidPublishedScoreValues(review, criteriaIds))) { + reasons.push('reviewer-score-value-invalid'); + } + + const score = weightedScore(criteriaListFor(round), nonConflictedReviews); + const passesThreshold = score >= round.passThreshold; + + if ( + (applicant.sponsorDecision === 'reject' && passesThreshold) || + (applicant.sponsorDecision === 'accept' && !passesThreshold) + ) { + reasons.push('inconsistent-threshold-decision'); + } + + if (applicant.sponsorDecision === 'reject' && applicantRejectionReasons(applicant).length === 0) { + reasons.push('missing-rejection-reason'); + } + + if (applicant.sponsorDecision === 'reject' && applicantAppealStatus === 'missing') { + reasons.push('missing-appeal-window'); + } + + if (applicant.sponsorDecision === 'reject' && applicantAppealStatus === 'expired') { + reasons.push('expired-appeal-window'); + } + + if (applicant.sponsorDecision === 'reject' && applicantAppealStatus === 'invalid') { + reasons.push('invalid-appeal-window'); + } + + return uniqueSorted(reasons); +} + +function remediationAction(applicant, reasons) { + if (reasons.includes('malformed-prequalification-round')) { + return 'complete-prequalification-evidence'; + } + + if (reasons.includes('missing-challenge-identity')) { + return 'complete-challenge-identity-evidence'; + } + + if (reasons.includes('anonymous-screening-leak')) { + return 'rerun-blinded-prequalification-review'; + } + + if (reasons.includes('missing-published-criteria-list')) { + return 'publish-complete-screening-criteria'; + } + + if (reasons.includes('malformed-published-criterion-entry')) { + return 'publish-complete-screening-criteria'; + } + + if (reasons.includes('unpublished-screening-criterion')) { + return 'remove-unpublished-criterion-and-rescore'; + } + + if (reasons.includes('duplicate-published-criterion')) { + return 'publish-unique-screening-criteria'; + } + + if (reasons.includes('missing-published-criterion-id')) { + return 'publish-complete-screening-criteria'; + } + + if (reasons.includes('criteria-weight-total-invalid')) { + return 'publish-valid-weighted-scoring-rubric'; + } + + if (reasons.includes('criteria-weight-value-invalid')) { + return 'publish-valid-weighted-scoring-rubric'; + } + + if (reasons.includes('pass-threshold-invalid')) { + return 'publish-valid-prequalification-threshold'; + } + + if (reasons.includes('reviewer-quorum-invalid')) { + return 'publish-valid-reviewer-quorum'; + } + + if (reasons.includes('sponsor-decision-invalid')) { + return 'publish-valid-sponsor-decision'; + } + + if (reasons.includes('reviewer-score-value-invalid')) { + return 'publish-valid-reviewer-score-evidence'; + } + + if (reasons.includes('reviewer-conflict')) { + return 'replace-conflicted-reviewer'; + } + + if (reasons.includes('duplicate-reviewer-score-evidence')) { + return 'deduplicate-reviewer-score-evidence'; + } + + if ( + reasons.includes('missing-rejection-reason') || + reasons.includes('missing-appeal-window') || + reasons.includes('expired-appeal-window') || + reasons.includes('invalid-appeal-window') + ) { + return 'publish-rejection-reasons-and-appeal-window'; + } + + if (reasons.includes('missing-published-criterion-score')) { + return 'complete-prequalification-evidence'; + } + + if (reasons.includes('missing-reviewer-identity')) { + return 'complete-prequalification-evidence'; + } + + if (reasons.includes('malformed-review-entry')) { + return 'complete-prequalification-evidence'; + } + + if (reasons.includes('missing-review-list')) { + return 'complete-prequalification-evidence'; + } + + if (reasons.includes('missing-applicant-list')) { + return 'complete-prequalification-evidence'; + } + + if (reasons.includes('malformed-applicant-entry')) { + return 'complete-prequalification-evidence'; + } + + if (reasons.includes('missing-applicant-identity')) { + return 'complete-prequalification-evidence'; + } + + if (reasons.includes('duplicate-applicant-identity')) { + return 'complete-prequalification-evidence'; + } + + if (reasons.includes('inconsistent-threshold-decision')) { + return 'reconcile-score-threshold-decision'; + } + + return 'complete-prequalification-evidence'; +} + +function evaluatePrequalificationRound(round) { + round = prequalificationRoundRecordFor(round); + + const challengeId = outputChallengeIdFor(round); + const reviewsByApplicant = groupBy(reviewListFor(round), reviewApplicantIdFor); + const applicants = applicantListIsMissing(round) + ? [ + { + id: '', + sponsorDecision: null, + rejectionReasons: [], + appealDueAt: null, + missingApplicantList: true + } + ] + : applicantListFor(round).map(applicantRecordFor); + + const decisions = applicants.map((applicant) => { + const reviews = reviewsByApplicant[applicantIdFor(applicant)] || []; + const applicantId = outputApplicantIdFor(applicant); + const nonConflictedReviews = countableNonConflictedReviews(reviews); + const reasons = reasonsForApplicant(applicant, reviews, round); + const score = weightedScore(criteriaListFor(round), nonConflictedReviews); + const decision = + reasons.length > 0 + ? 'hold-for-fairness-review' + : score >= round.passThreshold + ? 'accept-prequalified' + : 'reject-with-audit'; + + return { + id: applicantId, + applicantId, + challengeId, + decision, + sponsorDecision: applicant.sponsorDecision, + weightedScore: score, + passThreshold: round.passThreshold, + reviewersCounted: nonConflictedReviews.length, + criteriaApplied: publicCriteriaIds(round), + reasons, + rejectionReasons: applicantRejectionReasons(applicant), + appealStatus: appealStatus(applicant, round), + auditDigest: digest({ + applicantId, + challengeId, + score, + reasons, + reviews: reviews.map((review) => ({ + reviewerId: review.reviewerId, + conflict: review.conflict, + anonymousScreeningObserved: review.anonymousScreeningObserved, + scores: reviewScores(review) + })) + }) + }; + }); + + const remediationActions = decisions + .filter((decision) => decision.decision === 'hold-for-fairness-review') + .map((decision) => ({ + id: `remediate-${decision.applicantId}`, + applicantId: decision.applicantId, + action: remediationAction(decision, decision.reasons), + priority: + decision.reasons.includes('anonymous-screening-leak') || + decision.reasons.includes('reviewer-conflict') || + decision.reasons.includes('duplicate-published-criterion') || + decision.reasons.includes('missing-published-criterion-id') || + decision.reasons.includes('missing-published-criteria-list') || + decision.reasons.includes('malformed-published-criterion-entry') || + decision.reasons.includes('malformed-prequalification-round') || + decision.reasons.includes('missing-challenge-identity') || + decision.reasons.includes('duplicate-reviewer-score-evidence') || + decision.reasons.includes('missing-reviewer-identity') || + decision.reasons.includes('malformed-review-entry') || + decision.reasons.includes('missing-review-list') || + decision.reasons.includes('missing-applicant-list') || + decision.reasons.includes('malformed-applicant-entry') || + decision.reasons.includes('missing-applicant-identity') || + decision.reasons.includes('duplicate-applicant-identity') || + decision.reasons.includes('unpublished-screening-criterion') || + decision.reasons.includes('criteria-weight-total-invalid') || + decision.reasons.includes('criteria-weight-value-invalid') || + decision.reasons.includes('pass-threshold-invalid') || + decision.reasons.includes('reviewer-quorum-invalid') || + decision.reasons.includes('sponsor-decision-invalid') || + decision.reasons.includes('reviewer-score-value-invalid') + ? 'high' + : 'normal', + reasons: decision.reasons + })); + + const summary = { + accepted: decisions.filter((decision) => decision.decision === 'accept-prequalified').length, + held: decisions.filter((decision) => decision.decision === 'hold-for-fairness-review').length, + rejectedWithAudit: decisions.filter((decision) => decision.decision === 'reject-with-audit') + .length, + remediationActions: remediationActions.length + }; + + return { + challengeId, + generatedAt: round.generatedAt, + criteriaDigest: digest(criteriaListFor(round)), + decisions, + remediationActions, + summary, + auditDigest: digest({ + challengeId: round.challengeId, + generatedAt: round.generatedAt, + decisions, + remediationActions, + summary + }) + }; +} + +function buildSampleRound() { + return { + challengeId: 'challenge-18-prequalification-rna-biomarker', + generatedAt: '2026-05-28T08:00:00Z', + anonymousScreeningRequired: true, + minReviewers: 2, + passThreshold: 75, + criteria: [ + { + id: 'domain-fit', + label: 'Domain fit for the scientific challenge', + weight: 40 + }, + { + id: 'data-readiness', + label: 'Evidence that required data and tools are ready', + weight: 35 + }, + { + id: 'safety-plan', + label: 'Risk, NDA, and responsible-use plan', + weight: 25 + } + ], + applicants: [ + { + id: 'applicant-biofoundry', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + }, + { + id: 'applicant-neuro-lab', + sponsorDecision: 'reject', + rejectionReasons: ['methods plan did not match expected wet-lab validation'], + appealDueAt: '2026-06-04T08:00:00Z' + }, + { + id: 'applicant-sponsor-alumni', + sponsorDecision: 'reject', + rejectionReasons: [], + appealDueAt: null + }, + { + id: 'applicant-missing-reviewer-identity', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ], + reviews: [ + { + applicantId: 'applicant-biofoundry', + reviewerId: 'reviewer-alpha', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 90, + 'data-readiness': 86, + 'safety-plan': 84 + } + }, + { + applicantId: 'applicant-biofoundry', + reviewerId: 'reviewer-beta', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 88, + 'data-readiness': 85, + 'safety-plan': 85 + } + }, + { + applicantId: 'applicant-neuro-lab', + reviewerId: 'reviewer-alpha', + anonymousScreeningObserved: false, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['identity visible in screening packet'], + scores: { + 'domain-fit': 82, + 'data-readiness': 80, + 'safety-plan': 78 + } + }, + { + applicantId: 'applicant-neuro-lab', + reviewerId: 'reviewer-gamma', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['identity visible in screening packet'], + scores: { + 'domain-fit': 84, + 'data-readiness': 81, + 'safety-plan': 79 + } + }, + { + applicantId: 'applicant-sponsor-alumni', + reviewerId: 'reviewer-delta', + anonymousScreeningObserved: true, + conflict: true, + recommendedDecision: 'reject', + rejectionReasons: [], + scores: { + 'domain-fit': 74, + 'data-readiness': 72, + 'safety-plan': 76 + } + }, + { + applicantId: 'applicant-sponsor-alumni', + reviewerId: 'reviewer-epsilon', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: [], + scores: { + 'domain-fit': 70, + 'data-readiness': 68, + 'safety-plan': 73 + } + }, + { + applicantId: 'applicant-missing-reviewer-identity', + reviewerId: ' ', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 95, + 'safety-plan': 93 + } + }, + { + applicantId: 'applicant-missing-reviewer-identity', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 91 + } + } + ] + }; +} + +module.exports = { + evaluatePrequalificationRound, + buildSampleRound, + digest +}; diff --git a/challenge-prequalification-fairness-guard/make-demo-video.py b/challenge-prequalification-fairness-guard/make-demo-video.py new file mode 100644 index 00000000..da39b9d6 --- /dev/null +++ b/challenge-prequalification-fairness-guard/make-demo-video.py @@ -0,0 +1,62 @@ +import os +import subprocess + + +HERE = os.path.dirname(os.path.abspath(__file__)) +REPORTS = os.path.join(HERE, "reports") +FRAME = os.path.join(REPORTS, "demo-frame.png") +OUTPUT = os.path.join(REPORTS, "demo.mp4") +os.makedirs(REPORTS, exist_ok=True) + + +def draw_frame_with_pillow(): + from PIL import Image, ImageDraw, ImageFont + + image = Image.new("RGB", (1280, 720), "#102027") + draw = ImageDraw.Draw(image) + draw.rounded_rectangle((54, 58, 1226, 662), radius=16, fill="#17313a", outline="#9bd67a", width=4) + + try: + title_font = ImageFont.truetype("arial.ttf", 48) + body_font = ImageFont.truetype("arial.ttf", 30) + note_font = ImageFont.truetype("arial.ttf", 26) + except OSError: + title_font = ImageFont.load_default() + body_font = ImageFont.load_default() + note_font = ImageFont.load_default() + + draw.text((96, 102), "Challenge Prequalification Fairness Guard", fill="white", font=title_font) + draw.text((96, 190), "Published criteria plus weighted threshold checks", fill="#dff5d5", font=body_font) + draw.text((96, 248), "Invalid sponsor decisions cannot change access", fill="#dff5d5", font=body_font) + draw.text((96, 306), "Malformed packets, review lists, and identities are held", fill="#dff5d5", font=body_font) + draw.text((96, 402), "Synthetic data only. No sponsor, solver, payout, or identity systems are called.", fill="#ffd37a", font=note_font) + + image.save(FRAME) + + +draw_frame_with_pillow() + +cmd = [ + "ffmpeg", + "-y", + "-loop", + "1", + "-i", + FRAME, + "-t", + "4", + "-r", + "30", + "-c:v", + "libx264", + "-pix_fmt", + "yuv420p", + "-movflags", + "+faststart", + OUTPUT, +] + +subprocess.run(cmd, check=True) +if os.path.exists(FRAME): + os.remove(FRAME) +print(f"Wrote {os.path.relpath(OUTPUT, HERE)}") diff --git a/challenge-prequalification-fairness-guard/package.json b/challenge-prequalification-fairness-guard/package.json new file mode 100644 index 00000000..069a41a9 --- /dev/null +++ b/challenge-prequalification-fairness-guard/package.json @@ -0,0 +1,13 @@ +{ + "name": "challenge-prequalification-fairness-guard", + "version": "1.0.0", + "description": "Dependency-free prequalification fairness guard for SCIBASE scientific bounty challenges.", + "main": "index.js", + "private": true, + "scripts": { + "test": "node test.js", + "demo": "node demo.js", + "video": "python make-demo-video.py", + "check": "npm test && npm run demo && npm run video" + } +} diff --git a/challenge-prequalification-fairness-guard/reports/blank-rejection-reason-packet.json b/challenge-prequalification-fairness-guard/reports/blank-rejection-reason-packet.json new file mode 100644 index 00000000..909cbf56 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/blank-rejection-reason-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "applicant-blank-rejection-reason", + "applicantId": "applicant-blank-rejection-reason", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "reject", + "weightedScore": 60, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "missing-rejection-reason" + ], + "rejectionReasons": [], + "appealStatus": "open", + "auditDigest": "sha256:500be54f555dd977c9a3cfb05ab5e9a7bfbfb7217b7a484f5193fd00b7030c61" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-blank-rejection-reason", + "applicantId": "applicant-blank-rejection-reason", + "action": "publish-rejection-reasons-and-appeal-window", + "priority": "normal", + "reasons": [ + "missing-rejection-reason" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:bbd51d47794aadc8faa0eda8231781f66d0e4eacde31bd0362b6e723834a444c" +} diff --git a/challenge-prequalification-fairness-guard/reports/demo.mp4 b/challenge-prequalification-fairness-guard/reports/demo.mp4 new file mode 100644 index 00000000..07285783 Binary files /dev/null and b/challenge-prequalification-fairness-guard/reports/demo.mp4 differ diff --git a/challenge-prequalification-fairness-guard/reports/duplicate-applicant-identity-packet.json b/challenge-prequalification-fairness-guard/reports/duplicate-applicant-identity-packet.json new file mode 100644 index 00000000..a08b024c --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/duplicate-applicant-identity-packet.json @@ -0,0 +1,80 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "applicant-duplicate", + "applicantId": "applicant-duplicate", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 94, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "duplicate-applicant-identity" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:1b3676776e5454bab52424913826cb5714e450f80ff2c48a16008d85000ac89d" + }, + { + "id": "applicant-duplicate", + "applicantId": "applicant-duplicate", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "reject", + "weightedScore": 94, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "duplicate-applicant-identity", + "inconsistent-threshold-decision" + ], + "rejectionReasons": [ + "duplicate entry should be resolved before screening" + ], + "appealStatus": "open", + "auditDigest": "sha256:77ce84425359c61ac9ef3741dcb4e71b530bd8c8b547cb209f9e1a8125446fee" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-duplicate", + "applicantId": "applicant-duplicate", + "action": "complete-prequalification-evidence", + "priority": "high", + "reasons": [ + "duplicate-applicant-identity" + ] + }, + { + "id": "remediate-applicant-duplicate", + "applicantId": "applicant-duplicate", + "action": "complete-prequalification-evidence", + "priority": "high", + "reasons": [ + "duplicate-applicant-identity", + "inconsistent-threshold-decision" + ] + } + ], + "summary": { + "accepted": 0, + "held": 2, + "rejectedWithAudit": 0, + "remediationActions": 2 + }, + "auditDigest": "sha256:3f23ed40ab5aa7c06e33ca3dc3b149bae1d550f3764425c6955cedf04beb01e0" +} diff --git a/challenge-prequalification-fairness-guard/reports/invalid-reviewer-quorum-packet.json b/challenge-prequalification-fairness-guard/reports/invalid-reviewer-quorum-packet.json new file mode 100644 index 00000000..3352ae1a --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/invalid-reviewer-quorum-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "applicant-invalid-reviewer-quorum", + "applicantId": "applicant-invalid-reviewer-quorum", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 94, + "passThreshold": 75, + "reviewersCounted": 1, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "reviewer-quorum-invalid" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:d622405e7189eef202d2ccf557217bd26357f6249c7adf908537aa089bb8e24f" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-invalid-reviewer-quorum", + "applicantId": "applicant-invalid-reviewer-quorum", + "action": "publish-valid-reviewer-quorum", + "priority": "high", + "reasons": [ + "reviewer-quorum-invalid" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:61801b0cd7cf7c62fc38b0f6e62415e770d388bf3844e2132b8a379183e74ca1" +} diff --git a/challenge-prequalification-fairness-guard/reports/invalid-reviewer-score-packet.json b/challenge-prequalification-fairness-guard/reports/invalid-reviewer-score-packet.json new file mode 100644 index 00000000..a5f15bd7 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/invalid-reviewer-score-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "applicant-invalid-reviewer-score", + "applicantId": "applicant-invalid-reviewer-score", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 93, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "reviewer-score-value-invalid" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:fff8ddec8e225f5de87c9e73653b27e47f45a534d6b6759d74811656c403020c" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-invalid-reviewer-score", + "applicantId": "applicant-invalid-reviewer-score", + "action": "publish-valid-reviewer-score-evidence", + "priority": "high", + "reasons": [ + "reviewer-score-value-invalid" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:806cf8166a6f1d58824929c4964ec009b6d6a8ce4d613b9e357a1e0e684fda69" +} diff --git a/challenge-prequalification-fairness-guard/reports/invalid-sponsor-decision-packet.json b/challenge-prequalification-fairness-guard/reports/invalid-sponsor-decision-packet.json new file mode 100644 index 00000000..f8687340 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/invalid-sponsor-decision-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "applicant-invalid-sponsor-decision", + "applicantId": "applicant-invalid-sponsor-decision", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "waitlist", + "weightedScore": 94, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "sponsor-decision-invalid" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:479b6f96e9e11bce1a330628272c24df445b8ca0bbc93de32748df3c240aca70" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-invalid-sponsor-decision", + "applicantId": "applicant-invalid-sponsor-decision", + "action": "publish-valid-sponsor-decision", + "priority": "high", + "reasons": [ + "sponsor-decision-invalid" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:030c1d99cb355c21fb3d67e9876997fd4a8bb2042a76d205fe3c3b79beaeccc3" +} diff --git a/challenge-prequalification-fairness-guard/reports/malformed-applicant-entry-packet.json b/challenge-prequalification-fairness-guard/reports/malformed-applicant-entry-packet.json new file mode 100644 index 00000000..46c87782 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/malformed-applicant-entry-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "unidentified-applicant", + "applicantId": "unidentified-applicant", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": null, + "weightedScore": 0, + "passThreshold": 75, + "reviewersCounted": 0, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "malformed-applicant-entry" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:fb51cfbd84ab96e5595f67cb28b427e84f73a592276ba2866afb43fac47541e9" + } + ], + "remediationActions": [ + { + "id": "remediate-unidentified-applicant", + "applicantId": "unidentified-applicant", + "action": "complete-prequalification-evidence", + "priority": "high", + "reasons": [ + "malformed-applicant-entry" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:f186f54dec87f06c4cc6c61d8de82dfc74749a0ce103446d2084e1fb0f3b6de4" +} diff --git a/challenge-prequalification-fairness-guard/reports/malformed-criterion-entry-packet.json b/challenge-prequalification-fairness-guard/reports/malformed-criterion-entry-packet.json new file mode 100644 index 00000000..4ed8d050 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/malformed-criterion-entry-packet.json @@ -0,0 +1,54 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:2e78a5062ffa5673f5eefb1185b8616fa56e435f3c4320476454b240d3355ab6", + "decisions": [ + { + "id": "applicant-malformed-criterion-entry", + "applicantId": "applicant-malformed-criterion-entry", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 0, + "passThreshold": 75, + "reviewersCounted": 0, + "criteriaApplied": [ + "" + ], + "reasons": [ + "criteria-weight-total-invalid", + "criteria-weight-value-invalid", + "inconsistent-threshold-decision", + "malformed-published-criterion-entry", + "missing-published-criterion-id", + "reviewer-quorum-shortfall" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:b2e87c7470068a0abe378189749a4d088d4cd7334bd2c769259d75b813bba848" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-malformed-criterion-entry", + "applicantId": "applicant-malformed-criterion-entry", + "action": "publish-complete-screening-criteria", + "priority": "high", + "reasons": [ + "criteria-weight-total-invalid", + "criteria-weight-value-invalid", + "inconsistent-threshold-decision", + "malformed-published-criterion-entry", + "missing-published-criterion-id", + "reviewer-quorum-shortfall" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:288e8b93e7b316689b59f0d21d90d99e579ad0231804b4d64291f20e66d7ec4a" +} diff --git a/challenge-prequalification-fairness-guard/reports/malformed-prequalification-round-packet.json b/challenge-prequalification-fairness-guard/reports/malformed-prequalification-round-packet.json new file mode 100644 index 00000000..f85d7b24 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/malformed-prequalification-round-packet.json @@ -0,0 +1,50 @@ +{ + "challengeId": "unidentified-challenge", + "generatedAt": null, + "criteriaDigest": "sha256:4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945", + "decisions": [ + { + "id": "unidentified-applicant", + "applicantId": "unidentified-applicant", + "challengeId": "unidentified-challenge", + "decision": "hold-for-fairness-review", + "sponsorDecision": null, + "weightedScore": 0, + "passThreshold": null, + "reviewersCounted": 0, + "criteriaApplied": [], + "reasons": [ + "malformed-prequalification-round", + "missing-applicant-list", + "missing-challenge-identity", + "missing-published-criteria-list", + "missing-review-list" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:5c4cc793ce381dd5e0541d9fc2564f21717a0650337b35eeed69f347b081bd98" + } + ], + "remediationActions": [ + { + "id": "remediate-unidentified-applicant", + "applicantId": "unidentified-applicant", + "action": "complete-prequalification-evidence", + "priority": "high", + "reasons": [ + "malformed-prequalification-round", + "missing-applicant-list", + "missing-challenge-identity", + "missing-published-criteria-list", + "missing-review-list" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:83b5c9c1376cc3afd1111fb7c9ba79b3fcbd41be82907d10d40590a40d9d9f9e" +} diff --git a/challenge-prequalification-fairness-guard/reports/malformed-review-entry-packet.json b/challenge-prequalification-fairness-guard/reports/malformed-review-entry-packet.json new file mode 100644 index 00000000..43db53f6 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/malformed-review-entry-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "applicant-malformed-review-entry", + "applicantId": "applicant-malformed-review-entry", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 93, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "malformed-review-entry" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:212227740c1b3a1d0dcdc877a8ef7923dc919892d25c7cf0da7b10c9cae600e9" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-malformed-review-entry", + "applicantId": "applicant-malformed-review-entry", + "action": "complete-prequalification-evidence", + "priority": "high", + "reasons": [ + "malformed-review-entry" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:92f8e6dbb21bc4f859670a8add42b7e265487aec1eead2c48a87adc717d95976" +} diff --git a/challenge-prequalification-fairness-guard/reports/missing-applicant-identity-packet.json b/challenge-prequalification-fairness-guard/reports/missing-applicant-identity-packet.json new file mode 100644 index 00000000..a6b64b1c --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/missing-applicant-identity-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "unidentified-applicant", + "applicantId": "unidentified-applicant", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 94, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "missing-applicant-identity" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:ccf0124de14de94d0157b027bc0636050d77e4d7dca29aa756fcf3114dabdec4" + } + ], + "remediationActions": [ + { + "id": "remediate-unidentified-applicant", + "applicantId": "unidentified-applicant", + "action": "complete-prequalification-evidence", + "priority": "high", + "reasons": [ + "missing-applicant-identity" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:85aa774922f5707139444c13c705a88af31757014d98a8767b0cbe1725c4cf7c" +} diff --git a/challenge-prequalification-fairness-guard/reports/missing-applicant-list-packet.json b/challenge-prequalification-fairness-guard/reports/missing-applicant-list-packet.json new file mode 100644 index 00000000..615d69f6 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/missing-applicant-list-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "unidentified-applicant", + "applicantId": "unidentified-applicant", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": null, + "weightedScore": 0, + "passThreshold": 75, + "reviewersCounted": 0, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "missing-applicant-list" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:4de0629dfb7646c44064dfaad44ba573fc82612aba864d6f643989a0f69d4d0f" + } + ], + "remediationActions": [ + { + "id": "remediate-unidentified-applicant", + "applicantId": "unidentified-applicant", + "action": "complete-prequalification-evidence", + "priority": "high", + "reasons": [ + "missing-applicant-list" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:c77ccb7f1875504454711c406569be8b6b9b34789239e956635850890ecf1a90" +} diff --git a/challenge-prequalification-fairness-guard/reports/missing-challenge-identity-packet.json b/challenge-prequalification-fairness-guard/reports/missing-challenge-identity-packet.json new file mode 100644 index 00000000..ec8fc16d --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/missing-challenge-identity-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "unidentified-challenge", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "applicant-missing-challenge-identity", + "applicantId": "applicant-missing-challenge-identity", + "challengeId": "unidentified-challenge", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 93, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "missing-challenge-identity" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:91f727ed6c0f39a3fba57e3aff235d131b4e5a1944b1d8f2ede901bc1fa988a3" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-missing-challenge-identity", + "applicantId": "applicant-missing-challenge-identity", + "action": "complete-challenge-identity-evidence", + "priority": "high", + "reasons": [ + "missing-challenge-identity" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:dcd93740470a0b4a16d6e7913c2f62a81fc895d7cbf664b4317b239bddd1ae6d" +} diff --git a/challenge-prequalification-fairness-guard/reports/missing-criteria-list-packet.json b/challenge-prequalification-fairness-guard/reports/missing-criteria-list-packet.json new file mode 100644 index 00000000..6af8a3e0 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/missing-criteria-list-packet.json @@ -0,0 +1,48 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945", + "decisions": [ + { + "id": "applicant-missing-criteria-list", + "applicantId": "applicant-missing-criteria-list", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 0, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [], + "reasons": [ + "criteria-weight-total-invalid", + "inconsistent-threshold-decision", + "missing-published-criteria-list", + "unpublished-screening-criterion" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:21d6fa1aa15c41e101dd8914c23858396adfea0699a700a91fcb7501bef9da78" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-missing-criteria-list", + "applicantId": "applicant-missing-criteria-list", + "action": "publish-complete-screening-criteria", + "priority": "high", + "reasons": [ + "criteria-weight-total-invalid", + "inconsistent-threshold-decision", + "missing-published-criteria-list", + "unpublished-screening-criterion" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:c2d8b60e1ca8cc50e8b8c072ee2a22a9a832df09b59e63d60dd21b41b1f17fd3" +} diff --git a/challenge-prequalification-fairness-guard/reports/missing-criterion-id-packet.json b/challenge-prequalification-fairness-guard/reports/missing-criterion-id-packet.json new file mode 100644 index 00000000..59f34bec --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/missing-criterion-id-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:9e2cf980ff56d806dbc5ec4ff2a53670f3cf7986d200238134d1f120463db3eb", + "decisions": [ + { + "id": "applicant-missing-criterion-id", + "applicantId": "applicant-missing-criterion-id", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 94, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + " ", + "safety-plan" + ], + "reasons": [ + "missing-published-criterion-id" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:d30b6b4dd2eebd762e453ca03cd6c8a866bbdcc692e225bbfc042b97f0626728" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-missing-criterion-id", + "applicantId": "applicant-missing-criterion-id", + "action": "publish-complete-screening-criteria", + "priority": "high", + "reasons": [ + "missing-published-criterion-id" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:b9df8bf4b4cf0ab259501a17673e27b1537153778f87a19190f026759151b27e" +} diff --git a/challenge-prequalification-fairness-guard/reports/missing-review-list-packet.json b/challenge-prequalification-fairness-guard/reports/missing-review-list-packet.json new file mode 100644 index 00000000..3a4a7f38 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/missing-review-list-packet.json @@ -0,0 +1,50 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "applicant-missing-review-list", + "applicantId": "applicant-missing-review-list", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 0, + "passThreshold": 75, + "reviewersCounted": 0, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "inconsistent-threshold-decision", + "missing-review-list", + "reviewer-quorum-shortfall" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:792e589c9757b0b7d48f0f48fbc3863f5e42fb0926864968a582e47ebed09614" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-missing-review-list", + "applicantId": "applicant-missing-review-list", + "action": "complete-prequalification-evidence", + "priority": "high", + "reasons": [ + "inconsistent-threshold-decision", + "missing-review-list", + "reviewer-quorum-shortfall" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:205a2e89821f1f3e7c9b83e959f0b123967d8af29b659d89af898e473ed4033d" +} diff --git a/challenge-prequalification-fairness-guard/reports/normalized-criterion-id-packet.json b/challenge-prequalification-fairness-guard/reports/normalized-criterion-id-packet.json new file mode 100644 index 00000000..e03cfcd1 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/normalized-criterion-id-packet.json @@ -0,0 +1,46 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:2a1b7cf1647fefb5c70a5fa5ec271e5e0195c54bca77fcc3b54eb912092b2f8d", + "decisions": [ + { + "id": "applicant-normalized-criterion-id", + "applicantId": "applicant-normalized-criterion-id", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 94, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + " domain-fit ", + "safety-plan" + ], + "reasons": [ + "duplicate-published-criterion" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:ffd439a9c5cce5c2c8223654147ab3a7db7e3ac741d76ecc626766dee821634e" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-normalized-criterion-id", + "applicantId": "applicant-normalized-criterion-id", + "action": "publish-unique-screening-criteria", + "priority": "high", + "reasons": [ + "duplicate-published-criterion" + ] + } + ], + "summary": { + "accepted": 0, + "held": 1, + "rejectedWithAudit": 0, + "remediationActions": 1 + }, + "auditDigest": "sha256:fdfd47231c8acfdad73947f357080e3e6967bd11dd622ee60d0ebebb40688ee6" +} diff --git a/challenge-prequalification-fairness-guard/reports/prequalification-fairness-packet.json b/challenge-prequalification-fairness-guard/reports/prequalification-fairness-packet.json new file mode 100644 index 00000000..9635156a --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/prequalification-fairness-packet.json @@ -0,0 +1,139 @@ +{ + "challengeId": "challenge-18-prequalification-rna-biomarker", + "generatedAt": "2026-05-28T08:00:00Z", + "criteriaDigest": "sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721", + "decisions": [ + { + "id": "applicant-biofoundry", + "applicantId": "applicant-biofoundry", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "accept-prequalified", + "sponsorDecision": "accept", + "weightedScore": 87, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:0717493fb82a925e390d8b4fbd7cac2039dd71a4bf7d8db19848a69efd926c53" + }, + { + "id": "applicant-neuro-lab", + "applicantId": "applicant-neuro-lab", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "reject", + "weightedScore": 81, + "passThreshold": 75, + "reviewersCounted": 2, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "anonymous-screening-leak", + "inconsistent-threshold-decision" + ], + "rejectionReasons": [ + "methods plan did not match expected wet-lab validation" + ], + "appealStatus": "open", + "auditDigest": "sha256:74514248ce1bdd761a0c8034d9b39962227e44f53bd826152246014a29608bfc" + }, + { + "id": "applicant-sponsor-alumni", + "applicantId": "applicant-sponsor-alumni", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "reject", + "weightedScore": 70, + "passThreshold": 75, + "reviewersCounted": 1, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "missing-appeal-window", + "missing-rejection-reason", + "reviewer-conflict", + "reviewer-quorum-shortfall" + ], + "rejectionReasons": [], + "appealStatus": "missing", + "auditDigest": "sha256:155ac57faaf2c7ccabbb1799d38d4f73c1ed458c2da1a0bfc8fe49a2f4120dee" + }, + { + "id": "applicant-missing-reviewer-identity", + "applicantId": "applicant-missing-reviewer-identity", + "challengeId": "challenge-18-prequalification-rna-biomarker", + "decision": "hold-for-fairness-review", + "sponsorDecision": "accept", + "weightedScore": 0, + "passThreshold": 75, + "reviewersCounted": 0, + "criteriaApplied": [ + "domain-fit", + "data-readiness", + "safety-plan" + ], + "reasons": [ + "inconsistent-threshold-decision", + "missing-reviewer-identity", + "reviewer-quorum-shortfall" + ], + "rejectionReasons": [], + "appealStatus": "not-required", + "auditDigest": "sha256:155b16a3cbdbb41a53a5df5faf6e7e64e6854dbac37beb448deb2a829e4edeec" + } + ], + "remediationActions": [ + { + "id": "remediate-applicant-neuro-lab", + "applicantId": "applicant-neuro-lab", + "action": "rerun-blinded-prequalification-review", + "priority": "high", + "reasons": [ + "anonymous-screening-leak", + "inconsistent-threshold-decision" + ] + }, + { + "id": "remediate-applicant-sponsor-alumni", + "applicantId": "applicant-sponsor-alumni", + "action": "replace-conflicted-reviewer", + "priority": "high", + "reasons": [ + "missing-appeal-window", + "missing-rejection-reason", + "reviewer-conflict", + "reviewer-quorum-shortfall" + ] + }, + { + "id": "remediate-applicant-missing-reviewer-identity", + "applicantId": "applicant-missing-reviewer-identity", + "action": "complete-prequalification-evidence", + "priority": "high", + "reasons": [ + "inconsistent-threshold-decision", + "missing-reviewer-identity", + "reviewer-quorum-shortfall" + ] + } + ], + "summary": { + "accepted": 1, + "held": 3, + "rejectedWithAudit": 0, + "remediationActions": 3 + }, + "auditDigest": "sha256:5ec541696fa83ff5f5ac49891dbb4bdf0e1174638039f2a2e7b6e65f2df49d16" +} diff --git a/challenge-prequalification-fairness-guard/reports/prequalification-fairness-report.md b/challenge-prequalification-fairness-guard/reports/prequalification-fairness-report.md new file mode 100644 index 00000000..005767ac --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/prequalification-fairness-report.md @@ -0,0 +1,161 @@ +# Challenge Prequalification Fairness Guard + +Challenge: challenge-18-prequalification-rna-biomarker +Generated: 2026-05-28T08:00:00Z + +## Summary + +- Accepted applicants: 1 +- Held for fairness review: 3 +- Rejected with audit trail: 0 +- Remediation actions: 3 +- Criteria digest: sha256:d643b033793917b9d0488787518a11e97094e671d52b86b69a6153375d726721 +- Audit digest: sha256:5ec541696fa83ff5f5ac49891dbb4bdf0e1174638039f2a2e7b6e65f2df49d16 + +## Decisions + +- applicant-biofoundry: accept-prequalified, score 87, reasons: none +- applicant-neuro-lab: hold-for-fairness-review, score 81, reasons: anonymous-screening-leak, inconsistent-threshold-decision +- applicant-sponsor-alumni: hold-for-fairness-review, score 70, reasons: missing-appeal-window, missing-rejection-reason, reviewer-conflict, reviewer-quorum-shortfall +- applicant-missing-reviewer-identity: hold-for-fairness-review, score 0, reasons: inconsistent-threshold-decision, missing-reviewer-identity, reviewer-quorum-shortfall + +## Remediation Actions + +- remediate-applicant-neuro-lab: rerun-blinded-prequalification-review (high) +- remediate-applicant-sponsor-alumni: replace-conflicted-reviewer (high) +- remediate-applicant-missing-reviewer-identity: complete-prequalification-evidence (high) + +## Missing Criterion Identifier Packet + +- Applicant: applicant-missing-criterion-id +- Decision: hold-for-fairness-review +- Reasons: missing-published-criterion-id +- Remediation: publish-complete-screening-criteria +- Audit digest: sha256:b9df8bf4b4cf0ab259501a17673e27b1537153778f87a19190f026759151b27e + +## Normalized Criterion Identifier Packet + +- Applicant: applicant-normalized-criterion-id +- Decision: hold-for-fairness-review +- Reasons: duplicate-published-criterion +- Remediation: publish-unique-screening-criteria +- Audit digest: sha256:fdfd47231c8acfdad73947f357080e3e6967bd11dd622ee60d0ebebb40688ee6 + +## Invalid Reviewer Score Packet + +- Applicant: applicant-invalid-reviewer-score +- Decision: hold-for-fairness-review +- Reasons: reviewer-score-value-invalid +- Remediation: publish-valid-reviewer-score-evidence +- Audit digest: sha256:806cf8166a6f1d58824929c4964ec009b6d6a8ce4d613b9e357a1e0e684fda69 + +## Invalid Reviewer Quorum Packet + +- Applicant: applicant-invalid-reviewer-quorum +- Decision: hold-for-fairness-review +- Reasons: reviewer-quorum-invalid +- Remediation: publish-valid-reviewer-quorum +- Audit digest: sha256:61801b0cd7cf7c62fc38b0f6e62415e770d388bf3844e2132b8a379183e74ca1 + +## Invalid Sponsor Decision Packet + +- Applicant: applicant-invalid-sponsor-decision +- Decision: hold-for-fairness-review +- Reasons: sponsor-decision-invalid +- Remediation: publish-valid-sponsor-decision +- Audit digest: sha256:030c1d99cb355c21fb3d67e9876997fd4a8bb2042a76d205fe3c3b79beaeccc3 + +## Missing Applicant Identity Packet + +- Applicant: "unidentified-applicant" +- Decision: hold-for-fairness-review +- Reasons: missing-applicant-identity +- Remediation: complete-prequalification-evidence +- Audit digest: sha256:85aa774922f5707139444c13c705a88af31757014d98a8767b0cbe1725c4cf7c + +## Duplicate Applicant Identity Packet + +- Applicant: applicant-duplicate +- Decision: hold-for-fairness-review +- Reasons: duplicate-applicant-identity +- Remediation: complete-prequalification-evidence +- Audit digest: sha256:3f23ed40ab5aa7c06e33ca3dc3b149bae1d550f3764425c6955cedf04beb01e0 + +## Missing Review List Packet + +- Applicant: applicant-missing-review-list +- Decision: hold-for-fairness-review +- Reasons: inconsistent-threshold-decision, missing-review-list, reviewer-quorum-shortfall +- Remediation: complete-prequalification-evidence +- Audit digest: sha256:205a2e89821f1f3e7c9b83e959f0b123967d8af29b659d89af898e473ed4033d + +## Malformed Review Entry Packet + +- Applicant: applicant-malformed-review-entry +- Decision: hold-for-fairness-review +- Reviewers counted: 2 +- Reasons: malformed-review-entry +- Remediation: complete-prequalification-evidence +- Audit digest: sha256:92f8e6dbb21bc4f859670a8add42b7e265487aec1eead2c48a87adc717d95976 + +## Missing Criteria List Packet + +- Applicant: applicant-missing-criteria-list +- Decision: hold-for-fairness-review +- Reasons: criteria-weight-total-invalid, inconsistent-threshold-decision, missing-published-criteria-list, unpublished-screening-criterion +- Remediation: publish-complete-screening-criteria +- Audit digest: sha256:c2d8b60e1ca8cc50e8b8c072ee2a22a9a832df09b59e63d60dd21b41b1f17fd3 + +## Malformed Criterion Entry Packet + +- Applicant: applicant-malformed-criterion-entry +- Decision: hold-for-fairness-review +- Reasons: criteria-weight-total-invalid, criteria-weight-value-invalid, inconsistent-threshold-decision, malformed-published-criterion-entry, missing-published-criterion-id, reviewer-quorum-shortfall +- Remediation: publish-complete-screening-criteria +- Audit digest: sha256:288e8b93e7b316689b59f0d21d90d99e579ad0231804b4d64291f20e66d7ec4a + +## Missing Applicant List Packet + +- Applicant: unidentified-applicant +- Decision: hold-for-fairness-review +- Reasons: missing-applicant-list +- Remediation: complete-prequalification-evidence +- Audit digest: sha256:c77ccb7f1875504454711c406569be8b6b9b34789239e956635850890ecf1a90 + +## Malformed Prequalification Round Packet + +- Challenge: unidentified-challenge +- Applicant: unidentified-applicant +- Decision: hold-for-fairness-review +- Reasons: malformed-prequalification-round, missing-applicant-list, missing-challenge-identity, missing-published-criteria-list, missing-review-list +- Remediation: complete-prequalification-evidence +- Audit digest: sha256:83b5c9c1376cc3afd1111fb7c9ba79b3fcbd41be82907d10d40590a40d9d9f9e + +## Missing Challenge Identity Packet + +- Challenge: unidentified-challenge +- Applicant: applicant-missing-challenge-identity +- Decision: hold-for-fairness-review +- Reasons: missing-challenge-identity +- Remediation: complete-challenge-identity-evidence +- Audit digest: sha256:dcd93740470a0b4a16d6e7913c2f62a81fc895d7cbf664b4317b239bddd1ae6d + +## Malformed Applicant Entry Packet + +- Applicant: unidentified-applicant +- Decision: hold-for-fairness-review +- Reasons: malformed-applicant-entry +- Remediation: complete-prequalification-evidence +- Audit digest: sha256:f186f54dec87f06c4cc6c61d8de82dfc74749a0ce103446d2084e1fb0f3b6de4 + +## Blank Rejection Reason Packet + +- Applicant: applicant-blank-rejection-reason +- Decision: hold-for-fairness-review +- Reasons: missing-rejection-reason +- Remediation: publish-rejection-reasons-and-appeal-window +- Audit digest: sha256:bbd51d47794aadc8faa0eda8231781f66d0e4eacde31bd0362b6e723834a444c + +## Safety + +All fixtures are synthetic. The guard does not call payment processors, identity providers, private workspaces, sponsor systems, or external APIs. diff --git a/challenge-prequalification-fairness-guard/reports/summary.svg b/challenge-prequalification-fairness-guard/reports/summary.svg new file mode 100644 index 00000000..88345ce8 --- /dev/null +++ b/challenge-prequalification-fairness-guard/reports/summary.svg @@ -0,0 +1,12 @@ + + + + Challenge Prequalification Fairness Guard + Accepted applicants: 1 + Held for fairness review: 3 + Remediation actions: 3 + Checks: criteria, applicant identities, sponsor decisions, valid scores + thresholds, anonymity, conflicts, appeals + Unfair screening decisions are held before applicants are accepted or rejected. + sha256:5ec541696fa83ff5f5ac49891dbb4bdf0e1174638039f2a2e7b6e65f2df49d16 + diff --git a/challenge-prequalification-fairness-guard/requirements-map.md b/challenge-prequalification-fairness-guard/requirements-map.md new file mode 100644 index 00000000..a2bbb442 --- /dev/null +++ b/challenge-prequalification-fairness-guard/requirements-map.md @@ -0,0 +1,47 @@ +# Requirements Map + +## Challenge Posting Portal + +- Verifies that prequalification rounds use complete published criteria lists, complete and unique criterion identifiers after trimming, nonnegative weights, valid weight totals, valid 0-100 pass thresholds, valid positive reviewer quorum requirements, explicit sponsor accept/reject decisions, complete applicant lists, and complete unique applicant identities after trimming. +- Blocks unpublished sponsor preferences from entering solver-screening decisions. +- Keeps prequalification decisions tied to challenge timelines and parseable appeal windows. + +## Submission Engine + +- Protects anonymous or named participation settings during prequalification review. +- Requires a valid positive reviewer quorum before a solver team is accepted or rejected, and excludes missing or blank reviewer identities from quorum until reviewer evidence is completed. +- Holds missing, blank, duplicate, or malformed applicant identity entries before malformed applicant rows can change solver-team access. +- Holds missing round-level criteria lists and review lists before sparse prequalification packets can crash or change solver-team access. +- Holds malformed review entries before sparse reviewer evidence can crash or change solver-team access. +- Holds missing applicant lists before sparse prequalification packets can crash or change solver-team access. +- Holds malformed top-level prequalification packets before sparse challenge payloads can crash or change solver-team access. +- Holds incomplete reviewer score packets and invalid finite 0-100 score values for evidence completion instead of letting malformed review records crash or drive decisions. +- Preserves audit evidence for each applicant before access to private challenge workspaces changes. + +## Arbitration And Reward Distribution + +- Holds inconsistent threshold decisions for fairness review before a solver is excluded. +- Holds invalid pass thresholds for fairness review before sponsor accept/reject decisions can take effect. +- Holds invalid reviewer quorum requirements for fairness review before sponsor accept/reject decisions can take effect. +- Holds invalid sponsor decision values for fairness review before malformed accept/reject evidence can change solver access. +- Holds missing, duplicate, or malformed applicant identity evidence for fairness review before anonymous or malformed applicant rows can change solver access. +- Holds missing applicant-list evidence for fairness review before malformed challenge rounds can change solver access. +- Holds malformed top-level prequalification-round evidence for fairness review before sparse challenge payloads can change solver access. +- Holds invalid reviewer score values for fairness review before malformed score evidence can drive sponsor decisions. +- Holds duplicate published criterion identifiers for fairness review before ambiguous rubric evidence can drive sponsor decisions. +- Holds whitespace-variant duplicate published criterion identifiers for fairness review before ambiguous rubric evidence can drive sponsor decisions. +- Holds missing or blank published criterion identifiers for fairness review before unauditable rubric evidence can drive sponsor decisions. +- Flags reviewer conflicts, missing, omitted, or blank rejection reason evidence, and invalid appeal-window evidence for arbitration-ready remediation. +- Excludes conflicted reviewer scores from weighted threshold evidence while retaining the conflict finding. +- Deduplicates repeated reviewer identities before quorum and weighted threshold scoring while retaining the duplicate-evidence finding. +- Holds missing reviewer identity evidence before anonymous or malformed reviewer rows can satisfy quorum. +- Holds missing review-list evidence for fairness review before applicant decisions can take effect. +- Holds malformed review-entry evidence for fairness review before applicant decisions can take effect. +- Holds missing criteria-list evidence for fairness review before applicant decisions can take effect. +- Produces deterministic digests for challenge administrators and third-party reviewers. + +## Safety And Scope + +- Synthetic data only. +- No credentials, payment processors, identity providers, sponsor systems, private workspaces, or external APIs. +- This slice is distinct from intake compliance, workspace privacy, clarification freeze, arbitration scoring, payout eligibility, benchmark leakage, sponsor data-room access, and reviewer workload SLA guards. diff --git a/challenge-prequalification-fairness-guard/test.js b/challenge-prequalification-fairness-guard/test.js new file mode 100644 index 00000000..30218d53 --- /dev/null +++ b/challenge-prequalification-fairness-guard/test.js @@ -0,0 +1,1274 @@ +const assert = require('assert'); +const { + evaluatePrequalificationRound, + buildSampleRound +} = require('./index'); + +function byId(items, id) { + return items.find((item) => item.id === id); +} + +function testEligibleApplicantIsAcceptedWithPublishedCriteriaAndQuorum() { + const result = evaluatePrequalificationRound(buildSampleRound()); + const decision = byId(result.decisions, 'applicant-biofoundry'); + + assert.equal(decision.decision, 'accept-prequalified'); + assert.equal(decision.weightedScore, 87); + assert.equal(decision.reviewersCounted, 2); + assert.deepEqual(decision.criteriaApplied, [ + 'domain-fit', + 'data-readiness', + 'safety-plan' + ]); +} + +function testAnonymousScreeningLeakHoldsApplicantForFairnessReview() { + const result = evaluatePrequalificationRound(buildSampleRound()); + const decision = byId(result.decisions, 'applicant-neuro-lab'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('anonymous-screening-leak'), true); + assert.equal(decision.reasons.includes('inconsistent-threshold-decision'), true); + + const action = byId(result.remediationActions, 'remediate-applicant-neuro-lab'); + assert.equal(action.action, 'rerun-blinded-prequalification-review'); + assert.equal(action.priority, 'high'); +} + +function testConflictedAndIncompleteRejectionsStayAuditable() { + const result = evaluatePrequalificationRound(buildSampleRound()); + const decision = byId(result.decisions, 'applicant-sponsor-alumni'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('reviewer-conflict'), true); + assert.equal(decision.reasons.includes('missing-rejection-reason'), true); + assert.equal(decision.appealStatus, 'missing'); +} + +function testConflictedReviewerScoresDoNotInflateWeightedScore() { + const round = buildSampleRound(); + round.minReviewers = 1; + round.applicants = [ + { + id: 'applicant-conflicted-score', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-conflicted-score', + reviewerId: 'reviewer-sponsor-advisor', + anonymousScreeningObserved: true, + conflict: true, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 100, + 'data-readiness': 100, + 'safety-plan': 100 + } + }, + { + applicantId: 'applicant-conflicted-score', + reviewerId: 'reviewer-independent', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['missing independent validation plan'], + scores: { + 'domain-fit': 50, + 'data-readiness': 50, + 'safety-plan': 50 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-conflicted-score'); + + assert.equal(decision.weightedScore, 50); + assert.equal(decision.reasons.includes('reviewer-conflict'), true); + assert.equal(decision.reasons.includes('inconsistent-threshold-decision'), true); +} + +function testHiddenCriteriaAreBlockedBeforeScreeningResultsPublish() { + const round = buildSampleRound(); + round.reviews.push({ + applicantId: 'applicant-biofoundry', + reviewerId: 'reviewer-hidden', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['private sponsor preference'], + scores: { + 'domain-fit': 80, + 'data-readiness': 84, + 'safety-plan': 88, + 'brand-prestige': 15 + } + }); + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-biofoundry'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('unpublished-screening-criterion'), true); +} + +function testExpiredAppealWindowHoldsRejectedApplicantForFairnessReview() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-expired-appeal', + sponsorDecision: 'reject', + rejectionReasons: ['evidence package missed challenge-specific validation'], + appealDueAt: '2026-05-27T08:00:00Z' + } + ]; + round.reviews = [ + { + applicantId: 'applicant-expired-appeal', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['evidence package missed challenge-specific validation'], + scores: { + 'domain-fit': 60, + 'data-readiness': 58, + 'safety-plan': 62 + } + }, + { + applicantId: 'applicant-expired-appeal', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['evidence package missed challenge-specific validation'], + scores: { + 'domain-fit': 62, + 'data-readiness': 57, + 'safety-plan': 61 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-expired-appeal'); + const action = byId(result.remediationActions, 'remediate-applicant-expired-appeal'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.appealStatus, 'expired'); + assert.equal(decision.reasons.includes('expired-appeal-window'), true); + assert.equal(action.action, 'publish-rejection-reasons-and-appeal-window'); +} + +function testInvalidAppealWindowHoldsRejectedApplicantForFairnessReview() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-invalid-appeal', + sponsorDecision: 'reject', + rejectionReasons: ['evidence package missed challenge-specific validation'], + appealDueAt: 'not-a-date' + } + ]; + round.reviews = [ + { + applicantId: 'applicant-invalid-appeal', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['evidence package missed challenge-specific validation'], + scores: { + 'domain-fit': 60, + 'data-readiness': 58, + 'safety-plan': 62 + } + }, + { + applicantId: 'applicant-invalid-appeal', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['evidence package missed challenge-specific validation'], + scores: { + 'domain-fit': 62, + 'data-readiness': 57, + 'safety-plan': 61 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-invalid-appeal'); + const action = byId(result.remediationActions, 'remediate-applicant-invalid-appeal'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.appealStatus, 'invalid'); + assert.equal(decision.reasons.includes('invalid-appeal-window'), true); + assert.equal(action.action, 'publish-rejection-reasons-and-appeal-window'); +} + +function testInvalidCriterionWeightsHoldPrequalificationRound() { + const round = buildSampleRound(); + round.criteria = [ + { + id: 'domain-fit', + label: 'Domain fit for the scientific challenge', + weight: 60 + }, + { + id: 'data-readiness', + label: 'Evidence that required data and tools are ready', + weight: 60 + }, + { + id: 'safety-plan', + label: 'Risk, NDA, and responsible-use plan', + weight: 25 + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-biofoundry'); + const action = byId(result.remediationActions, 'remediate-applicant-biofoundry'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('criteria-weight-total-invalid'), true); + assert.equal(action.action, 'publish-valid-weighted-scoring-rubric'); + assert.equal(action.priority, 'high'); +} + +function testInvalidIndividualCriterionWeightsHoldPrequalificationRound() { + const round = buildSampleRound(); + round.criteria = [ + { + id: 'domain-fit', + label: 'Domain fit for the scientific challenge', + weight: 120 + }, + { + id: 'data-readiness', + label: 'Evidence that required data and tools are ready', + weight: -20 + }, + { + id: 'safety-plan', + label: 'Risk, NDA, and responsible-use plan', + weight: 0 + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-biofoundry'); + const action = byId(result.remediationActions, 'remediate-applicant-biofoundry'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('criteria-weight-value-invalid'), true); + assert.equal(action.action, 'publish-valid-weighted-scoring-rubric'); + assert.equal(action.priority, 'high'); +} + +function testDuplicatePublishedCriterionIdsHoldPrequalificationRound() { + const round = buildSampleRound(); + round.criteria = [ + { + id: 'domain-fit', + label: 'Domain fit for the scientific challenge', + weight: 50 + }, + { + id: 'domain-fit', + label: 'Duplicated sponsor rubric identifier', + weight: 25 + }, + { + id: 'safety-plan', + label: 'Risk, NDA, and responsible-use plan', + weight: 25 + } + ]; + round.applicants = [ + { + id: 'applicant-duplicate-criterion', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-duplicate-criterion', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'safety-plan': 92 + } + }, + { + applicantId: 'applicant-duplicate-criterion', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 96, + 'safety-plan': 94 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-duplicate-criterion'); + const action = byId(result.remediationActions, 'remediate-applicant-duplicate-criterion'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('duplicate-published-criterion'), true); + assert.equal(action.action, 'publish-unique-screening-criteria'); + assert.equal(action.priority, 'high'); +} + +function testWhitespaceVariantCriterionIdsHoldPrequalificationRound() { + const round = buildSampleRound(); + round.criteria = [ + { + id: 'domain-fit', + label: 'Domain fit for the scientific challenge', + weight: 50 + }, + { + id: ' domain-fit ', + label: 'Whitespace-padded duplicate sponsor rubric identifier', + weight: 25 + }, + { + id: 'safety-plan', + label: 'Risk, NDA, and responsible-use plan', + weight: 25 + } + ]; + round.applicants = [ + { + id: 'applicant-normalized-criterion-id', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-normalized-criterion-id', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + ' domain-fit ': 95, + 'safety-plan': 92 + } + }, + { + applicantId: 'applicant-normalized-criterion-id', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 96, + ' domain-fit ': 94, + 'safety-plan': 94 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-normalized-criterion-id'); + const action = byId(result.remediationActions, 'remediate-applicant-normalized-criterion-id'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('duplicate-published-criterion'), true); + assert.equal(action.action, 'publish-unique-screening-criteria'); + assert.equal(action.priority, 'high'); +} + +function testMissingPublishedCriterionIdsHoldPrequalificationRound() { + const round = buildSampleRound(); + round.criteria = [ + { + id: 'domain-fit', + label: 'Domain fit for the scientific challenge', + weight: 50 + }, + { + id: ' ', + label: 'Blank sponsor rubric identifier', + weight: 25 + }, + { + id: 'safety-plan', + label: 'Risk, NDA, and responsible-use plan', + weight: 25 + } + ]; + round.applicants = [ + { + id: 'applicant-missing-criterion-id', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-missing-criterion-id', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + ' ': 95, + 'safety-plan': 92 + } + }, + { + applicantId: 'applicant-missing-criterion-id', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 96, + ' ': 94, + 'safety-plan': 94 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-missing-criterion-id'); + const action = byId(result.remediationActions, 'remediate-applicant-missing-criterion-id'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('missing-published-criterion-id'), true); + assert.equal(action.action, 'publish-complete-screening-criteria'); + assert.equal(action.priority, 'high'); +} + +function testInvalidPassThresholdHoldsPrequalificationRound() { + const round = buildSampleRound(); + round.passThreshold = -5; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-biofoundry'); + const action = byId(result.remediationActions, 'remediate-applicant-biofoundry'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('pass-threshold-invalid'), true); + assert.equal(action.action, 'publish-valid-prequalification-threshold'); + assert.equal(action.priority, 'high'); +} + +function testInvalidReviewerQuorumHoldsPrequalificationRound() { + const round = buildSampleRound(); + round.minReviewers = 0; + round.applicants = [ + { + id: 'applicant-invalid-reviewer-quorum', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-invalid-reviewer-quorum', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 93 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-invalid-reviewer-quorum'); + const action = byId(result.remediationActions, 'remediate-applicant-invalid-reviewer-quorum'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('reviewer-quorum-invalid'), true); + assert.equal(action.action, 'publish-valid-reviewer-quorum'); + assert.equal(action.priority, 'high'); +} + +function testInvalidReviewerScoreValuesHoldPrequalificationRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-invalid-reviewer-score', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-invalid-reviewer-score', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 140, + 'data-readiness': 96, + 'safety-plan': 94 + } + }, + { + applicantId: 'applicant-invalid-reviewer-score', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 93 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-invalid-reviewer-score'); + const action = byId(result.remediationActions, 'remediate-applicant-invalid-reviewer-score'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('reviewer-score-value-invalid'), true); + assert.equal(action.action, 'publish-valid-reviewer-score-evidence'); + assert.equal(action.priority, 'high'); +} + +function testInvalidSponsorDecisionHoldsPrequalificationRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-invalid-sponsor-decision', + sponsorDecision: 'waitlist', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-invalid-sponsor-decision', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 94 + } + }, + { + applicantId: 'applicant-invalid-sponsor-decision', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 93 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-invalid-sponsor-decision'); + const action = byId(result.remediationActions, 'remediate-applicant-invalid-sponsor-decision'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('sponsor-decision-invalid'), true); + assert.equal(action.action, 'publish-valid-sponsor-decision'); + assert.equal(action.priority, 'high'); +} + +function testMissingApplicantIdentityHoldsPrequalificationRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: ' ', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: ' ', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 94 + } + }, + { + applicantId: ' ', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 93 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'unidentified-applicant'); + const action = byId(result.remediationActions, 'remediate-unidentified-applicant'); + + assert.equal(decision.applicantId, 'unidentified-applicant'); + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('missing-applicant-identity'), true); + assert.equal(action.action, 'complete-prequalification-evidence'); + assert.equal(action.priority, 'high'); +} + +function testDuplicateApplicantIdentitiesHoldPrequalificationRound() { + const round = buildSampleRound(); + round.applicants = [ + { + id: ' applicant-duplicate ', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + }, + { + id: 'applicant-duplicate', + sponsorDecision: 'reject', + rejectionReasons: ['duplicate entry should be resolved before screening'], + appealDueAt: '2026-06-04T08:00:00Z' + } + ]; + round.reviews = [ + { + applicantId: 'applicant-duplicate', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 94 + } + }, + { + applicantId: 'applicant-duplicate', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 93 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decisions = result.decisions.filter( + (decision) => decision.applicantId === 'applicant-duplicate' + ); + const action = byId(result.remediationActions, 'remediate-applicant-duplicate'); + + assert.equal(decisions.length, 2); + assert.equal(decisions.every((decision) => decision.decision === 'hold-for-fairness-review'), true); + assert.equal(decisions.every((decision) => decision.reasons.includes('duplicate-applicant-identity')), true); + assert.equal(action.action, 'complete-prequalification-evidence'); + assert.equal(action.priority, 'high'); +} + +function testMalformedApplicantEntryHoldsPrequalificationRoundWithoutCrashing() { + const round = buildSampleRound(); + round.applicants = [null]; + round.reviews = []; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'unidentified-applicant'); + const action = byId(result.remediationActions, 'remediate-unidentified-applicant'); + + assert.equal(decision.applicantId, 'unidentified-applicant'); + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('malformed-applicant-entry'), true); + assert.equal(action.action, 'complete-prequalification-evidence'); + assert.equal(action.priority, 'high'); +} + +function testMissingRejectionReasonListHoldsWithoutCrashing() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-missing-rejection-list', + sponsorDecision: 'reject', + appealDueAt: '2026-06-04T08:00:00Z' + } + ]; + round.reviews = [ + { + applicantId: 'applicant-missing-rejection-list', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['insufficient validation plan'], + scores: { + 'domain-fit': 58, + 'data-readiness': 60, + 'safety-plan': 62 + } + }, + { + applicantId: 'applicant-missing-rejection-list', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['insufficient validation plan'], + scores: { + 'domain-fit': 59, + 'data-readiness': 61, + 'safety-plan': 60 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-missing-rejection-list'); + const action = byId(result.remediationActions, 'remediate-applicant-missing-rejection-list'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.deepEqual(decision.rejectionReasons, []); + assert.equal(decision.reasons.includes('missing-rejection-reason'), true); + assert.equal(action.action, 'publish-rejection-reasons-and-appeal-window'); +} + +function testBlankRejectionReasonTextHoldsRejectedApplicant() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-blank-rejection-reason', + sponsorDecision: 'reject', + rejectionReasons: [' '], + appealDueAt: '2026-06-04T08:00:00Z' + } + ]; + round.reviews = [ + { + applicantId: 'applicant-blank-rejection-reason', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['insufficient validation plan'], + scores: { + 'domain-fit': 58, + 'data-readiness': 60, + 'safety-plan': 62 + } + }, + { + applicantId: 'applicant-blank-rejection-reason', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'reject', + rejectionReasons: ['insufficient validation plan'], + scores: { + 'domain-fit': 59, + 'data-readiness': 61, + 'safety-plan': 60 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-blank-rejection-reason'); + const action = byId(result.remediationActions, 'remediate-applicant-blank-rejection-reason'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.deepEqual(decision.rejectionReasons, []); + assert.equal(decision.reasons.includes('missing-rejection-reason'), true); + assert.equal(action.action, 'publish-rejection-reasons-and-appeal-window'); +} + +function testIncompleteReviewerScoreEvidenceHoldsWithoutCrashing() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-incomplete-review-evidence', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-incomplete-review-evidence', + reviewerId: 'reviewer-incomplete', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [] + }, + { + applicantId: 'applicant-incomplete-review-evidence', + reviewerId: 'reviewer-partial', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 90 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-incomplete-review-evidence'); + const action = byId(result.remediationActions, 'remediate-applicant-incomplete-review-evidence'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.weightedScore, 36); + assert.equal(decision.reasons.includes('missing-published-criterion-score'), true); + assert.equal(action.action, 'complete-prequalification-evidence'); +} + +function testMissingReviewListHoldsPrequalificationRoundWithoutCrashing() { + const round = buildSampleRound(); + delete round.reviews; + round.applicants = [ + { + id: 'applicant-missing-review-list', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-missing-review-list'); + const action = byId(result.remediationActions, 'remediate-applicant-missing-review-list'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('missing-review-list'), true); + assert.equal(decision.reviewersCounted, 0); + assert.equal(action.action, 'complete-prequalification-evidence'); + assert.equal(action.priority, 'high'); +} + +function testMalformedReviewEntryHoldsPrequalificationRoundWithoutCrashing() { + const round = buildSampleRound(); + round.applicants = [ + { + id: 'applicant-malformed-review-entry', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + null, + { + applicantId: 'applicant-malformed-review-entry', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 93 + } + }, + { + applicantId: 'applicant-malformed-review-entry', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 91 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-malformed-review-entry'); + const action = byId(result.remediationActions, 'remediate-applicant-malformed-review-entry'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reviewersCounted, 2); + assert.equal(decision.reasons.includes('malformed-review-entry'), true); + assert.equal(action.action, 'complete-prequalification-evidence'); + assert.equal(action.priority, 'high'); +} + +function testMissingCriteriaListHoldsPrequalificationRoundWithoutCrashing() { + const round = buildSampleRound(); + delete round.criteria; + round.applicants = [ + { + id: 'applicant-missing-criteria-list', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-missing-criteria-list', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 93 + } + }, + { + applicantId: 'applicant-missing-criteria-list', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 91 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-missing-criteria-list'); + const action = byId(result.remediationActions, 'remediate-applicant-missing-criteria-list'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('missing-published-criteria-list'), true); + assert.deepEqual(decision.criteriaApplied, []); + assert.equal(decision.weightedScore, 0); + assert.equal(action.action, 'publish-complete-screening-criteria'); + assert.equal(action.priority, 'high'); + assert.ok(result.criteriaDigest.startsWith('sha256:')); +} + +function testMalformedCriterionEntryHoldsPrequalificationRoundWithoutCrashing() { + const round = buildSampleRound(); + round.criteria = [null]; + round.applicants = [ + { + id: 'applicant-malformed-criterion-entry', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-malformed-criterion-entry'); + const action = byId(result.remediationActions, 'remediate-applicant-malformed-criterion-entry'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('malformed-published-criterion-entry'), true); + assert.equal(action.action, 'publish-complete-screening-criteria'); + assert.equal(action.priority, 'high'); + assert.ok(result.criteriaDigest.startsWith('sha256:')); +} + +function testMissingApplicantListHoldsPrequalificationRoundWithoutCrashing() { + const round = buildSampleRound(); + delete round.applicants; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'unidentified-applicant'); + const action = byId(result.remediationActions, 'remediate-unidentified-applicant'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('missing-applicant-list'), true); + assert.deepEqual(decision.rejectionReasons, []); + assert.equal(action.action, 'complete-prequalification-evidence'); + assert.equal(action.priority, 'high'); + assert.equal(result.summary.held, 1); +} + +function testMalformedPrequalificationRoundHoldsWithoutCrashing() { + const result = evaluatePrequalificationRound(null); + const decision = byId(result.decisions, 'unidentified-applicant'); + const action = byId(result.remediationActions, 'remediate-unidentified-applicant'); + + assert.equal(result.challengeId, 'unidentified-challenge'); + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('malformed-prequalification-round'), true); + assert.equal(decision.reasons.includes('missing-published-criteria-list'), true); + assert.equal(decision.reasons.includes('missing-review-list'), true); + assert.equal(decision.reasons.includes('missing-applicant-list'), true); + assert.deepEqual(decision.criteriaApplied, []); + assert.equal(action.action, 'complete-prequalification-evidence'); + assert.equal(action.priority, 'high'); + assert.equal(result.summary.held, 1); +} + +function testMissingChallengeIdentityHoldsPrequalificationRound() { + const round = buildSampleRound(); + round.challengeId = ' '; + round.applicants = [ + { + id: 'applicant-missing-challenge-identity', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-missing-challenge-identity', + reviewerId: 'reviewer-independent-a', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 94, + 'data-readiness': 96, + 'safety-plan': 93 + } + }, + { + applicantId: 'applicant-missing-challenge-identity', + reviewerId: 'reviewer-independent-b', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 94, + 'safety-plan': 91 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-missing-challenge-identity'); + const action = byId(result.remediationActions, 'remediate-applicant-missing-challenge-identity'); + + assert.equal(result.challengeId, 'unidentified-challenge'); + assert.equal(decision.challengeId, 'unidentified-challenge'); + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reasons.includes('missing-challenge-identity'), true); + assert.equal(action.action, 'complete-challenge-identity-evidence'); + assert.equal(action.priority, 'high'); +} + +function testDuplicateReviewerScoreEvidenceDoesNotSatisfyQuorum() { + const round = buildSampleRound(); + round.minReviewers = 2; + round.applicants = [ + { + id: 'applicant-duplicate-reviewer', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-duplicate-reviewer', + reviewerId: 'reviewer-repeat', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 96, + 'data-readiness': 94, + 'safety-plan': 95 + } + }, + { + applicantId: 'applicant-duplicate-reviewer', + reviewerId: 'reviewer-repeat', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 93, + 'safety-plan': 94 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-duplicate-reviewer'); + const action = byId(result.remediationActions, 'remediate-applicant-duplicate-reviewer'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reviewersCounted, 1); + assert.equal(decision.reasons.includes('duplicate-reviewer-score-evidence'), true); + assert.equal(decision.reasons.includes('reviewer-quorum-shortfall'), true); + assert.equal(action.action, 'deduplicate-reviewer-score-evidence'); + assert.equal(action.priority, 'high'); +} + +function testMissingReviewerIdentityDoesNotSatisfyQuorum() { + const round = buildSampleRound(); + round.minReviewers = 2; + round.applicants = [ + { + id: 'applicant-missing-reviewer-identity', + sponsorDecision: 'accept', + rejectionReasons: [], + appealDueAt: null + } + ]; + round.reviews = [ + { + applicantId: 'applicant-missing-reviewer-identity', + reviewerId: ' ', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 96, + 'data-readiness': 94, + 'safety-plan': 95 + } + }, + { + applicantId: 'applicant-missing-reviewer-identity', + anonymousScreeningObserved: true, + conflict: false, + recommendedDecision: 'accept', + rejectionReasons: [], + scores: { + 'domain-fit': 92, + 'data-readiness': 93, + 'safety-plan': 94 + } + } + ]; + + const result = evaluatePrequalificationRound(round); + const decision = byId(result.decisions, 'applicant-missing-reviewer-identity'); + const action = byId(result.remediationActions, 'remediate-applicant-missing-reviewer-identity'); + + assert.equal(decision.decision, 'hold-for-fairness-review'); + assert.equal(decision.reviewersCounted, 0); + assert.equal(decision.reasons.includes('missing-reviewer-identity'), true); + assert.equal(decision.reasons.includes('reviewer-quorum-shortfall'), true); + assert.equal(action.action, 'complete-prequalification-evidence'); +} + +function testAuditDigestIsDeterministicAndPrivateFree() { + const first = evaluatePrequalificationRound(buildSampleRound()); + const second = evaluatePrequalificationRound(buildSampleRound()); + + assert.equal(first.auditDigest, second.auditDigest); + assert.ok(first.auditDigest.startsWith('sha256:')); + assert.equal(first.summary.accepted, 1); + assert.equal(first.summary.held, 3); + assert.equal(JSON.stringify(first).includes('private@'), false); + assert.equal(JSON.stringify(first).includes('government_id'), false); +} + +const tests = [ + testEligibleApplicantIsAcceptedWithPublishedCriteriaAndQuorum, + testAnonymousScreeningLeakHoldsApplicantForFairnessReview, + testConflictedAndIncompleteRejectionsStayAuditable, + testConflictedReviewerScoresDoNotInflateWeightedScore, + testHiddenCriteriaAreBlockedBeforeScreeningResultsPublish, + testExpiredAppealWindowHoldsRejectedApplicantForFairnessReview, + testInvalidAppealWindowHoldsRejectedApplicantForFairnessReview, + testInvalidCriterionWeightsHoldPrequalificationRound, + testInvalidIndividualCriterionWeightsHoldPrequalificationRound, + testDuplicatePublishedCriterionIdsHoldPrequalificationRound, + testWhitespaceVariantCriterionIdsHoldPrequalificationRound, + testMissingPublishedCriterionIdsHoldPrequalificationRound, + testInvalidPassThresholdHoldsPrequalificationRound, + testInvalidReviewerQuorumHoldsPrequalificationRound, + testInvalidReviewerScoreValuesHoldPrequalificationRound, + testInvalidSponsorDecisionHoldsPrequalificationRound, + testMissingApplicantIdentityHoldsPrequalificationRound, + testDuplicateApplicantIdentitiesHoldPrequalificationRound, + testMalformedApplicantEntryHoldsPrequalificationRoundWithoutCrashing, + testMissingRejectionReasonListHoldsWithoutCrashing, + testBlankRejectionReasonTextHoldsRejectedApplicant, + testIncompleteReviewerScoreEvidenceHoldsWithoutCrashing, + testMissingReviewListHoldsPrequalificationRoundWithoutCrashing, + testMalformedReviewEntryHoldsPrequalificationRoundWithoutCrashing, + testMissingCriteriaListHoldsPrequalificationRoundWithoutCrashing, + testMalformedCriterionEntryHoldsPrequalificationRoundWithoutCrashing, + testMissingApplicantListHoldsPrequalificationRoundWithoutCrashing, + testMalformedPrequalificationRoundHoldsWithoutCrashing, + testMissingChallengeIdentityHoldsPrequalificationRound, + testDuplicateReviewerScoreEvidenceDoesNotSatisfyQuorum, + testMissingReviewerIdentityDoesNotSatisfyQuorum, + testAuditDigestIsDeterministicAndPrivateFree +]; + +for (const test of tests) { + test(); +} + +console.log(`${tests.length} challenge prequalification fairness tests passed`);