Skip to content

discovery research gate passes an artifact with zero accepted claims without saying so #5833

Description

@kyle-sexton

Summary

After #5827, /discovery:research outcome-gate row 7 reads: "Every accepted claim is HIGH confidence; a MEDIUM or LOW claim listed in the Gaps section is not accepted". Rows 7 and 12 are both quantified over accepted claims. An artifact that lists every claim as a Gap therefore passes both rows with zero accepted claims, and nothing in the gate or the artifact tells the reader that the run settled nothing.

Row 4 also says "Every claim has ≥2 INDEPENDENT current corroborators". The intent is "every accepted claim", because a Gap claim is by definition not corroborated. As written, row 4 contradicts the Gap route that rows 7 and 12 allow.

Evidence (PR #5827 head)

  • plugins/discovery/skills/research/SKILL.md:75, row 4: "Every claim has ≥2 INDEPENDENT current corroborators ..."
  • plugins/discovery/skills/research/SKILL.md:78, row 7: "Every accepted claim is HIGH confidence; a MEDIUM or LOW claim listed in the Gaps section is not accepted"
  • plugins/discovery/skills/research/SKILL.md:83, row 12: "Every accepted claim follows jointly from its cited sources ..."
  • No row requires at least one accepted claim, and none requires the artifact to state that it accepted zero claims.

Related, not a duplicate: #5805 covers the acceptance-bar wording and single-publisher facts.

Design question

Pick one of these for an artifact with zero accepted claims:

  • (a) A gate row that fails it.
  • (b) A row that requires a verdict line in the artifact (for example accepted: 0 — inconclusive) and leaves it passing.
  • (c) An explicit "inconclusive" outcome distinct from pass and fail.

Recommendation: (b). Basis: judgment. A research run that honestly finds nothing is a valid result. A run that does not tell the reader it found nothing is the defect.

Acceptance criteria

  • Row 4 is quantified over accepted claims, or its wording is reconciled with the Gap route.
  • The gate treats an artifact with zero accepted claims according to the chosen option, and research-verifier.md grades it.
  • A contract.test.sh pin and an eval case cover an artifact whose every claim is a Gap.
  • plugins/discovery is version-bumped and has a CHANGELOG entry.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageNot yet classified. Floor until a type and one priority tier are set.priority: mediumReal value, no hard deadline; normal backlog flow.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions