Skip to content

Keep closeout item decisions consistent through publication - #174

Merged
unbraind merged 15 commits into
mainfrom
fix/closeout-delivery-attribution
Oct 24, 2026
Merged

unbraind merged 15 commits into
mainfrom
fix/closeout-delivery-attribution

Conversation

@unbraind

@unbraind unbraind commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

A fully inspected mention-only PR with failing CI could hold a unique valid delivery because every linked PR entered attribution. Candidacy now requires an explicit delivery tag, a closing declaration referencing the item, or a commit claim naming it. deliveryRole classifies candidates, unresolved claim evidence and context in one place. Context stays listed as informational skip / not_delivery in the CLI and publication summary; its evidence gaps never hold the item. Missing evidence on a candidate, or unreadable PR/commit evidence that may hide a claim, still holds. Revision binding and existing selection policies remain in place.

Before: PR 1 claims delivery and passes; PR 2 only mentions the item and fails CI; both report needs_review. After: PR 1 closes and publishes with its own provenance; PR 2 remains context without candidate numbers or missing-evidence holds.

Validation:

  • The publication decision table retains all 36 earlier selection/evidence/peer rows and adds all 25 ordered primary/competitor claim-kind pairs across six evidence states: 186/186 rows passed, zero failures/cancellations/skips. Claim kinds are closing keyword, delivery tag, commit claim, mention-only and link-only. Every row asserts its exact chosen PR, candidate/context sets, closure/history/provenance, publication outcome and public summary. The two earlier mention-only/unmatched-selector rows keep their original inputs and now expect context instead of an invented candidate hold; a peer edit alone stays local. Verified totals: 71 closures, 87 holds, 28 context-only runs without publication.
  • fully inspected context cannot hold a unique delivery: <gap> covers six merge/check gaps in real non-git trackers. fully inspected mentions are context even without any delivery candidate covers context-only reporting. Existing attribution regressions now keep mentions outside ambiguous candidate sets.
  • Fix-only revert: replace only closeout.ts with pre-round head 4dd53552f825b3b42959ae96b9e61b27fe96e900, retaining tests and workflow changes. node --test --test-name-pattern='fully inspected context|fully inspected mentions' test/closeout-selection.test.ts fails 8/8 assertions, exit 1. Restore byte-for-byte: the PM runner passes 8/8, exit 0, zero failures/cancellations/skips. The PM comment retains the receipt.
  • Local typecheck, test typecheck, lint, docstrings, zero duplication, privacy, canonical changelog and isolated strict tracker health pass. Full npm/Bun release-gate results on the pushed head are recorded below. Dependency pins, coverage source inventory and thresholds are unchanged.

PM: Closeout attribution. The item remains in progress for orchestrator verification. Bounded coverage aggregation remains existing follow-up work on pm-github-vhi6.

The PM-linked focused c8 command across test/closeout.test.ts, test/closeout-attribution.test.ts, test/closeout-selection.test.ts and test/closeout-revision.test.ts passes 294/294 tests, exact 100/100/100/100 for closeout.ts, with zero failures, cancellations or skips.

Final-head CI passed 4/4 jobs on 5851c1c3af3b9ebd10944dcf5e60f292824d5f97. npm run release:check and bun run release:check each passed 1,495/1,495 tests on Node 22/26, zero failures/cancellations/skips/TODOs, exact 100/100/100/100 across all 19 authored modules. Native Bun 1.3.14 acceptance passed 1,495/1,495 tests across 52 files on each Node version, zero failures. All other release stages passed with unchanged pins, source inventory and thresholds. Both final npm logs retain the 186 publication rows and the exact PR-1-delivery/PR-2-context reproduction. Final-head CodeQL passed too.

@sourcery-ai

sourcery-ai Bot commented Oct 24, 2026

Copy link
Copy Markdown

Sorry @unbraind, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 3 days and 8 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@coderabbitai

coderabbitai Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 2ad2af3a-3969-455b-847c-ec11e7a293e2
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Oct 24, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR changes closeout from silently selecting a latest merged delivery to fail-closed attribution: an item-owned canonical delivery tag takes precedence, otherwise exactly one verified delivery claim must occur after the last acceptance-criteria change. Ambiguous, unverifiable, or competing claims remain open with all candidates reported; the workflow can publish a privacy-gated review artifact and disables automatic landing while dispatching caller checks.

Sequence diagram for ambiguous closeout review handling

sequenceDiagram
    participant Workflow
    participant Closeout as runCloseout
    participant History as SDK history
    participant GitHub
    participant Tracker

    Workflow->>Closeout: runCloseout()
    Closeout->>GitHub: Read merged PRs and commit messages
    Closeout->>History: readHistoryEntries()
    History-->>Closeout: Acceptance-criteria cutoff
    Closeout->>Closeout: Select explicit tag or unique post-cutoff claim
    Closeout-->>Workflow: needs_review with candidates
    Workflow->>Tracker: Write closeout-review.json
    Workflow->>Workflow: dispatch_checks()
    Workflow-->>Workflow: Disable automatic landing
Loading

Flow diagram for acceptance-aware closeout attribution

flowchart TD
    A[Collect linked merged PRs] --> B{Explicit delivery-pr tag?}
    B -->|Yes| C{Exactly one tag matches a candidate?}
    B -->|No| D[Read verified SDK history]
    D --> E{Exactly one delivery claim after criteria cutoff?}
    C -->|Yes| F[Verify merge facts and head checks]
    C -->|No| G[needs_review with every candidate]
    E -->|Yes| F
    E -->|No| G
    F -->|Pass| H[Close item using selected merge evidence]
    F -->|Fail| I[Keep item open]
Loading

File-Level Changes

Change Details Files
Reworked closeout attribution to fail closed on ambiguous delivery claims and use acceptance-aware chronology.
  • Added item-owned delivery-pr URL tags as explicit attribution overrides.
  • Require exactly one declared claim after the last verified acceptance-criteria change when no override exists.
  • Read and validate SDK history, including commit-message delivery claims, and hold cases with missing or competing evidence.
  • Preserve mandatory merge and head-check gates and use the selected PR’s merge facts for completion metadata.
  • Emit needs_review rows containing all candidate PRs without mutating the item or its history.
closeout.ts
README.md
docs/explicit-closeout-attribution.md
Updated the reusable workflow to publish reviewable attribution holds without automatically landing them.
  • Persisted a privacy-filtered closeout-review.json report for held items.
  • Included candidates and review status in the closeout summary and PR body.
  • Dispatched caller checks while disabling automatic merge when attribution requires review.
  • Added review-only workflow coverage for all-held runs and candidate reporting.
.github/workflows/pm-github-closeout.yml
test/closeout-workflow.test.ts
Expanded regression coverage for explicit attribution, acceptance chronology, ambiguity, evidence failures, and durable completion facts.
  • Added real SDK tracker tests that run with git unavailable and verify mutation and history behavior.
  • Replaced latest-merge and PR-number winner expectations with ambiguity-holds.
  • Covered commit claims, invalid or unavailable history, explicit overrides, failing checks, and exact completion evidence.
  • Updated existing fixtures and tests for commit API requests, new report actions, and canonical metadata.
test/closeout-selection.test.ts
test/closeout-attribution.test.ts
test/closeout.test.ts
Recorded the feature’s PM tracking artifacts and changelog entry.
  • Added the PM issue and append-only history receipt.
  • Documented validation and attribution regression evidence.
  • Added the closeout behavior change to the changelog.
.agents/pm/issues/pm-github-closeout-attribution.toon
.agents/pm/history/pm-github-closeout-attribution.jsonl
CHANGELOG.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

Copy link
Copy Markdown
Owner Author

Additional consumer provenance evidence: pm-jev#48 (older pinned caller)

Static inspection of pm-jev#48, immutable head 09b76d08a23b6e3fca735e91788f1844cfafe678, finds a concrete instance of the attribution problem this work addresses. The proposed pm-jev-w1zj closeout records PR24 in resolution, close_reason, completed_at and appended close history. PR24 explicitly says partial failure receipts remain an open follow-up; merged PR45 implements interruption receipts and links that item. Thus closure may now be justified, but PR24 is not truthful delivery provenance.

Version boundary: the immutable caller workflow pins pm-github 2026.10.16 and reusable workflow 67a5e3e1514fddeeae29951da00513f06f7aa0ab. Installed runtime version was not independently verified. This is not evidence that latest #174 head bf17864 regresses. Bounded isolated review of that head separately passed 73 attribution/selection tests, 10 synthetic workflow auto-merge revocation cases, typecheck and build; full hosted checks remain pending.

This matters to #159's agreed delivery-attribution rule: completion fields and history should identify the delivering PR, not a mention-only earlier PR. All-status owners and the current conversation were checked; this adds a new consumer example rather than a duplicate issue.

Acceptance/regression: use this sanitized two-PR pattern to ensure mention-only PR24 cannot win and actual delivery PR45 is consistently credited; retain the correctly attributed Unicode/PR47 item as a control. Any consumer repair should preserve durable history and existing evidence/check gates. No tracker edit, publication, provider mutation or special-file reproduction was performed.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@unbraind unbraind changed the title Hold ambiguous closeouts until the delivering PR is verified Bind closeout delivery decisions to exact item revisions Oct 24, 2026

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@unbraind unbraind changed the title Bind closeout delivery decisions to exact item revisions Keep closeout item decisions consistent through publication Oct 24, 2026

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@unbraind unbraind left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orchestrator review (built-in Claude review agent, verified by the orchestrator; bots are not waited for since 2026-10-22):

Pre-merge review (codex gpt-6.1-sol read-only review agent; merged by the orchestrator's pipeline after a MERGE verdict and a private-data scan of the added lines).

VERDICT: MERGE
FINDINGS:

  • none
    SUMMARY: The PR keeps delivery attribution, revision checks, and publication decisions consistent while preserving informational context. Tests cover non-git trackers, incomplete evidence, concurrent edits, and publication outcomes; I reviewed them statically without rerunning them.

@unbraind
unbraind merged commit 1ec3a76 into main Oct 24, 2026
11 checks passed
@unbraind
unbraind deleted the fix/closeout-delivery-attribution branch October 24, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant