diff --git a/AGENTS.md b/AGENTS.md index fc9f755..bd99c5f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,6 +31,9 @@ Read these files before editing: but are not product behavior. - Keep the builder unable to push, merge, deploy, or rewrite its own acceptance oracle. The shipper may push only an already committed, verified candidate. +- Keep forge credentials behind the attended shipper boundary. Persist external + intent before a call, treat provider readback as authoritative, and block + mutation while an effect outcome is unknown. - Do not claim hostile-code containment for attended host execution. - No ambient credentials, implicit network, auto-merge, deployment, parallel writers, daemon, hosted control plane, or self-modification in v1. @@ -67,8 +70,8 @@ iteration, but skipped required checks block promotion. - Default branch: `main`. - Work branches use `codex/` unless a task says otherwise. -- Required checks and exact-head local review must settle before David marks a - draft ready and merges it. +- Required checks and exact-head local review must settle before a human marks a + draft ready and David, the configured merger, merges it. - Use conventional commits and DCO sign-off. - Releases come only from immutable tags through trusted npm publishing with provenance. The first release follows the genesis qualification protocol. diff --git a/CHANGELOG.md b/CHANGELOG.md index 5c6e438..0b23185 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,9 +19,14 @@ All notable changes follow Keep a Changelog and Semantic Versioning. and exact-candidate read-only Codex review. - Repository-namespaced SQLite state with append-only events, exclusive writer leases, cancellation, interruption recovery, backup/validated restore, - terminal-only purge, and redacted support export. + reviewed-or-terminal purge, and redacted support export. - Executable task-packet, context-manifest, validation-evidence, and review-result contracts with aligned runtime and JSON Schema validation. +- Exact-candidate draft-PR planning, attended push/open, unknown-effect + reconciliation, paginated GitHub policy observation, human merge gating, and + exact post-merge closure with an executable delivery-record contract. +- Coordinator-enforced remote attendance, one readback-authorized effect retry, + changing-blocker replacement, and database-swap state-restore recovery. ### Changed @@ -67,6 +72,9 @@ All notable changes follow Keep a Changelog and Semantic Versioning. active attempts only through compare-and-swap. - Named OCI verifier containers are force-removed under an independent cleanup deadline after success, failure, timeout, output exhaustion, or cancellation. +- OCI verification safely aliases comma-bearing bind paths, and restoring an + older state backup quarantines newer unreferenced worktrees with durable + recovery evidence rather than deleting them. ### Security @@ -86,3 +94,10 @@ All notable changes follow Keep a Changelog and Semantic Versioning. - Codex invocations ignore ambient execution rules, and OCI verification uses a clean read-only candidate workspace so uncommitted ignored artifacts cannot affect promotion evidence. +- GitHub credentials stay behind the operator-owned `gh` and Git credential + helper boundary. Exact actor/repository/remote binding, expiring approval, + expected-head push leases, immutable PR markers, effect journaling, and + authoritative readback prevent builder access and blind duplicate mutation. + Durable cancellation is polled before and during mutations; top-level and + inline review findings, exact merger identity, provable merge shape, and + non-false-green post-merge results remain fail-closed. diff --git a/README.md b/README.md index ebda6fe..7b96c54 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,11 @@ approved product outcome into a tested, reviewed draft pull request. The project is pre-alpha. Wave 1 provides the installable source package, compact schemas, static PRD/repository inspection, and readiness diagnostics. Wave 2 adds an attended local path from one explicit task approval to an exact -committed, OCI-validated, independently reviewed candidate. The CLI is -`millctl`, published eventually as `@davidahmann/mill` to avoid collision with -the existing `mill` command and npm package. +committed, OCI-validated, independently reviewed candidate. Wave 3 adds an +attended, exact-candidate path to one draft GitHub pull request, bounded +CI/review observation, human merge, and truthful closure. The CLI is `millctl`, +published eventually as `@davidahmann/mill` to avoid collision with the existing +`mill` command and npm package. Mill's v1 boundary is deliberately narrow: @@ -80,6 +82,61 @@ transient or invalid provider review; the durable per-candidate attempt budget prevents an unbounded token loop while still allowing the one reviewed repair generation. +## Open one attended draft pull request + +The downstream repository must explicitly raise `trustCeiling` to `propose` and +bind its immutable GitHub repository node ID, target branch, accepted operators, +required checks, review policy, and allowed human merge methods. Mill reads the +live actor, repository, remote, and default branch before it returns an approval +digest. That plan performs no remote mutation. The separate `pr open` command +requires the exact unexpired digest and an attended operator: + +```yaml +trustCeiling: propose +propose: + forge: github + host: github.com + owner: example + repository: app + repositoryNodeId: R_kgDOExample + remoteName: origin + baseBranch: main + branchPrefix: mill/ + allowedActors: [founder] + allowedMergerLogins: [founder] + requiredChecks: [validate, CodeQL] + reviewPolicy: + mode: local_only + requiredReviewerLogins: [] + allowedMergeMethods: [linear_tree_preserving] +``` + +```sh +node dist/cli.js --json pr plan --task product/tasks/TASK.yaml --run +node dist/cli.js --json pr open --task product/tasks/TASK.yaml --run \ + --approve sha256: --attended +node dist/cli.js --json pr observe --task product/tasks/TASK.yaml --run +# A human may mark ready; a configured merger merges in GitHub. +node dist/cli.js --json pr finalize --task product/tasks/TASK.yaml --run +``` + +Only the shipper reads the operator-owned `gh` session. Builder and reviewer +processes receive neither GitHub credentials nor mutation tools. Mill journals +intent before push and PR creation, uses an expected-old-head lease, and reads +GitHub back before claiming an effect. An uncertain outcome becomes +`effect_unknown`; `pr reconcile` is read-only and must classify it before any +retry. Exact readback proving absence authorizes one retry; a second absent +outcome blocks. Required checks pass only when every latest exact-head result is +successful. A configured `github_required` reviewer may complete a current-head +`APPROVED` or `COMMENTED` review, but any current-head actionable finding still +blocks, including a severity-tagged top-level review body. Mill stops at +`awaiting_human`; draft readiness is not closure authority and Mill never +changes it or merges. Finalization verifies the recorded merger against +`allowedMergerLogins`. Because GitHub does not expose an authoritative +distinction between a one-commit squash and rebase, the provable policy is +`linear_tree_preserving`; Mill never guesses a specific linear method from its +allowlist. + Use `--json` before the command for the stable machine-readable envelope. `--json --version` is machine-readable; help is human-only and combining it with `--json` returns a typed usage error. `doctor` and static adoption never execute @@ -124,9 +181,10 @@ the repository. ## Status -Not published. Local attended delivery is implemented, but no GitHub mutation, -hostile-host containment, release, or generalized stack-compatibility claim -exists until its corresponding later-wave canary passes. +Not published. Local attended delivery and the bounded draft-PR lifecycle are +implemented and covered by fake-provider and packed-package canaries. The first +attended disposable real-GitHub canary, hostile-host containment, release, and +generalized stack-compatibility claims remain pending their explicit gates. ## License diff --git a/WORKFLOW.md b/WORKFLOW.md index 93f3c2f..916daf6 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -21,7 +21,8 @@ For each wave: 6. Run a complete exact-candidate local review and one systemic repair wave if necessary. 7. Push the unchanged candidate, open/update one PR, and observe required CI. -8. David marks ready and merges. Observe the resulting main commit and checks. +8. A human may mark the draft ready; David, the configured merger, merges it. + Observe the resulting main commit and checks. Current Factory skills are optional maintainer-side bootstrap tools. Their prompts, profiles, artifacts, or state are not Mill runtime or product diff --git a/architecture/ARCHITECTURE.md b/architecture/ARCHITECTURE.md index 3176c7c..f208738 100644 --- a/architecture/ARCHITECTURE.md +++ b/architecture/ARCHITECTURE.md @@ -1,6 +1,6 @@ # Mill architecture -Status: approved foundation decision Last updated: 2026-08-31 +Status: approved v1 decision Last updated: 2026-09-01 ## Form @@ -11,7 +11,7 @@ and exits to resumable state for long waits—there is no daemon. ## Boundaries -The implemented Wave 2 boundary is: +The implemented Wave 3 boundary is: ```text exact human-authored task + product/scenario/policy digests @@ -22,6 +22,11 @@ exact human-authored task + product/scenario/policy digests -> selected digest-pinned OCI commands without network -> fresh read-only Codex review of the exact verified commit -> reviewed local candidate or one repair-and-revalidate cycle + -> exact actor/repository/remote proposal digest + -> expected-head push + immutable-marker draft PR + -> exact-head CI and optional GitHub-review observation + -> human readiness and merge + -> exact merge/tree/default-branch check readback and closure ``` The complete planned v1 boundary extends that path: @@ -45,6 +50,36 @@ The builder never receives forge/deployment authority. The shipper cannot create or amend the candidate commit. Product/oracle changes invalidate the candidate. Provider state is authoritative for external effects. +The GitHub adapter is isolated behind the delivery coordinator. Planning reads +the live delegated actor, repository node identity, clone URL, fork status and +default branch, then binds them with the candidate commit/tree, task/config, +branch, required checks, review policy, allowed merge methods, approval expiry, +and intended effects. Only `pr open` mutates. Its effect journal records intent +and call start before each push or PR request, caps each effect at two attempts, +and makes ambiguous results enter `effect_unknown`. Reconciliation performs +authoritative branch/marker/PR readback without mutation. Exact absence permits +one retry; a second absent outcome blocks for human disposition. The same +recorded PR number, node identity, marker, branch, base, open-draft state, and +observed head are invariant whether an ambiguous repair push is absent or +landed. A retry performs that check again from a fresh readback immediately +before recording call start and invoking Git. GitHub API collections are +paginated under one deadline and output budget. Tokens remain behind the +operator-owned `gh` and Git credential-helper boundary and are not passed to +Codex or stored in state. + +One stable delivery key and branch identify the PR across the single allowed +repair. A new candidate gets new validation, review, approval, and push-effect +identity while updating that same PR. Required checks are evaluated on the exact +current head; missing, pending, conflicting, cancelled, neutral, skipped, +timed-out, or failed results do not pass. Mill never changes draft readiness or +merge state, and readiness is not treated as closure authority. Finalization +requires GitHub to prove the PR head, merge commit, tree, authorized merger +identity, containment in the configured default branch, allowed merge shape, and +successful required checks on the exact merge commit. One-parent tree-preserving +history is classified only as `linear_tree_preserving`, never guessed to be +squash or rebase from policy. A tree-changing merge requires separate +revalidation rather than inferred closure. + In Wave 2, the qualification approval digest binds a passing baseline's exact base commit, canonical task and repository configuration, selected command definitions, and normalized evidence identity. The context manifest, candidate @@ -80,15 +115,20 @@ deadline and output cap. The persisted absolute run deadline is reused for verification, review, retry, repair, and resume; no checkpoint grants a fresh budget. An attempt ID plus PID, PGID, and process-start digest is diagnostic state, not signalling authority. Cancellation is durable state polled by the -foreground lease owner, which terminates its own in-memory child; no command -signals a stored PID. If the lease is free but a recorded process may still -exist, resume and terminal cancellation fail closed for attended reconciliation. -State events are append-only, backup restore validates SQLite integrity, schema, -and required objects before atomic replacement, and purge requires every run to -be terminal. A failed pre-build context setup removes its provisional worktree -and branch. Review attempt budgets are scoped to an exact candidate generation, -and repair reasserts the reviewed commit/tree before allowing writes. There is -no background daemon or implicit retry. +foreground lease owner, which terminates its own in-memory child, including a +GitHub mutation process; no command signals a stored PID. Cancellation is +rechecked before each external effect, and an interrupted effect remains unknown +until authoritative readback. If the lease is free but a recorded process may +still exist, resume and terminal cancellation fail closed for attended +reconciliation. State events are append-only, backup restore validates SQLite +integrity, schema, and required objects before atomic replacement. Restoring +older state moves newer unreferenced Mill worktrees into a mode-restricted +quarantine. Its immutable recovery manifest records the database-swap commit +point and exact moved paths; restore never silently deletes them. Purge requires +every run to be reviewed or terminal. A failed pre-build context setup removes +its provisional worktree and branch. Review attempt budgets are scoped to an +exact candidate generation, and repair reasserts the reviewed commit/tree before +allowing writes. There is no background daemon or implicit retry. ## Core modules @@ -118,11 +158,14 @@ digest-pinned OCI environment with no network, a read-only root and workspace, dropped capabilities, no-new-privileges, resource bounds, and no implicit image pull. Every command receives an opaque Mill-owned container name and evidence is withheld until an unconditional, separately bounded `docker rm --force` -succeeds. Codex runs in attended trusted-host mode using workspace-write -sandboxing and promotion-time Git scope and identity checks. Mill does not claim -this prevents all host access. Stronger containment requires a separately -qualified container/VM worker with controlled model authentication and no -host-home, Docker-socket, keychain, or forge credential access. +succeeds. If a canonical workspace path contains a comma, Mill mounts it through +a mode-restricted, exact-realpath temporary alias so Docker's comma-delimited +long syntax does not truncate the source. Codex runs in attended trusted-host +mode using workspace-write sandboxing and promotion-time Git scope and identity +checks. Mill does not claim this prevents all host access. Stronger containment +requires a separately qualified container/VM worker with controlled model +authentication and no host-home, Docker-socket, keychain, or forge credential +access. ## Release trust diff --git a/docs/development.md b/docs/development.md index a89937b..1f37e65 100644 --- a/docs/development.md +++ b/docs/development.md @@ -42,9 +42,10 @@ Only applicable tiers are active. A skipped required lane blocks promotion. | Scenario | active | normal, exception, degradation, recovery, adversarial | | Cross-system | Wave 3+ | Codex, OCI and GitHub canaries | -Wave 2 keeps a deterministic fake-adapter suite in CI and requires an attended -real Codex/OCI canary before the wave is accepted. The realistic scenario set -covers: +Wave 3 keeps deterministic fake Codex, OCI, GitHub, and Git adapters in CI and +runs the packed CLI through the human-merge gate in a disposable repository. A +real Codex/OCI or GitHub canary remains attended maintainer evidence, never a CI +job with personal credentials. The realistic scenario set covers: - normal approval, build, lifecycle commit, verification, and clean review; - negative controls for failed, stale, inspect-only, or interrupted baseline @@ -58,15 +59,31 @@ covers: - recovery through crash-released writer leases, PID-reuse-safe orphan reconciliation, explicit OCI container cleanup, provisional workspace cleanup, exact-candidate repair revalidation, per-candidate review budgets, validated - state backup/restore, and terminal-only purge; + state backup/restore, quarantine of worktrees newer than a restored backup, + external-effect readback, one readback-authorized retry, retry exhaustion, + coordinator-level attendance enforcement, changing blocker identity, and purge + only after a locally reviewed or terminal state; - provenance through exact base, context, candidate commit/tree, validation, and review identity checks; +- remote delivery through wrong-actor/fork/remote denial, stale approvals, + expected-head pushes, effect-before-receipt recovery, unknown-effect blocking, + cancellation before and during mutations, paginated exact-head inline and + top-level review feedback, one aggregated repair, stable PR identity and + open-draft preflight before retry whether an ambiguous push is absent or + landed, unauthorized merger and disallowed merge-shape rejection, merge-tree + binding, and non-false-green post-merge checks; +- hostile filesystem coverage for Docker bind paths containing commas without + weakening read-only/no-network verification; +- restore recovery through an immutable pre-commit quarantine manifest and a + database swap as the final fallible commit point; - packaging through installation of the generated tarball and execution of its public CLI and schema exports. -The real provider canary is maintainer evidence, not a deterministic CI job: it -uses the maintainer's personal Codex account and a pre-pulled digest-pinned -image, and it must never push or create a pull request in Wave 2. +The real-provider canaries use the maintainer's personal Codex and GitHub +accounts, a pre-pulled digest-pinned image, and an explicitly named disposable +repository. They may exercise only the wave's approved effects and must preserve +authoritative readback evidence. No test may provision a repository, mark a PR +ready, merge, deploy, or rerun remote checks. ## Architecture questions diff --git a/docs/repository-settings.md b/docs/repository-settings.md index 2d4a8a2..6033090 100644 --- a/docs/repository-settings.md +++ b/docs/repository-settings.md @@ -10,7 +10,9 @@ After the Wave 1 checks have run at least once, configure: - conversation resolution required; - no force pushes or default-branch deletion; - merge queue disabled initially; -- squash merge as the only merge method; +- squash merge as the only GitHub UI method; Mill records the independently + provable result as `linear_tree_preserving` because one-commit squash and + rebase are not distinguishable from post-merge topology alone; - automatic branch deletion after merge; - zero required approving human reviews, because David is the sole maintainer; - maintainer bypass allowed only for emergencies and recorded as repair/audit diff --git a/product/PLAN.md b/product/PLAN.md index b29ae0a..615825f 100644 --- a/product/PLAN.md +++ b/product/PLAN.md @@ -20,6 +20,7 @@ Each item is one vertical delivery wave, not a bucket of microtasks. Later-wave choices close only before their wave. The current detailed task is in `product/tasks/`. -Wave 1 is landed. Wave 2 is implemented and remains active until its packed CLI -and attended real-provider canaries, exact-candidate review, PR CI, human merge, -and resulting-main checks are complete. +Waves 1 and 2 are landed. Wave 3 is active: its implementation, fake-provider +fault matrix, and packed draft-PR canary live in the same candidate change. +Exact-candidate review, PR CI, human merge, resulting-main checks, and the +explicitly authorized disposable real-GitHub canary remain its promotion gates. diff --git a/product/PRD.md b/product/PRD.md index cd35dcc..74f1e51 100644 --- a/product/PRD.md +++ b/product/PRD.md @@ -1,7 +1,6 @@ # Mill product requirements -Status: approved foundation contract Owner: David Ahmann Last updated: -2026-08-31 +Status: approved v1 contract Owner: David Ahmann Last updated: 2026-09-01 ## Problem diff --git a/product/tasks/WAVE_2.yaml b/product/tasks/WAVE_2.yaml index 1379431..c6592c5 100644 --- a/product/tasks/WAVE_2.yaml +++ b/product/tasks/WAVE_2.yaml @@ -1,5 +1,5 @@ task_id: mill-wave-2-local-delivery -status: active +status: complete objective: >- Turn one explicitly approved task packet into an exact lifecycle-owned local commit in a disposable worktree, validate it through declared OCI commands, diff --git a/product/tasks/WAVE_3.yaml b/product/tasks/WAVE_3.yaml new file mode 100644 index 0000000..55c0091 --- /dev/null +++ b/product/tasks/WAVE_3.yaml @@ -0,0 +1,219 @@ +task_id: mill-wave-3-draft-pr +status: active +objective: >- + Publish one already committed, locally verified, locally reviewed candidate as + a draft GitHub pull request through the operator's own authenticated gh + session, reconcile ambiguous effects without duplication, observe exact-head + policy, and close truthfully only after David merges and main checks settle. +risk_class: high +support_decision: + host: macOS-arm64 + git_observed: 2.50.0 + gh_observed: 2.74.2 + forge: github.com + actor_mode: interactive-delegated-operator + credential_mode: operator-owned-gh-session + builder_forge_credentials: forbidden + merge_authority: exact configured human login; David only for Mill +design_decisions: + - local exact-candidate review remains mandatory; a GitHub reviewer is + optional unless the repository configuration explicitly requires one + - propose authority is an exact digest over the run, candidate, target + repository identity, base branch, delivery branch, required checks, review + policy, and intended effects + - read, plan, push, create-PR, observe, reconcile, and finalize are distinct + operations; a plan or successful preflight never grants an external effect + - the shipper persists intent before each external call, records unknown + outcomes, and requires authoritative GitHub readback before any retry or + completion claim + - a stable delivery key identifies one intended PR across retries and one + stable branch; every candidate generation receives a distinct push effect + identity and exact expected-old-head precondition + - the shipper can publish only the candidate commit and tree already bound to + passing validation and local review; it cannot create or amend a commit + - forks, lookalike remotes, changed owners, changed repository node IDs, + changed operator identities, or changed policy invalidate propose authority + - Mill reports policy evidence and waits; it never marks a draft ready, + merges, deploys, reruns GitHub checks, or infers human authority from the + draft-ready transition + - paths containing commas use a protected temporary bind alias for OCI + verification rather than Docker's comma-delimited source syntax + - state restore quarantines newer unreferenced Mill worktrees with a durable + recovery manifest and never silently deletes them +action_boundary: + disclosures: + - exact repository owner/name/node ID, actor login/ID, branch, candidate + SHA, task title, delivery marker, required-check names, and review state + effects: + - read GitHub actor, repository, refs, pull requests, checks, reviews, and + merge state + - push one exact candidate SHA to one Mill-owned branch with an expected-old + remote-head precondition + - create one draft pull request identified by an immutable delivery marker + authority_owner: repository mill.yaml plus explicit operator approval digest + credential_owner: operator; credentials remain behind gh/git credential helper + source_of_truth: GitHub API readback + idempotency: stable delivery key plus per-candidate effect IDs + unknown_effect: + block mutation and reconcile by exact branch, marker, target, candidate, and + PR identity + controls: + - TOL-001 + - TOL-003 + - TOL-004 + - TOL-005 + - IAM-001 + - IAM-002 + - IAM-003 + - SEC-001 + - SEC-002 + - SEC-004 + - SEC-006 + - REL-001 + - REL-003 + - REL-005 +allowed_paths: + - architecture/** + - docs/** + - product/** + - schemas/** + - scripts/** + - src/** + - test/** + - AGENTS.md + - CHANGELOG.md + - README.md + - WORKFLOW.md + - package.json + - package-lock.json + - vitest.config.ts +forbidden_paths: + - TEMP_MILL_GREENFIELD_WORK_PLAN_2026-08-31.md + - .env* + - .mill/** + - .github/workflows/release.yml +acceptance_items: + - W3-A1 propose configuration binds the exact GitHub host, repository node + identity, owner/name, remote, base branch, required checks, allowed merge + methods, and optional remote-review policy + - W3-A2 plan is read-only and returns an approval digest bound to the exact + reviewed candidate, actor, target, policy, delivery branch, and effects + - W3-A3 missing, stale, mismatched, inspect/build-only, fork, lookalike, + changed-account, or changed-repository authority produces zero mutations + - W3-A4 builder and reviewer processes never receive GitHub credentials or + remote-mutation capability; only the attended shipper invokes gh or push + - W3-A5 push intent is durable before the call and only the exact reviewed + commit is published under an expected-old-head precondition + - W3-A6 draft-PR intent is durable before the call; the immutable delivery + marker and authoritative readback prevent duplicate PR creation + - W3-A7 crash before a call, after effect before receipt, and after receipt + before response reconcile truthfully without duplicate mutation; exact + readback proving absence authorizes at most one retry, while durable + cancellation is polled before and during each remote mutation and never + permits a subsequent effect + - W3-A8 latest-head required checks and configured remote-review policy are + observed with bounded polling; pending, skipped, stale, neutral, cancelled, + timed-out, or failed required evidence never passes + - W3-A9 one aggregated attended remote-feedback repair may create a new local + candidate, invalidates old evidence and approval, and updates the same PR + only after revalidation, local review, and a new exact approval + - W3-A10 Mill stops at awaiting_human and never changes draft readiness; only + an exact configured human login may close delivery by merging with an + independently provable allowed merge shape + - W3-A11 finalize reads back PR head, merger identity, merge shape/result + SHA/tree, and exact default-branch checks before post_merge_verified and + closed; pending or failed post-merge evidence is never reported as success + - W3-A12 ambiguous or conflicting branch, PR, merge, or check identity blocks + and emits typed reconciliation evidence rather than retrying blindly + - W3-A13 OCI validation supports repository/state paths containing commas + without weakening the read-only/no-network verifier boundary + - W3-A14 restoring older state quarantines newer unreferenced worktrees with + recoverable evidence and never silently deletes them + - W3-A15 fake-provider, delivered-package, fault-injection, and one attended + disposable GitHub canary prove the lifecycle; no normal closure-only PR is + produced +commands_in_scope: + - doctor --mode propose + - pr plan + - pr open + - pr reconcile + - pr observe + - pr finalize + - resume + - status + - state restore +validation_commands: + - npm run format:check + - npm run lint + - npm run typecheck + - npm test + - npm run test:coverage + - npm run test:package +final_validation_commands: + - npm run check + - packed CLI disposable-repository draft-PR lifecycle canary with fake gh + - attended disposable GitHub canary with no merge automation +required_worker_chain: + - task-executor + - validation-gate + - code-review + - commit-push +lifecycle_gates: + architecture_review_required: true + security_review_required: true + code_review_required: true + required_pr_ci: true + human_merge_required: true +evidence_required: + - red-first tests for denied authority, duplicate effects, unknown outcomes, + path-safe OCI mounting, and restore reconciliation + - visible command results and item-level acceptance results + - exact committed candidate validation and local review + - fake-provider external-effect fault matrix + - clean-checkout packed-CLI canary + - attended disposable GitHub canary and authoritative readback +scope_exclusions: + - automatic merge, draft-ready transition, deployment, check rerun, issue or + tracker projection, repository provisioning, or branch-protection mutation + - daemon, parallel workers, hosted state, shared credentials, GitHub App, or + unattended workload identity + - generalized PRD compiler, stack research, bootstrap recipe, or retrofit + - automated upgrade, detach, rollback, release, or npm publication + - hostile-host containment claim for Codex or gh +stop_conditions: + - repository, actor, credential, remote destination, base branch, required + checks, review policy, or merge method cannot be bound exactly + - a model-visible process can access forge credentials or invoke a remote + effect + - state cannot persist intent before a call or authoritative readback cannot + classify an unknown outcome + - the shipper can publish a commit/tree other than the reviewed candidate + - a retry can create a duplicate branch or pull request + - the same subsystem produces recurring P0/P1 review findings +retry_budget: 1 +runtime_pins: + node: 24.20.0 + npm: 11.x + typescript: 6.0.3 + git_observed: 2.50.0 + gh_observed: 2.74.2 + github_api: v3 +alignment_gate_ref: product/PRD.md +plan_drift_policy_ref: WORKFLOW.md +test_matrix_refs: + - docs/development.md#testing-matrix +engineering_policy_refs: + - AGENTS.md#engineering-rules +architecture_guidance_refs: + - architecture/ARCHITECTURE.md +changelog_intent: add exact-candidate draft-PR delivery and truthful closure +versioning_impact: pre-alpha additive CLI, config, state, and result contracts +migration_impact: + additive operational-state tables and run transitions with validated restore + compatibility +docs_sync_refs: + - README.md + - product/PLAN.md + - product/PRD.md + - architecture/ARCHITECTURE.md + - docs/development.md diff --git a/schemas/README.md b/schemas/README.md index 719bfbd..0a1041e 100644 --- a/schemas/README.md +++ b/schemas/README.md @@ -15,9 +15,11 @@ documents use the same data model. - `context-manifest.schema.json` - `validation-evidence.schema.json` - `review-result.schema.json` +- `delivery-record.schema.json` Task packets are Git-owned approval contracts. Context manifests, validation -evidence, and review results are schema-versioned operational artifacts bound to -an exact task/base/candidate. SQLite runs and events, credentials, prompts, raw -model streams, and raw command output are deliberately not repository contracts -and are never accepted as authority. +evidence, review results, and delivery records are schema-versioned operational +artifacts bound to an exact task/base/candidate and external-effect identity. +SQLite runs and events, credentials, prompts, raw model streams, and raw command +output are deliberately not repository contracts and are never accepted as +authority. diff --git a/schemas/delivery-record.schema.json b/schemas/delivery-record.schema.json new file mode 100644 index 0000000..376e7f6 --- /dev/null +++ b/schemas/delivery-record.schema.json @@ -0,0 +1,220 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/davidahmann/mill/schemas/delivery-record.schema.json", + "title": "DeliveryRecord", + "type": "object", + "additionalProperties": false, + "required": [ + "schemaVersion", + "runId", + "deliveryKey", + "proposalDigest", + "approvalExpiresAt", + "state", + "target", + "branchName", + "candidateCommit", + "candidateTree", + "requiredChecks", + "reviewPolicy", + "allowedMergerLogins", + "allowedMergeMethods", + "effects", + "remoteHeadCommit", + "pullRequest", + "observation", + "merge", + "lastErrorCode", + "createdAt", + "updatedAt" + ], + "properties": { + "schemaVersion": { "const": "1" }, + "runId": { "type": "string", "format": "uuid" }, + "deliveryKey": { "$ref": "#/$defs/digest" }, + "proposalDigest": { "$ref": "#/$defs/digest" }, + "approvalExpiresAt": { "type": "string", "format": "date-time" }, + "state": { + "enum": [ + "planned", + "proposing", + "effect_unknown", + "awaiting_ci", + "awaiting_human", + "merged", + "post_merge_verified", + "closed", + "cancelled", + "blocked" + ] + }, + "target": { + "type": "object", + "additionalProperties": false, + "required": [ + "forge", + "host", + "owner", + "repository", + "repositoryNodeId", + "cloneUrl", + "remoteName", + "baseBranch", + "actorLogin", + "actorId" + ], + "properties": { + "forge": { "const": "github" }, + "host": { "const": "github.com" }, + "owner": { "type": "string", "minLength": 1 }, + "repository": { "type": "string", "minLength": 1 }, + "repositoryNodeId": { "type": "string", "minLength": 1 }, + "cloneUrl": { "type": "string", "format": "uri" }, + "remoteName": { "type": "string", "minLength": 1 }, + "baseBranch": { "type": "string", "minLength": 1 }, + "actorLogin": { "type": "string", "minLength": 1 }, + "actorId": { "type": "integer", "minimum": 1 } + } + }, + "branchName": { "type": "string", "minLength": 1 }, + "candidateCommit": { "$ref": "#/$defs/sha" }, + "candidateTree": { "$ref": "#/$defs/sha" }, + "requiredChecks": { + "type": "array", + "items": { "type": "string", "minLength": 1 } + }, + "reviewPolicy": { + "type": "object", + "additionalProperties": false, + "required": ["mode", "requiredReviewerLogins"], + "properties": { + "mode": { "enum": ["local_only", "github_required"] }, + "requiredReviewerLogins": { + "type": "array", + "items": { "type": "string", "minLength": 1 } + } + }, + "allOf": [ + { + "if": { + "properties": { "mode": { "const": "github_required" } }, + "required": ["mode"] + }, + "then": { + "properties": { + "requiredReviewerLogins": { + "type": "array", + "minItems": 1 + } + } + } + } + ] + }, + "allowedMergerLogins": { + "type": "array", + "minItems": 1, + "items": { "type": "string", "minLength": 1 } + }, + "allowedMergeMethods": { + "type": "array", + "minItems": 1, + "items": { "enum": ["merge", "linear_tree_preserving"] } + }, + "effects": { + "type": "array", + "items": { "$ref": "#/$defs/effect" } + }, + "remoteHeadCommit": { + "oneOf": [{ "$ref": "#/$defs/sha" }, { "type": "null" }] + }, + "pullRequest": { + "oneOf": [ + { + "type": "object", + "additionalProperties": false, + "required": ["number", "nodeId", "url"], + "properties": { + "number": { "type": "integer", "minimum": 1 }, + "nodeId": { "type": "string", "minLength": 1 }, + "url": { "type": "string", "format": "uri" } + } + }, + { "type": "null" } + ] + }, + "observation": { "type": ["object", "null"] }, + "merge": { + "oneOf": [ + { + "type": "object", + "additionalProperties": false, + "required": [ + "commit", + "tree", + "method", + "mergedByLogin", + "mergedAt", + "defaultBranchHead" + ], + "properties": { + "commit": { "$ref": "#/$defs/sha" }, + "tree": { "$ref": "#/$defs/sha" }, + "method": { + "enum": ["merge", "linear_tree_preserving"] + }, + "mergedByLogin": { "type": "string", "minLength": 1 }, + "mergedAt": { "type": "string", "format": "date-time" }, + "defaultBranchHead": { "$ref": "#/$defs/sha" } + } + }, + { "type": "null" } + ] + }, + "lastErrorCode": { "type": ["string", "null"], "minLength": 1 }, + "createdAt": { "type": "string", "format": "date-time" }, + "updatedAt": { "type": "string", "format": "date-time" } + }, + "$defs": { + "digest": { + "type": "string", + "pattern": "^sha256:[a-f0-9]{64}$" + }, + "sha": { "type": "string", "pattern": "^[a-f0-9]{40}$" }, + "effect": { + "type": "object", + "additionalProperties": false, + "required": [ + "id", + "kind", + "candidateCommit", + "status", + "attemptCount", + "expectedOldCommit", + "errorCode", + "updatedAt" + ], + "properties": { + "id": { "$ref": "#/$defs/digest" }, + "kind": { "enum": ["push", "pull_request"] }, + "candidateCommit": { "$ref": "#/$defs/sha" }, + "status": { + "enum": [ + "intent", + "call_started", + "effect_unknown", + "retryable_absent", + "verified", + "blocked" + ] + }, + "attemptCount": { "type": "integer", "minimum": 0, "maximum": 2 }, + "expectedOldCommit": { + "oneOf": [{ "$ref": "#/$defs/sha" }, { "type": "null" }] + }, + "errorCode": { "type": ["string", "null"], "minLength": 1 }, + "updatedAt": { "type": "string", "format": "date-time" } + } + } + } +} diff --git a/schemas/mill-config.schema.json b/schemas/mill-config.schema.json index 3a715f6..bd15f54 100644 --- a/schemas/mill-config.schema.json +++ b/schemas/mill-config.schema.json @@ -29,6 +29,103 @@ "network": { "const": "none" } } }, + "propose": { + "type": "object", + "additionalProperties": false, + "required": [ + "forge", + "host", + "owner", + "repository", + "repositoryNodeId", + "remoteName", + "baseBranch", + "branchPrefix", + "allowedActors", + "allowedMergerLogins", + "requiredChecks", + "reviewPolicy", + "allowedMergeMethods" + ], + "properties": { + "forge": { "const": "github" }, + "host": { "const": "github.com" }, + "owner": { "type": "string", "pattern": "^[A-Za-z0-9_.-]+$" }, + "repository": { + "type": "string", + "pattern": "^[A-Za-z0-9_.-]+$" + }, + "repositoryNodeId": { "type": "string", "minLength": 1 }, + "remoteName": { + "type": "string", + "pattern": "^[A-Za-z0-9._-]+$" + }, + "baseBranch": { + "type": "string", + "pattern": "^(?!-)(?!.*\\.\\.)[^\\s~^:?*[\\\\]+$" + }, + "branchPrefix": { "const": "mill/" }, + "allowedActors": { + "type": "array", + "minItems": 1, + "items": { "type": "string", "minLength": 1 } + }, + "allowedMergerLogins": { + "type": "array", + "minItems": 1, + "items": { "type": "string", "minLength": 1 } + }, + "requiredChecks": { + "type": "array", + "items": { "type": "string", "minLength": 1 } + }, + "reviewPolicy": { + "type": "object", + "additionalProperties": false, + "required": ["mode", "requiredReviewerLogins"], + "properties": { + "mode": { "enum": ["local_only", "github_required"] }, + "requiredReviewerLogins": { + "type": "array", + "items": { "type": "string", "minLength": 1 } + } + }, + "allOf": [ + { + "if": { + "properties": { "mode": { "const": "github_required" } }, + "required": ["mode"] + }, + "then": { + "properties": { + "requiredReviewerLogins": { + "type": "array", + "minItems": 1 + } + } + } + } + ] + }, + "allowedMergeMethods": { + "type": "array", + "minItems": 1, + "items": { "enum": ["merge", "linear_tree_preserving"] } + }, + "approvalTtlSeconds": { + "type": "integer", + "minimum": 60, + "maximum": 3600, + "default": 900 + }, + "pollTimeoutSeconds": { + "type": "integer", + "minimum": 1, + "maximum": 1800, + "default": 600 + } + } + }, "commands": { "type": "object", "propertyNames": { "minLength": 1 }, @@ -66,5 +163,17 @@ } } } - } + }, + "allOf": [ + { + "if": { + "properties": { "trustCeiling": { "const": "propose" } }, + "required": ["trustCeiling"] + }, + "then": { + "properties": { "propose": true }, + "required": ["propose"] + } + } + ] } diff --git a/scripts/test-package.mjs b/scripts/test-package.mjs index 454e121..c050ba0 100644 --- a/scripts/test-package.mjs +++ b/scripts/test-package.mjs @@ -70,6 +70,8 @@ try { "README.md", "LICENSE", "schemas/context-manifest.schema.json", + "schemas/delivery-record.schema.json", + "schemas/mill-config.schema.json", "schemas/review-result.schema.json", "schemas/task-packet.schema.json", "schemas/validation-evidence.schema.json", @@ -143,6 +145,7 @@ try { "status", "verify", "review", + "pr", "resume", "cancel", "state", @@ -186,11 +189,29 @@ try { path.join(consumer, "mill.yaml"), `schemaVersion: "1" repositoryId: "22222222-2222-4222-8222-222222222222" -trustCeiling: build +trustCeiling: propose sensitivePaths: [.env] verifier: image: "node@sha256:ba849c60be29959425b8734d57b8b4b7d56f98edd9504c9af091d5281095a71e" network: none +propose: + forge: github + host: github.com + owner: example + repository: app + repositoryNodeId: R_package_canary + remoteName: origin + baseBranch: main + branchPrefix: mill/ + allowedActors: [package-operator] + allowedMergerLogins: [package-operator] + requiredChecks: [validate] + reviewPolicy: + mode: local_only + requiredReviewerLogins: [] + allowedMergeMethods: [linear_tree_preserving] + approvalTtlSeconds: 900 + pollTimeoutSeconds: 30 commands: test: argv: ["node", "--test"] @@ -238,6 +259,11 @@ budget: ), ]); command("/usr/bin/git", ["init", "--initial-branch=main"], consumer); + command( + "/usr/bin/git", + ["remote", "add", "origin", "https://github.com/example/app.git"], + consumer, + ); command("/usr/bin/git", ["add", "."], consumer); command( "/usr/bin/git", @@ -262,6 +288,8 @@ budget: ]); const codex = path.join(tools, "codex"); const docker = path.join(tools, "docker"); + const gh = path.join(tools, "gh"); + const git = path.join(tools, "git"); await writeFile( codex, `#!${process.execPath} @@ -305,7 +333,57 @@ process.exit(/value = [1-9]/u.test(value)?0:1); `, { mode: 0o700 }, ); - await Promise.all([chmod(codex, 0o700), chmod(docker, 0o700)]); + await writeFile( + git, + `#!${process.execPath} +import {spawnSync} from "node:child_process"; +import {writeFileSync} from "node:fs"; +const args=process.argv.slice(2); +if(args.includes("push")){ + const refspec=args.at(-1)??"";const candidate=refspec.split(":",1)[0]??""; + if(!/^[a-f0-9]{40}$/.test(candidate))process.exit(2); + writeFileSync(new URL("./remote-head",import.meta.url),candidate+"\\n",{mode:0o600}); + console.log("done");process.exit(0); +} +const result=spawnSync("/usr/bin/git",args,{cwd:process.cwd(),env:process.env,encoding:"utf8"}); +process.stdout.write(result.stdout??"");process.stderr.write(result.stderr??"");process.exit(result.status??1); +`, + { mode: 0o700 }, + ); + await writeFile( + gh, + `#!${process.execPath} +import {readFileSync,writeFileSync} from "node:fs"; +const args=process.argv.slice(2);const endpoint=args.find((value)=>value.startsWith("repos/"))??args.at(-1)??""; +const read=(name)=>{try{return readFileSync(new URL(name,import.meta.url),"utf8").trim()}catch{return null}}; +const remoteHead=()=>read("./remote-head"); +const pullPath=new URL("./pull-request.json",import.meta.url); +const pull=()=>{const value=read("./pull-request.json");return value===null?null:JSON.parse(value)}; +const field=(name)=>{for(let index=0;index + JSON.parse( + command( + bin, + ["--json", "--cwd", consumer, ...args], + consumer, + proposalEnvironment, + ), + ); + const proposal = proposalMill([ + "pr", + "plan", + "--task", + "product/tasks/canary.yaml", + "--run", + runId, + ]); + const opened = proposalMill([ + "pr", + "open", + "--task", + "product/tasks/canary.yaml", + "--run", + runId, + "--approve", + proposal.data.delivery.proposalDigest, + "--attended", + ]); + if (opened.data.run.status !== "awaiting_ci") { + throw new Error("packed lifecycle did not open one draft pull request"); + } + const observed = proposalMill([ + "pr", + "observe", + "--task", + "product/tasks/canary.yaml", + "--run", + runId, + ]); + if (observed.data.run.status !== "awaiting_human") { + throw new Error("packed lifecycle did not reach the human merge gate"); + } process.stdout.write( - `package lifecycle canary passed: ${packResult.filename}\n`, + `package draft-PR lifecycle canary passed: ${packResult.filename}\n`, ); } finally { await rm(temporary, { force: true, recursive: true }); diff --git a/src/cli-program.ts b/src/cli-program.ts index 3d61582..3a9ca89 100644 --- a/src/cli-program.ts +++ b/src/cli-program.ts @@ -9,6 +9,13 @@ import { doctor, doctorReady, type DoctorMode } from "./doctor.js"; import { asMillError, ExitCode, MillError } from "./errors.js"; import { inspectPrd } from "./intake/prd.js"; import { scanRepository } from "./repository/scan.js"; +import { + finalizeDraftPr, + observeDraftPr, + openDraftPr, + planDraftPr, + reconcileDraftPr, +} from "./runtime/delivery.js"; import { cancelRun, codexAuthStatus, @@ -464,6 +471,174 @@ export function createProgram(io: CliIo, jsonErrors = false): Command { ); }); + const pr = program + .command("pr") + .description("deliver an exact reviewed candidate through a draft PR"); + pr.command("plan") + .description("read live GitHub identity and create a local approval plan") + .requiredOption("--task ", "approved task packet path") + .requiredOption("--run ", "run identifier") + .action(async (options: { task: string; run: string }) => { + const global = globals(program); + const root = await findRepositoryRoot(global.cwd); + await enforceExactVersion(root); + const result = await planDraftPr({ + root, + taskPath: options.task, + runId: options.run, + }); + emit( + io, + global.json === true, + commandResult({ command: "pr.plan", ok: true, data: result }), + ); + }); + pr.command("open") + .description("push and open the explicitly approved draft PR") + .requiredOption("--task ", "approved task packet path") + .requiredOption("--run ", "run identifier") + .requiredOption( + "--approve ", + "exact, unexpired proposal digest returned by pr plan", + ) + .requiredOption( + "--attended", + "acknowledge attended use of the operator-owned GitHub session", + ) + .action( + async (options: { + task: string; + run: string; + approve: string; + attended: boolean; + }) => { + const global = globals(program); + const root = await findRepositoryRoot(global.cwd); + await enforceExactVersion(root); + const result = await openDraftPr({ + root, + taskPath: options.task, + runId: options.run, + approvalDigest: options.approve, + attended: options.attended, + }); + emit( + io, + global.json === true, + commandResult({ command: "pr.open", ok: true, data: result }), + ); + }, + ); + pr.command("reconcile") + .description("classify one unknown GitHub effect through readback only") + .requiredOption("--task ", "approved task packet path") + .requiredOption("--run ", "run identifier") + .action(async (options: { task: string; run: string }) => { + const global = globals(program); + const root = await findRepositoryRoot(global.cwd); + await enforceExactVersion(root); + const result = await reconcileDraftPr({ + root, + taskPath: options.task, + runId: options.run, + }); + emit( + io, + global.json === true, + commandResult({ command: "pr.reconcile", ok: true, data: result }), + ); + }); + pr.command("observe") + .description("read exact-head checks and configured GitHub review policy") + .requiredOption("--task ", "approved task packet path") + .requiredOption("--run ", "run identifier") + .action(async (options: { task: string; run: string }) => { + const global = globals(program); + const root = await findRepositoryRoot(global.cwd); + await enforceExactVersion(root); + const result = await observeDraftPr({ + root, + taskPath: options.task, + runId: options.run, + }); + const blocked = result.run.status === "blocked"; + emit( + io, + global.json === true, + commandResult({ + command: "pr.observe", + ok: !blocked, + status: blocked ? "blocked" : "ok", + data: result, + reasons: blocked + ? [ + { + code: result.run.blockCode ?? "REMOTE_POLICY_BLOCKED", + message: + "GitHub checks or configured review policy blocked this exact candidate.", + }, + ] + : [], + }), + ); + if (blocked) { + throw new MillError( + result.run.blockCode ?? "REMOTE_POLICY_BLOCKED", + "GitHub policy blocked this exact candidate.", + ExitCode.configuration, + { resultAlreadyEmitted: true }, + ); + } + }); + pr.command("finalize") + .description("read back human merge and exact main checks before closure") + .requiredOption("--task ", "approved task packet path") + .requiredOption("--run ", "run identifier") + .action(async (options: { task: string; run: string }) => { + const global = globals(program); + const root = await findRepositoryRoot(global.cwd); + await enforceExactVersion(root); + const result = await finalizeDraftPr({ + root, + taskPath: options.task, + runId: options.run, + }); + const closed = result.run.status === "closed"; + const reasonCode = + result.run.status === "blocked" + ? (result.run.blockCode ?? "POST_MERGE_CHECKS_FAILED") + : "POST_MERGE_CHECKS_PENDING"; + emit( + io, + global.json === true, + commandResult({ + command: "pr.finalize", + ok: closed, + status: closed ? "ok" : "blocked", + data: result, + reasons: closed + ? [] + : [ + { + code: reasonCode, + message: + "Post-merge required checks have not produced passing exact-commit evidence.", + }, + ], + }), + ); + if (!closed) { + throw new MillError( + reasonCode, + "Post-merge verification is not complete.", + result.run.status === "blocked" + ? ExitCode.configuration + : ExitCode.temporary, + { resultAlreadyEmitted: true }, + ); + } + }); + program .command("cancel") .description("persist cancellation for the exact foreground controller") @@ -509,16 +684,16 @@ export function createProgram(io: CliIo, jsonErrors = false): Command { const global = globals(program); const root = await findRepositoryRoot(global.cwd); await enforceExactVersion(root); - await stateRestore({ root, backupPath: options.from }); + const report = await stateRestore({ root, backupPath: options.from }); emit( io, global.json === true, - commandResult({ command: "state.restore", ok: true, data: {} }), + commandResult({ command: "state.restore", ok: true, data: report }), ); }); state .command("purge") - .description("remove terminal local state and disposable worktrees") + .description("remove purge-safe local state and disposable worktrees") .requiredOption( "--confirm ", "exact repository UUID acknowledgement", diff --git a/src/contracts/schemas.ts b/src/contracts/schemas.ts index 7e1bef4..b2f410d 100644 --- a/src/contracts/schemas.ts +++ b/src/contracts/schemas.ts @@ -85,30 +85,81 @@ const repositoryPathPatternSchema = z .string() .regex(/^(?!\/)(?!.*(?:^|\/)\.\.(?:\/|$))[^*?[\]\\]+(?:\/\*\*)?$/u); -export const millConfigSchema = z.strictObject({ - schemaVersion: z.literal("1"), - repositoryId: z.uuid(), - trustCeiling: z.enum(["inspect", "build", "propose"]), - sensitivePaths: z.array(repositoryPathPatternSchema).default([]), - verifier: z - .strictObject({ - image: z.string().regex(/^[^@\s]+@sha256:[a-f0-9]{64}$/u), - network: z.literal("none"), - }) - .optional(), - commands: z.record( - z.string().min(1), - z.strictObject({ - argv: z.array(z.string().min(1)).min(1), - cwd: z.string().min(1), - controlPaths: z.array(repositoryPathPatternSchema).min(1), - capability: z.enum(["read", "build", "test", "package"]), - required: z.boolean().default(true), - timeoutSeconds: z.number().int().min(1).max(3600).default(600), - execution: z.enum(["oci", "host"]).default("oci"), - }), - ), -}); +const githubReviewPolicySchema = z + .strictObject({ + mode: z.enum(["local_only", "github_required"]), + requiredReviewerLogins: z.array(z.string().min(1)), + }) + .superRefine((policy, context) => { + if ( + policy.mode === "github_required" && + policy.requiredReviewerLogins.length === 0 + ) { + context.addIssue({ + code: "custom", + path: ["requiredReviewerLogins"], + message: + "github_required review policy needs at least one reviewer login", + }); + } + }); + +export const millConfigSchema = z + .strictObject({ + schemaVersion: z.literal("1"), + repositoryId: z.uuid(), + trustCeiling: z.enum(["inspect", "build", "propose"]), + sensitivePaths: z.array(repositoryPathPatternSchema).default([]), + verifier: z + .strictObject({ + image: z.string().regex(/^[^@\s]+@sha256:[a-f0-9]{64}$/u), + network: z.literal("none"), + }) + .optional(), + propose: z + .strictObject({ + forge: z.literal("github"), + host: z.literal("github.com"), + owner: z.string().regex(/^[A-Za-z0-9_.-]+$/u), + repository: z.string().regex(/^[A-Za-z0-9_.-]+$/u), + repositoryNodeId: z.string().min(1), + remoteName: z.string().regex(/^[A-Za-z0-9._-]+$/u), + baseBranch: z.string().regex(/^(?!-)(?!.*\.\.)[^\s~^:?*[\\]+$/u), + branchPrefix: z.literal("mill/"), + allowedActors: z.array(z.string().min(1)).min(1), + allowedMergerLogins: z.array(z.string().min(1)).min(1), + requiredChecks: z.array(z.string().min(1)), + reviewPolicy: githubReviewPolicySchema, + allowedMergeMethods: z + .array(z.enum(["merge", "linear_tree_preserving"])) + .min(1), + approvalTtlSeconds: z.number().int().min(60).max(3600).default(900), + pollTimeoutSeconds: z.number().int().min(1).max(1800).default(600), + }) + .optional(), + commands: z.record( + z.string().min(1), + z.strictObject({ + argv: z.array(z.string().min(1)).min(1), + cwd: z.string().min(1), + controlPaths: z.array(repositoryPathPatternSchema).min(1), + capability: z.enum(["read", "build", "test", "package"]), + required: z.boolean().default(true), + timeoutSeconds: z.number().int().min(1).max(3600).default(600), + execution: z.enum(["oci", "host"]).default("oci"), + }), + ), + }) + .superRefine((value, context) => { + if (value.trustCeiling === "propose" && value.propose === undefined) { + context.addIssue({ + code: "custom", + path: ["propose"], + message: + "propose configuration is required at the propose trust ceiling", + }); + } + }); const authorityReferenceSchema = z.strictObject({ path: z.string().min(1), @@ -219,6 +270,94 @@ export const validationEvidenceSchema = z.strictObject({ passed: z.boolean(), }); +const remoteEffectSchema = z.strictObject({ + id: digestSchema, + kind: z.enum(["push", "pull_request"]), + candidateCommit: z.string().regex(/^[a-f0-9]{40}$/u), + status: z.enum([ + "intent", + "call_started", + "effect_unknown", + "retryable_absent", + "verified", + "blocked", + ]), + attemptCount: z.number().int().min(0).max(2), + expectedOldCommit: z + .string() + .regex(/^[a-f0-9]{40}$/u) + .nullable(), + errorCode: z.string().min(1).nullable(), + updatedAt: z.iso.datetime(), +}); + +export const deliveryRecordSchema = z.strictObject({ + schemaVersion: z.literal("1"), + runId: z.uuid(), + deliveryKey: digestSchema, + proposalDigest: digestSchema, + approvalExpiresAt: z.iso.datetime(), + state: z.enum([ + "planned", + "proposing", + "effect_unknown", + "awaiting_ci", + "awaiting_human", + "merged", + "post_merge_verified", + "closed", + "cancelled", + "blocked", + ]), + target: z.strictObject({ + forge: z.literal("github"), + host: z.literal("github.com"), + owner: z.string().min(1), + repository: z.string().min(1), + repositoryNodeId: z.string().min(1), + cloneUrl: z.url(), + remoteName: z.string().min(1), + baseBranch: z.string().min(1), + actorLogin: z.string().min(1), + actorId: z.number().int().positive(), + }), + branchName: z.string().min(1), + candidateCommit: z.string().regex(/^[a-f0-9]{40}$/u), + candidateTree: z.string().regex(/^[a-f0-9]{40}$/u), + requiredChecks: z.array(z.string().min(1)), + reviewPolicy: githubReviewPolicySchema, + allowedMergerLogins: z.array(z.string().min(1)).min(1), + allowedMergeMethods: z + .array(z.enum(["merge", "linear_tree_preserving"])) + .min(1), + effects: z.array(remoteEffectSchema), + remoteHeadCommit: z + .string() + .regex(/^[a-f0-9]{40}$/u) + .nullable(), + pullRequest: z + .strictObject({ + number: z.number().int().positive(), + nodeId: z.string().min(1), + url: z.url(), + }) + .nullable(), + observation: z.record(z.string(), z.unknown()).nullable(), + merge: z + .strictObject({ + commit: z.string().regex(/^[a-f0-9]{40}$/u), + tree: z.string().regex(/^[a-f0-9]{40}$/u), + method: z.enum(["merge", "linear_tree_preserving"]), + mergedByLogin: z.string().min(1), + mergedAt: z.iso.datetime(), + defaultBranchHead: z.string().regex(/^[a-f0-9]{40}$/u), + }) + .nullable(), + lastErrorCode: z.string().min(1).nullable(), + createdAt: z.iso.datetime(), + updatedAt: z.iso.datetime(), +}); + export const millLockSchema = z.strictObject({ schemaVersion: z.literal("1"), mill: z.strictObject({ @@ -248,6 +387,7 @@ export const contractSchemas = { scenarioSet: scenarioSetSchema, taskPacket: taskPacketSchema, validationEvidence: validationEvidenceSchema, + deliveryRecord: deliveryRecordSchema, } as const; export type ContractKind = keyof typeof contractSchemas; diff --git a/src/index.ts b/src/index.ts index 1eb6c66..ee388ea 100644 --- a/src/index.ts +++ b/src/index.ts @@ -14,6 +14,14 @@ export { export { MillError, ExitCode } from "./errors.js"; export { inspectPrd, type PrdInspection } from "./intake/prd.js"; export { scanRepository, type RepositoryScan } from "./repository/scan.js"; +export { + finalizeDraftPr, + observeDraftPr, + openDraftPr, + planDraftPr, + reconcileDraftPr, + type DeliveryRecord, +} from "./runtime/delivery.js"; export { cancelRun, codexAuthStatus, diff --git a/src/runtime/delivery.ts b/src/runtime/delivery.ts new file mode 100644 index 0000000..6ca6ee5 --- /dev/null +++ b/src/runtime/delivery.ts @@ -0,0 +1,1712 @@ +import type { z } from "zod"; + +import { canonicalDigest } from "../contracts/canonical.js"; +import { + deliveryRecordSchema, + reviewResultSchema, + validationEvidenceSchema, +} from "../contracts/schemas.js"; +import { ExitCode, MillError, asMillError } from "../errors.js"; +import { + createGitHubAdapter, + type GitHubAdapter, + type GitHubBinding, + type GitHubCheck, + type GitHubFeedback, + type GitHubObservation, + type GitHubPullRequest, + type ProposeConfig, +} from "./github.js"; +import { loadRuntimeInputs, type RuntimeInputs } from "./inputs.js"; +import { assertRunBindings } from "./lifecycle.js"; +import { commonGitDirectory, repositoryRemoteUrl } from "./repository.js"; +import { + acquireWriterLease, + publicRunRecord, + StateStore, + type PublicRunRecord, + type RunRecord, +} from "./state.js"; + +export type DeliveryRecord = z.infer; +type RemoteEffect = DeliveryRecord["effects"][number]; +const maximumRemoteEffectAttempts = 2; + +interface DeliveryContext { + inputs: RuntimeInputs; + config: ProposeConfig; + store: StateStore; +} + +function operationDeadline(config: ProposeConfig): number { + return Date.now() + config.pollTimeoutSeconds * 1000; +} + +async function openDeliveryContext( + root: string, + taskPath: string, +): Promise { + const inputs = await loadRuntimeInputs(root, taskPath); + if ( + inputs.config.trustCeiling !== "propose" || + inputs.config.propose === undefined + ) { + throw new MillError( + "PROPOSE_NOT_AUTHORIZED", + "mill.yaml does not grant the propose trust ceiling and exact GitHub policy.", + ExitCode.configuration, + ); + } + const commonDirectory = await commonGitDirectory(root); + return { + inputs, + config: inputs.config.propose, + store: await StateStore.open(inputs.config.repositoryId, commonDirectory), + }; +} + +function storedDelivery(run: RunRecord): DeliveryRecord { + if (run.deliveryJson === undefined) { + throw new MillError( + "DELIVERY_PLAN_MISSING", + "Run has no approved draft-PR delivery plan.", + ExitCode.configuration, + ); + } + try { + return deliveryRecordSchema.parse(JSON.parse(run.deliveryJson)); + } catch (error) { + throw new MillError( + "DELIVERY_RECORD_INVALID", + "Stored draft-PR delivery state is invalid.", + ExitCode.data, + { cause: String(error) }, + ); + } +} + +function persistDelivery( + store: StateStore, + runId: string, + value: DeliveryRecord, + eventType: string, + details: Record = {}, +): DeliveryRecord { + const parsed = deliveryRecordSchema.parse({ + ...value, + updatedAt: new Date().toISOString(), + }); + store.setDelivery(runId, JSON.stringify(parsed), eventType, details); + return parsed; +} + +function cancellationRequested(store: StateStore, runId: string): boolean { + return store.getRun(runId).cancelRequested; +} + +function stopCancelledDelivery( + store: StateStore, + runId: string, + delivery: DeliveryRecord, +): never { + const current = store.getRun(runId); + if (!current.cancelRequested) { + throw new MillError( + "CANCELLATION_STATE_INVALID", + "Remote delivery cancellation was requested without durable state.", + ExitCode.data, + ); + } + persistDelivery( + store, + runId, + { ...delivery, state: "cancelled", lastErrorCode: "OPERATOR_CANCELLED" }, + "delivery.cancelled", + ); + if (current.status !== "cancelled") { + store.transition(runId, "cancelled", "run.cancelled", { + code: "OPERATOR_CANCELLED", + }); + } + throw new MillError( + "OPERATOR_CANCELLED", + "The operator cancelled remote delivery before another external effect.", + ExitCode.temporary, + ); +} + +function assertRemoteMutationNotCancelled( + store: StateStore, + runId: string, + delivery: DeliveryRecord, +): void { + if (cancellationRequested(store, runId)) { + stopCancelledDelivery(store, runId, delivery); + } +} + +function setRunBlocker( + store: StateStore, + run: RunRecord, + code: string, + eventType: string, + details: Record = {}, +): RunRecord { + return run.status === "blocked" + ? store.replaceBlocker(run.id, code, eventType, details) + : store.transition(run.id, "blocked", eventType, { code, ...details }); +} + +function reconcileAbsentEffect( + store: StateStore, + run: RunRecord, + delivery: DeliveryRecord, + effectValue: RemoteEffect, +): { run: PublicRunRecord; delivery: DeliveryRecord } { + if (cancellationRequested(store, run.id)) { + stopCancelledDelivery(store, run.id, delivery); + } + if (effectValue.attemptCount >= maximumRemoteEffectAttempts) { + const blocked = persistDelivery( + store, + run.id, + { + ...upsertEffect(delivery, { + ...effectValue, + status: "blocked", + errorCode: "REMOTE_EFFECT_RETRY_EXHAUSTED", + updatedAt: new Date().toISOString(), + }), + state: "blocked", + lastErrorCode: "REMOTE_EFFECT_RETRY_EXHAUSTED", + }, + "delivery.retry_exhausted", + { effectId: effectValue.id }, + ); + const blockedRun = setRunBlocker( + store, + run, + "REMOTE_EFFECT_RETRY_EXHAUSTED", + "delivery.blocked", + ); + return { run: publicRunRecord(blockedRun), delivery: blocked }; + } + const retryable = persistDelivery( + store, + run.id, + { + ...upsertEffect(delivery, { + ...effectValue, + status: "retryable_absent", + errorCode: null, + updatedAt: new Date().toISOString(), + }), + state: "proposing", + lastErrorCode: null, + }, + "delivery.effect_absent", + { effectId: effectValue.id, attemptCount: effectValue.attemptCount }, + ); + const retryableRun = store.transition( + run.id, + "proposing", + "delivery.retry_authorized", + ); + return { run: publicRunRecord(retryableRun), delivery: retryable }; +} + +async function assertReviewedCandidate( + root: string, + run: RunRecord, + inputs: RuntimeInputs, +): Promise<{ commit: string; tree: string }> { + if (run.status !== "reviewed" && run.status !== "proposing") { + throw new MillError( + "RUN_NOT_REVIEWED", + "Only an exact locally reviewed candidate may enter draft-PR delivery.", + ExitCode.configuration, + ); + } + if (run.validationJson === undefined || run.reviewJson === undefined) { + throw new MillError( + "LOCAL_EVIDENCE_MISSING", + "Draft-PR delivery requires exact validation and local review evidence.", + ExitCode.configuration, + ); + } + let validation: z.infer; + let review: z.infer; + try { + validation = validationEvidenceSchema.parse(JSON.parse(run.validationJson)); + review = reviewResultSchema.parse(JSON.parse(run.reviewJson)); + } catch (error) { + throw new MillError( + "LOCAL_EVIDENCE_INVALID", + "Stored validation or local review evidence is invalid.", + ExitCode.data, + { cause: String(error) }, + ); + } + if ( + !validation.passed || + validation.candidateCommit !== run.candidateCommit || + review.candidateCommit !== run.candidateCommit || + review.findings.length > 0 + ) { + throw new MillError( + "LOCAL_EVIDENCE_STALE", + "Validation and local review must pass on the exact candidate head.", + ExitCode.configuration, + ); + } + const candidate = await assertRunBindings(root, run, inputs); + return { commit: candidate.commit, tree: candidate.tree }; +} + +function expectedRemoteUrls( + config: ProposeConfig, + cloneUrl: string, +): Set { + return new Set([ + cloneUrl, + `git@${config.host}:${config.owner}/${config.repository}.git`, + `ssh://git@${config.host}/${config.owner}/${config.repository}.git`, + ]); +} + +async function assertBinding( + root: string, + config: ProposeConfig, + binding: GitHubBinding, +): Promise { + const expectedFullName = `${config.owner}/${config.repository}`; + const remote = await repositoryRemoteUrl(root, config.remoteName); + if ( + binding.repositoryNodeId !== config.repositoryNodeId || + binding.fullName !== expectedFullName || + binding.defaultBranch !== config.baseBranch || + binding.fork || + !config.allowedActors.includes(binding.actorLogin) || + !expectedRemoteUrls(config, binding.cloneUrl).has(remote) + ) { + throw new MillError( + "GITHUB_BINDING_MISMATCH", + "The live actor, repository, default branch, node identity, or local remote does not match mill.yaml.", + ExitCode.configuration, + { + actorLogin: binding.actorLogin, + fullName: binding.fullName, + repositoryNodeId: binding.repositoryNodeId, + }, + ); + } +} + +function branchName(config: ProposeConfig, run: RunRecord): string { + return `${config.branchPrefix}${run.taskId.slice(0, 32)}-${run.id.slice(0, 8)}`; +} + +function target( + config: ProposeConfig, + binding: GitHubBinding, +): DeliveryRecord["target"] { + return { + forge: "github", + host: config.host, + owner: config.owner, + repository: config.repository, + repositoryNodeId: binding.repositoryNodeId, + cloneUrl: binding.cloneUrl, + remoteName: config.remoteName, + baseBranch: config.baseBranch, + actorLogin: binding.actorLogin, + actorId: binding.actorId, + }; +} + +function proposalDigest(input: { + run: RunRecord; + candidate: { commit: string; tree: string }; + target: DeliveryRecord["target"]; + branchName: string; + approvalExpiresAt: string; + config: ProposeConfig; +}): string { + return canonicalDigest({ + schemaVersion: "1", + runId: input.run.id, + taskDigest: input.run.taskDigest, + configDigest: input.run.configDigest, + candidateCommit: input.candidate.commit, + candidateTree: input.candidate.tree, + target: input.target, + branchName: input.branchName, + approvalExpiresAt: input.approvalExpiresAt, + requiredChecks: input.config.requiredChecks, + reviewPolicy: input.config.reviewPolicy, + allowedMergeMethods: input.config.allowedMergeMethods, + allowedMergerLogins: input.config.allowedMergerLogins, + effects: ["push_exact_candidate", "create_draft_pull_request"], + }); +} + +function effectId( + deliveryKey: string, + kind: RemoteEffect["kind"], + candidateCommit: string, +): string { + return canonicalDigest({ + schemaVersion: "1", + deliveryKey, + kind, + candidateCommit, + }); +} + +function effect( + delivery: DeliveryRecord, + kind: RemoteEffect["kind"], + candidateCommit: string, +): RemoteEffect | undefined { + return delivery.effects.find( + (item) => item.id === effectId(delivery.deliveryKey, kind, candidateCommit), + ); +} + +function upsertEffect( + delivery: DeliveryRecord, + value: RemoteEffect, +): DeliveryRecord { + return { + ...delivery, + effects: [ + ...delivery.effects.filter((item) => item.id !== value.id), + value, + ], + }; +} + +function deliveryMarker(deliveryKey: string): string { + return ``; +} + +function exactPullRequest( + pullRequest: GitHubPullRequest, + delivery: DeliveryRecord, +): boolean { + return ( + pullRequest.headRef === delivery.branchName && + pullRequest.baseRef === delivery.target.baseBranch && + pullRequest.body.includes(deliveryMarker(delivery.deliveryKey)) + ); +} + +function assertExactPullRequest( + pullRequest: GitHubPullRequest, + delivery: DeliveryRecord, + requireDraft: boolean, +): void { + const recorded = delivery.pullRequest; + if ( + !exactPullRequest(pullRequest, delivery) || + (recorded !== null && + (pullRequest.number !== recorded.number || + pullRequest.nodeId !== recorded.nodeId)) || + pullRequest.headSha !== delivery.candidateCommit || + pullRequest.state !== "open" || + (requireDraft && !pullRequest.draft) + ) { + throw new MillError( + "PULL_REQUEST_IDENTITY_MISMATCH", + "GitHub pull request identity does not match the exact delivery plan.", + ExitCode.configuration, + ); + } +} + +function assertPushBoundaryPullRequest( + pullRequest: GitHubPullRequest | null, + delivery: DeliveryRecord, + expectedOldCommit: string | null, +): void { + const recorded = delivery.pullRequest; + if (recorded === null) { + if (pullRequest !== null) { + throw new MillError( + "PULL_REQUEST_IDENTITY_MISMATCH", + "An unrecorded pull request conflicts with the push boundary readback.", + ExitCode.configuration, + ); + } + return; + } + if ( + pullRequest === null || + !exactPullRequest(pullRequest, delivery) || + pullRequest.number !== recorded.number || + pullRequest.nodeId !== recorded.nodeId || + pullRequest.headSha !== expectedOldCommit || + pullRequest.state !== "open" || + !pullRequest.draft + ) { + throw new MillError( + "PULL_REQUEST_IDENTITY_MISMATCH", + "The recorded pull request is no longer the expected open draft at the observed branch head.", + ExitCode.configuration, + ); + } +} + +function assertDeliveryContinuity(input: { + run: RunRecord; + inputs: RuntimeInputs; + config: ProposeConfig; + delivery: DeliveryRecord; + binding: GitHubBinding; +}): void { + const { run, inputs, config, delivery, binding } = input; + if ( + run.taskDigest !== inputs.taskDigest || + run.configDigest !== inputs.configDigest || + delivery.runId !== run.id || + delivery.candidateCommit !== run.candidateCommit || + delivery.candidateTree !== run.candidateTree || + delivery.target.owner !== config.owner || + delivery.target.repository !== config.repository || + delivery.target.repositoryNodeId !== config.repositoryNodeId || + delivery.target.remoteName !== config.remoteName || + delivery.target.baseBranch !== config.baseBranch || + delivery.target.actorLogin !== binding.actorLogin || + delivery.target.actorId !== binding.actorId || + delivery.target.repositoryNodeId !== binding.repositoryNodeId || + delivery.target.cloneUrl !== binding.cloneUrl || + JSON.stringify(delivery.requiredChecks) !== + JSON.stringify(config.requiredChecks) || + JSON.stringify(delivery.reviewPolicy) !== + JSON.stringify(config.reviewPolicy) || + JSON.stringify(delivery.allowedMergerLogins) !== + JSON.stringify(config.allowedMergerLogins) || + JSON.stringify(delivery.allowedMergeMethods) !== + JSON.stringify(config.allowedMergeMethods) + ) { + throw new MillError( + "DELIVERY_AUTHORITY_DRIFT", + "The task, configuration, candidate, actor, or repository changed after delivery approval.", + ExitCode.configuration, + ); + } +} + +function checkDecision( + required: readonly string[], + checks: readonly GitHubCheck[], +): { + status: "passed" | "pending" | "failed"; + missing: string[]; + failed: string[]; +} { + const missing: string[] = []; + const failed: string[] = []; + let pending = false; + for (const name of required) { + const matching = checks.filter((check) => check.name === name); + if (matching.length === 0) { + missing.push(name); + pending = true; + continue; + } + if ( + matching.some( + (check) => check.status !== "completed" || check.conclusion === null, + ) + ) { + pending = true; + continue; + } + if (!matching.every((check) => check.conclusion === "success")) { + failed.push(name); + } + } + return { + status: failed.length > 0 ? "failed" : pending ? "pending" : "passed", + missing, + failed, + }; +} + +function actionableFeedback( + observation: GitHubObservation, + reviewPolicy: DeliveryRecord["reviewPolicy"], + candidateCommit: string, +): GitHubFeedback[] { + if (reviewPolicy.mode !== "github_required") return []; + return observation.feedback.filter( + (item) => + item.commitId === candidateCommit && + reviewPolicy.requiredReviewerLogins.includes(item.actorLogin) && + item.priority !== "P3", + ); +} + +function reviewsPassed( + observation: GitHubObservation, + reviewPolicy: DeliveryRecord["reviewPolicy"], + candidateCommit: string, +): boolean { + if (reviewPolicy.mode === "local_only") return true; + return reviewPolicy.requiredReviewerLogins.every((login) => { + const latest = observation.reviews + .filter( + (review) => + review.actorLogin === login && review.commitId === candidateCommit, + ) + .at(-1); + return latest?.state === "APPROVED" || latest?.state === "COMMENTED"; + }); +} + +function feedbackAsReview( + candidateCommit: string, + feedback: readonly GitHubFeedback[], +): z.infer { + return reviewResultSchema.parse({ + schemaVersion: "1", + candidateCommit, + summary: "GitHub review requires one attended systemic repair.", + findings: feedback.map((item) => ({ + id: `github-${item.id}`, + severity: item.priority === "unclassified" ? "P1" : item.priority, + class: "correctness", + title: + item.body.split("\n", 1)[0]?.slice(0, 200) ?? "GitHub review finding", + body: `${item.body}\n\nSource: ${item.url}`, + file: item.path, + line: item.line, + })), + }); +} + +async function reconcileReadback(input: { + adapter: GitHubAdapter; + config: ProposeConfig; + delivery: DeliveryRecord; + deadlineMs: number; + signal?: AbortSignal; +}): Promise<{ + branchSha: string | null; + pullRequest: GitHubPullRequest | null; +}> { + const [branchSha, pullRequests] = await Promise.all([ + input.adapter.readBranch({ + config: input.config, + branch: input.delivery.branchName, + deadlineMs: input.deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }), + input.adapter.findPullRequests({ + config: input.config, + branch: input.delivery.branchName, + deadlineMs: input.deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }), + ]); + const matching = pullRequests.filter((item) => + exactPullRequest(item, input.delivery), + ); + if (matching.length > 1) { + throw new MillError( + "DUPLICATE_PULL_REQUESTS", + "More than one pull request carries the immutable delivery identity.", + ExitCode.configuration, + ); + } + return { branchSha, pullRequest: matching[0] ?? null }; +} + +export async function planDraftPr(input: { + root: string; + taskPath: string; + runId: string; + adapter?: GitHubAdapter; + signal?: AbortSignal; +}): Promise<{ run: PublicRunRecord; delivery: DeliveryRecord }> { + const context = await openDeliveryContext(input.root, input.taskPath); + const { inputs, config, store } = context; + let lease: Awaited> | undefined; + try { + lease = await acquireWriterLease(store); + const run = store.getRun(input.runId); + const candidate = await assertReviewedCandidate(input.root, run, inputs); + const adapter = input.adapter ?? createGitHubAdapter(input.root); + const binding = await adapter.inspect({ + config, + deadlineMs: operationDeadline(config), + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + await assertBinding(input.root, config, binding); + const plannedBranch = branchName(config, run); + const approvalExpiresAt = new Date( + Date.now() + config.approvalTtlSeconds * 1000, + ).toISOString(); + const plannedTarget = target(config, binding); + const existing = + run.deliveryJson === undefined ? undefined : storedDelivery(run); + const deliveryKey = + existing?.deliveryKey ?? + canonicalDigest({ + schemaVersion: "1", + repositoryId: inputs.config.repositoryId, + runId: run.id, + repositoryNodeId: binding.repositoryNodeId, + branchName: plannedBranch, + }); + const now = new Date().toISOString(); + const delivery = deliveryRecordSchema.parse({ + schemaVersion: "1", + runId: run.id, + deliveryKey, + proposalDigest: proposalDigest({ + run, + candidate, + target: plannedTarget, + branchName: plannedBranch, + approvalExpiresAt, + config, + }), + approvalExpiresAt, + state: "planned", + target: plannedTarget, + branchName: plannedBranch, + candidateCommit: candidate.commit, + candidateTree: candidate.tree, + requiredChecks: config.requiredChecks, + reviewPolicy: config.reviewPolicy, + allowedMergerLogins: config.allowedMergerLogins, + allowedMergeMethods: config.allowedMergeMethods, + effects: existing?.effects ?? [], + remoteHeadCommit: existing?.remoteHeadCommit ?? null, + pullRequest: existing?.pullRequest ?? null, + observation: null, + merge: null, + lastErrorCode: null, + createdAt: existing?.createdAt ?? now, + updatedAt: now, + }); + const persisted = persistDelivery( + store, + run.id, + delivery, + "delivery.planned", + { candidateCommit: candidate.commit, deliveryKey }, + ); + const updatedRun = + run.status === "reviewed" + ? store.transition(run.id, "proposing", "delivery.awaiting_approval") + : store.getRun(run.id); + return { run: publicRunRecord(updatedRun), delivery: persisted }; + } finally { + try { + await lease?.release(); + } finally { + store.close(); + } + } +} + +function markUnknown( + store: StateStore, + run: RunRecord, + delivery: DeliveryRecord, + effectValue: RemoteEffect, + code: string, +): never { + const unknown = persistDelivery( + store, + run.id, + { + ...upsertEffect(delivery, { + ...effectValue, + status: "effect_unknown", + errorCode: code, + updatedAt: new Date().toISOString(), + }), + state: "effect_unknown", + lastErrorCode: code, + }, + "delivery.effect_unknown", + { effectId: effectValue.id, code }, + ); + const current = store.getRun(run.id); + if (current.status !== "effect_unknown") { + store.transition( + run.id, + "effect_unknown", + "delivery.reconciliation_required", + { + code, + }, + ); + } + throw new MillError( + code, + "A GitHub effect has an unknown outcome; run pr reconcile before any further mutation.", + ExitCode.temporary, + { deliveryKey: unknown.deliveryKey, effectId: effectValue.id }, + ); +} + +export async function openDraftPr(input: { + root: string; + taskPath: string; + runId: string; + approvalDigest: string; + attended: boolean; + adapter?: GitHubAdapter; + signal?: AbortSignal; +}): Promise<{ run: PublicRunRecord; delivery: DeliveryRecord }> { + if (!input.attended) { + throw new MillError( + "ATTENDED_ACKNOWLEDGEMENT_REQUIRED", + "Draft-PR mutation requires an explicit attended acknowledgement.", + ExitCode.configuration, + ); + } + const context = await openDeliveryContext(input.root, input.taskPath); + const { inputs, config, store } = context; + let lease: Awaited> | undefined; + try { + lease = await acquireWriterLease(store); + let run = store.getRun(input.runId); + if (run.status !== "proposing") { + throw new MillError( + "DELIVERY_NOT_APPLICABLE", + "Draft-PR open requires an approved plan with no unknown external effect.", + ExitCode.configuration, + ); + } + const candidate = await assertReviewedCandidate(input.root, run, inputs); + let delivery = storedDelivery(run); + assertRemoteMutationNotCancelled(store, run.id, delivery); + if ( + delivery.candidateCommit !== candidate.commit || + delivery.candidateTree !== candidate.tree || + delivery.proposalDigest !== input.approvalDigest || + Date.parse(delivery.approvalExpiresAt) <= Date.now() + ) { + throw new MillError( + "DELIVERY_APPROVAL_MISMATCH", + "Approval is missing, expired, or bound to another candidate or delivery plan.", + ExitCode.configuration, + ); + } + const adapter = input.adapter ?? createGitHubAdapter(input.root); + const deadlineMs = operationDeadline(config); + const binding = await adapter.inspect({ + config, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + await assertBinding(input.root, config, binding); + const liveDigest = proposalDigest({ + run, + candidate, + target: target(config, binding), + branchName: delivery.branchName, + approvalExpiresAt: delivery.approvalExpiresAt, + config, + }); + if (liveDigest !== input.approvalDigest) { + throw new MillError( + "DELIVERY_BINDING_DRIFT", + "Live GitHub identity or proposal policy changed after approval.", + ExitCode.configuration, + ); + } + delivery = persistDelivery( + store, + run.id, + { ...delivery, state: "proposing", lastErrorCode: null }, + "delivery.started", + ); + let readback = await reconcileReadback({ + adapter, + config, + delivery, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + assertPushBoundaryPullRequest( + readback.pullRequest, + delivery, + readback.branchSha, + ); + let push = effect(delivery, "push", candidate.commit); + if (readback.branchSha !== candidate.commit) { + if ( + readback.branchSha !== null && + readback.branchSha !== delivery.remoteHeadCommit + ) { + throw new MillError( + "REMOTE_BRANCH_CONFLICT", + "The Mill delivery branch moved outside the exact expected-head precondition.", + ExitCode.configuration, + ); + } + if ( + push !== undefined && + (push.status === "call_started" || push.status === "effect_unknown") + ) { + markUnknown(store, run, delivery, push, "GITHUB_PUSH_OUTCOME_UNKNOWN"); + } + if (push?.status === "verified") { + throw new MillError( + "REMOTE_BRANCH_CONFLICT", + "A previously verified Mill branch no longer has its bound candidate head.", + ExitCode.configuration, + ); + } + if ((push?.attemptCount ?? 0) >= maximumRemoteEffectAttempts) { + throw new MillError( + "REMOTE_EFFECT_RETRY_EXHAUSTED", + "The exact push retry budget is exhausted.", + ExitCode.configuration, + ); + } + push = { + ...(push ?? { + id: effectId(delivery.deliveryKey, "push", candidate.commit), + kind: "push" as const, + candidateCommit: candidate.commit, + attemptCount: 0, + expectedOldCommit: delivery.remoteHeadCommit, + }), + status: "intent", + errorCode: null, + updatedAt: new Date().toISOString(), + }; + delivery = persistDelivery( + store, + run.id, + upsertEffect(delivery, push), + "delivery.push_intent", + { effectId: push.id }, + ); + readback = await reconcileReadback({ + adapter, + config, + delivery, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + if (readback.branchSha !== push.expectedOldCommit) { + throw new MillError( + "REMOTE_BRANCH_CONFLICT", + "The Mill delivery branch changed after push intent and before the mutation boundary.", + ExitCode.configuration, + ); + } + assertPushBoundaryPullRequest( + readback.pullRequest, + delivery, + push.expectedOldCommit, + ); + push = { + ...push, + status: "call_started", + attemptCount: push.attemptCount + 1, + updatedAt: new Date().toISOString(), + }; + delivery = persistDelivery( + store, + run.id, + upsertEffect(delivery, push), + "delivery.push_started", + { effectId: push.id }, + ); + assertRemoteMutationNotCancelled(store, run.id, delivery); + try { + await adapter.pushExact({ + root: input.root, + config, + cloneUrl: delivery.target.cloneUrl, + branch: delivery.branchName, + candidateCommit: candidate.commit, + expectedOldCommit: push.expectedOldCommit, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + cancellationRequested: () => cancellationRequested(store, run.id), + }); + } catch (error) { + const failure = asMillError(error); + try { + readback = await reconcileReadback({ + adapter, + config, + delivery, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + } catch { + markUnknown(store, run, delivery, push, failure.code); + } + if (readback.branchSha !== candidate.commit) { + markUnknown(store, run, delivery, push, failure.code); + } + } + readback = await reconcileReadback({ + adapter, + config, + delivery, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + assertPushBoundaryPullRequest( + readback.pullRequest, + delivery, + readback.branchSha, + ); + if (readback.branchSha !== candidate.commit) { + markUnknown( + store, + run, + delivery, + push, + "GITHUB_PUSH_READBACK_MISMATCH", + ); + } + } + if (push === undefined) { + push = { + id: effectId(delivery.deliveryKey, "push", candidate.commit), + kind: "push", + candidateCommit: candidate.commit, + status: "verified", + attemptCount: 0, + expectedOldCommit: delivery.remoteHeadCommit, + errorCode: null, + updatedAt: new Date().toISOString(), + }; + } else { + push = { + ...push, + status: "verified", + errorCode: null, + updatedAt: new Date().toISOString(), + }; + } + delivery = persistDelivery( + store, + run.id, + { + ...upsertEffect(delivery, push), + remoteHeadCommit: candidate.commit, + }, + "delivery.push_verified", + { effectId: push.id, candidateCommit: candidate.commit }, + ); + assertRemoteMutationNotCancelled(store, run.id, delivery); + readback = await reconcileReadback({ + adapter, + config, + delivery, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + let pullRequest = readback.pullRequest; + let prEffect = effect(delivery, "pull_request", candidate.commit); + if (pullRequest === null && delivery.pullRequest !== null) { + throw new MillError( + "PULL_REQUEST_DISAPPEARED", + "The recorded pull request cannot be found by immutable delivery identity.", + ExitCode.configuration, + ); + } + if (pullRequest === null) { + if ( + prEffect !== undefined && + (prEffect.status === "call_started" || + prEffect.status === "effect_unknown") + ) { + markUnknown( + store, + run, + delivery, + prEffect, + "GITHUB_PR_OUTCOME_UNKNOWN", + ); + } + if (prEffect?.status === "verified" || delivery.pullRequest !== null) { + throw new MillError( + "PULL_REQUEST_DISAPPEARED", + "A previously verified pull request cannot be found by immutable delivery identity.", + ExitCode.configuration, + ); + } + if ((prEffect?.attemptCount ?? 0) >= maximumRemoteEffectAttempts) { + throw new MillError( + "REMOTE_EFFECT_RETRY_EXHAUSTED", + "The draft pull-request retry budget is exhausted.", + ExitCode.configuration, + ); + } + prEffect = { + ...(prEffect ?? { + id: effectId(delivery.deliveryKey, "pull_request", candidate.commit), + kind: "pull_request" as const, + candidateCommit: candidate.commit, + attemptCount: 0, + expectedOldCommit: null, + }), + status: "intent", + errorCode: null, + updatedAt: new Date().toISOString(), + }; + delivery = persistDelivery( + store, + run.id, + upsertEffect(delivery, prEffect), + "delivery.pull_request_intent", + { effectId: prEffect.id }, + ); + prEffect = { + ...prEffect, + status: "call_started", + attemptCount: prEffect.attemptCount + 1, + updatedAt: new Date().toISOString(), + }; + delivery = persistDelivery( + store, + run.id, + upsertEffect(delivery, prEffect), + "delivery.pull_request_started", + { effectId: prEffect.id }, + ); + assertRemoteMutationNotCancelled(store, run.id, delivery); + try { + pullRequest = await adapter.createDraftPullRequest({ + config, + branch: delivery.branchName, + title: inputs.task.commit.message.slice(0, 240), + body: `${deliveryMarker(delivery.deliveryKey)}\n\nGenerated by Mill from an attended, locally validated and reviewed run. The current candidate identity is the pull-request head; configured human merge authority remains external.`, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + cancellationRequested: () => cancellationRequested(store, run.id), + }); + } catch (error) { + const failure = asMillError(error); + try { + readback = await reconcileReadback({ + adapter, + config, + delivery, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + pullRequest = readback.pullRequest; + } catch { + markUnknown(store, run, delivery, prEffect, failure.code); + } + if (pullRequest === null) { + markUnknown(store, run, delivery, prEffect, failure.code); + } + } + assertExactPullRequest(pullRequest, delivery, true); + prEffect = { + ...prEffect, + status: "verified", + errorCode: null, + updatedAt: new Date().toISOString(), + }; + delivery = upsertEffect(delivery, prEffect); + } else { + assertExactPullRequest(pullRequest, delivery, true); + prEffect = { + ...(prEffect ?? { + id: effectId(delivery.deliveryKey, "pull_request", candidate.commit), + kind: "pull_request" as const, + candidateCommit: candidate.commit, + attemptCount: 0, + expectedOldCommit: null, + }), + status: "verified", + errorCode: null, + updatedAt: new Date().toISOString(), + }; + delivery = upsertEffect(delivery, prEffect); + } + delivery = persistDelivery( + store, + run.id, + { + ...delivery, + state: "awaiting_ci", + pullRequest: { + number: pullRequest.number, + nodeId: pullRequest.nodeId, + url: pullRequest.url, + }, + observation: null, + lastErrorCode: null, + }, + "delivery.pull_request_verified", + { pullRequestNumber: pullRequest.number }, + ); + assertRemoteMutationNotCancelled(store, run.id, delivery); + run = store.getRun(run.id); + if (run.status !== "awaiting_ci") { + run = store.transition(run.id, "awaiting_ci", "delivery.awaiting_ci"); + } + return { run: publicRunRecord(run), delivery }; + } finally { + try { + await lease?.release(); + } finally { + store.close(); + } + } +} + +export async function reconcileDraftPr(input: { + root: string; + taskPath: string; + runId: string; + adapter?: GitHubAdapter; + signal?: AbortSignal; +}): Promise<{ run: PublicRunRecord; delivery: DeliveryRecord }> { + const context = await openDeliveryContext(input.root, input.taskPath); + const { inputs, config, store } = context; + let lease: Awaited> | undefined; + try { + lease = await acquireWriterLease(store); + let run = store.getRun(input.runId); + if (run.status !== "effect_unknown") { + throw new MillError( + "RECONCILIATION_NOT_REQUIRED", + "The run has no unknown GitHub effect to reconcile.", + ExitCode.configuration, + ); + } + let delivery = storedDelivery(run); + const adapter = input.adapter ?? createGitHubAdapter(input.root); + const deadlineMs = operationDeadline(config); + const binding = await adapter.inspect({ + config, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + await assertBinding(input.root, config, binding); + assertDeliveryContinuity({ run, inputs, config, delivery, binding }); + const readback = await reconcileReadback({ + adapter, + config, + delivery, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + const unknownEffects = delivery.effects.filter( + (item) => item.status === "effect_unknown", + ); + if (unknownEffects.length !== 1) { + throw new MillError( + "GITHUB_RECONCILIATION_STATE_INVALID", + "Exactly one external effect must be unknown before reconciliation.", + ExitCode.data, + ); + } + const unknownEffect = unknownEffects[0]; + if (unknownEffect === undefined) { + throw new MillError( + "GITHUB_RECONCILIATION_STATE_INVALID", + "The unknown external effect could not be identified.", + ExitCode.data, + ); + } + if ( + unknownEffect.kind === "push" && + (readback.branchSha === unknownEffect.expectedOldCommit || + readback.branchSha === delivery.candidateCommit) + ) { + assertPushBoundaryPullRequest( + readback.pullRequest, + delivery, + readback.branchSha, + ); + } + let effectAbsent = false; + if (unknownEffect.kind === "pull_request") { + effectAbsent = + readback.pullRequest === null && + readback.branchSha === delivery.candidateCommit; + } else if (readback.branchSha === unknownEffect.expectedOldCommit) { + effectAbsent = true; + } + if (effectAbsent) { + return reconcileAbsentEffect(store, run, delivery, unknownEffect); + } + if ( + unknownEffect.kind === "push" && + readback.branchSha === delivery.candidateCommit && + readback.pullRequest === null + ) { + delivery = persistDelivery( + store, + run.id, + { + ...upsertEffect(delivery, { + ...unknownEffect, + status: "verified", + errorCode: null, + updatedAt: new Date().toISOString(), + }), + state: "proposing", + remoteHeadCommit: readback.branchSha, + lastErrorCode: null, + }, + "delivery.push_reconciled", + { effectId: unknownEffect.id }, + ); + assertRemoteMutationNotCancelled(store, run.id, delivery); + run = store.transition(run.id, "proposing", "delivery.reconciled"); + return { run: publicRunRecord(run), delivery }; + } + if (readback.branchSha !== delivery.candidateCommit) { + throw new MillError( + "REMOTE_BRANCH_CONFLICT", + "Authoritative readback found a branch head outside the reconciled effect precondition.", + ExitCode.configuration, + ); + } + if (readback.pullRequest === null) { + throw new MillError( + "GITHUB_RECONCILIATION_STATE_INVALID", + "Reconciliation could not classify the pull-request effect.", + ExitCode.data, + ); + } + assertExactPullRequest(readback.pullRequest, delivery, true); + delivery = persistDelivery( + store, + run.id, + { + ...delivery, + state: "awaiting_ci", + remoteHeadCommit: readback.branchSha, + pullRequest: { + number: readback.pullRequest.number, + nodeId: readback.pullRequest.nodeId, + url: readback.pullRequest.url, + }, + effects: delivery.effects.map((item) => + item.id === unknownEffect.id + ? { + ...item, + status: "verified" as const, + errorCode: null, + updatedAt: new Date().toISOString(), + } + : item, + ), + lastErrorCode: null, + }, + "delivery.reconciled", + { pullRequestNumber: readback.pullRequest.number }, + ); + assertRemoteMutationNotCancelled(store, run.id, delivery); + run = store.transition(run.id, "awaiting_ci", "delivery.awaiting_ci"); + return { run: publicRunRecord(run), delivery }; + } finally { + try { + await lease?.release(); + } finally { + store.close(); + } + } +} + +function assertObservationIdentity( + observation: GitHubObservation, + delivery: DeliveryRecord, + allowDeletedBranch: boolean, +): void { + const pull = observation.pullRequest; + if ( + !exactPullRequest(pull, delivery) || + pull.number !== delivery.pullRequest?.number || + pull.nodeId !== delivery.pullRequest.nodeId || + pull.headSha !== delivery.candidateCommit || + (!allowDeletedBranch && + observation.branchSha !== delivery.candidateCommit) || + (allowDeletedBranch && + observation.branchSha !== null && + observation.branchSha !== delivery.candidateCommit) + ) { + throw new MillError( + "REMOTE_IDENTITY_DRIFT", + "GitHub branch or pull request identity drifted from the exact delivery record.", + ExitCode.configuration, + ); + } +} + +export async function observeDraftPr(input: { + root: string; + taskPath: string; + runId: string; + adapter?: GitHubAdapter; + signal?: AbortSignal; +}): Promise<{ run: PublicRunRecord; delivery: DeliveryRecord }> { + const context = await openDeliveryContext(input.root, input.taskPath); + const { inputs, config, store } = context; + let lease: Awaited> | undefined; + try { + lease = await acquireWriterLease(store); + let run = store.getRun(input.runId); + if ( + run.status !== "awaiting_ci" && + run.status !== "awaiting_human" && + !( + run.status === "blocked" && + ["REMOTE_CHECKS_FAILED", "REMOTE_REVIEW_FINDINGS"].includes( + run.blockCode ?? "", + ) + ) + ) { + throw new MillError( + "DELIVERY_NOT_OBSERVABLE", + "Run has no draft pull request awaiting policy observation.", + ExitCode.configuration, + ); + } + let delivery = storedDelivery(run); + if (delivery.pullRequest === null) { + throw new MillError( + "PULL_REQUEST_IDENTITY_MISSING", + "Delivery has no verified pull request identity.", + ExitCode.configuration, + ); + } + const adapter = input.adapter ?? createGitHubAdapter(input.root); + const deadlineMs = operationDeadline(config); + const binding = await adapter.inspect({ + config, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + await assertBinding(input.root, config, binding); + assertDeliveryContinuity({ run, inputs, config, delivery, binding }); + const observation = await adapter.observe({ + config, + pullRequestNumber: delivery.pullRequest.number, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + assertObservationIdentity(observation, delivery, false); + const checks = checkDecision(delivery.requiredChecks, observation.checks); + const feedback = actionableFeedback( + observation, + delivery.reviewPolicy, + delivery.candidateCommit, + ); + const observationRecord = { + headSha: observation.pullRequest.headSha, + branchSha: observation.branchSha, + checkDecision: checks, + checks: observation.checks, + reviews: observation.reviews, + feedback, + observedAt: new Date().toISOString(), + }; + if (feedback.length > 0) { + store.setRemoteFeedback( + run.id, + JSON.stringify(feedbackAsReview(delivery.candidateCommit, feedback)), + ); + delivery = persistDelivery( + store, + run.id, + { + ...delivery, + state: "blocked", + observation: observationRecord, + lastErrorCode: "REMOTE_REVIEW_FINDINGS", + }, + "delivery.remote_review_blocked", + { findings: feedback.length }, + ); + run = setRunBlocker( + store, + run, + "REMOTE_REVIEW_FINDINGS", + "delivery.blocked", + ); + return { run: publicRunRecord(run), delivery }; + } + if (checks.status === "failed") { + delivery = persistDelivery( + store, + run.id, + { + ...delivery, + state: "blocked", + observation: observationRecord, + lastErrorCode: "REMOTE_CHECKS_FAILED", + }, + "delivery.remote_checks_failed", + { failed: checks.failed.length }, + ); + run = setRunBlocker( + store, + run, + "REMOTE_CHECKS_FAILED", + "delivery.blocked", + ); + return { run: publicRunRecord(run), delivery }; + } + const ready = + checks.status === "passed" && + reviewsPassed( + observation, + delivery.reviewPolicy, + delivery.candidateCommit, + ); + const nextState = ready ? "awaiting_human" : "awaiting_ci"; + delivery = persistDelivery( + store, + run.id, + { + ...delivery, + state: nextState, + observation: observationRecord, + lastErrorCode: null, + }, + ready ? "delivery.awaiting_human" : "delivery.policy_pending", + ); + if (run.status === "blocked") { + run = store.transition(run.id, "awaiting_ci", "delivery.reobserved"); + } + if (ready && run.status !== "awaiting_human") { + run = store.transition( + run.id, + "awaiting_human", + "delivery.awaiting_human", + ); + } else if (!ready && run.status !== "awaiting_ci") { + run = store.transition(run.id, "awaiting_ci", "delivery.policy_pending"); + } + return { run: publicRunRecord(run), delivery }; + } finally { + try { + await lease?.release(); + } finally { + store.close(); + } + } +} + +function mergeMethod( + observation: GitHubObservation, +): "merge" | "linear_tree_preserving" { + const commit = observation.mergeCommit; + if (commit === null) { + throw new MillError( + "MERGE_COMMIT_MISSING", + "GitHub did not return the exact merge commit identity.", + ExitCode.data, + ); + } + if (commit.parents.length >= 2) return "merge"; + if (commit.parents.length === 1) { + return "linear_tree_preserving"; + } + throw new MillError( + "MERGE_TOPOLOGY_INVALID", + "The observed merge commit has no parent and cannot be classified safely.", + ExitCode.data, + ); +} + +export async function finalizeDraftPr(input: { + root: string; + taskPath: string; + runId: string; + adapter?: GitHubAdapter; + signal?: AbortSignal; +}): Promise<{ run: PublicRunRecord; delivery: DeliveryRecord }> { + const context = await openDeliveryContext(input.root, input.taskPath); + const { inputs, config, store } = context; + let lease: Awaited> | undefined; + try { + lease = await acquireWriterLease(store); + let run = store.getRun(input.runId); + if ( + run.status !== "awaiting_human" && + run.status !== "merged" && + run.status !== "post_merge_verified" && + !( + run.status === "blocked" && run.blockCode === "POST_MERGE_CHECKS_FAILED" + ) + ) { + throw new MillError( + "DELIVERY_NOT_FINALIZABLE", + "Run is not awaiting human merge or post-merge verification.", + ExitCode.configuration, + ); + } + let delivery = storedDelivery(run); + if (delivery.pullRequest === null) { + throw new MillError( + "PULL_REQUEST_IDENTITY_MISSING", + "Delivery has no verified pull request identity.", + ExitCode.configuration, + ); + } + const adapter = input.adapter ?? createGitHubAdapter(input.root); + const deadlineMs = operationDeadline(config); + const binding = await adapter.inspect({ + config, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + await assertBinding(input.root, config, binding); + assertDeliveryContinuity({ run, inputs, config, delivery, binding }); + const observation = await adapter.observe({ + config, + pullRequestNumber: delivery.pullRequest.number, + deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }); + assertObservationIdentity(observation, delivery, true); + if (!observation.pullRequest.merged) { + if (observation.pullRequest.state === "closed") { + throw new MillError( + "PULL_REQUEST_CLOSED_UNMERGED", + "The delivery pull request was closed without a merge.", + ExitCode.configuration, + ); + } + throw new MillError( + "HUMAN_MERGE_PENDING", + "The draft pull request has not been merged by the human owner.", + ExitCode.temporary, + ); + } + if (observation.mergeCommit === null) { + throw new MillError( + "MERGE_READBACK_MISMATCH", + "GitHub readback cannot prove the exact merge commit.", + ExitCode.configuration, + ); + } + const mergeCommit = observation.mergeCommit; + if ( + observation.pullRequest.mergeCommitSha !== mergeCommit.sha || + observation.pullRequest.mergedByLogin === null || + observation.pullRequest.mergedAt === null || + !observation.mergeIsOnDefaultBranch + ) { + throw new MillError( + "MERGE_READBACK_MISMATCH", + "GitHub readback cannot prove the exact merge is contained by the default branch.", + ExitCode.configuration, + ); + } + const mergedByLogin = observation.pullRequest.mergedByLogin; + if (!delivery.allowedMergerLogins.includes(mergedByLogin)) { + throw new MillError( + "MERGER_NOT_ALLOWED", + "The pull request was merged by an identity outside the approved human merge authority.", + ExitCode.configuration, + { mergedByLogin }, + ); + } + const method = mergeMethod(observation); + if (!delivery.allowedMergeMethods.includes(method)) { + throw new MillError( + "MERGE_METHOD_NOT_ALLOWED", + "The observed merge method is outside the repository policy.", + ExitCode.configuration, + ); + } + if (mergeCommit.tree !== delivery.candidateTree) { + throw new MillError( + "MERGE_TREE_REVALIDATION_REQUIRED", + "The merged tree differs from the reviewed candidate and requires a fresh exact-tree validation before closure.", + ExitCode.configuration, + ); + } + delivery = persistDelivery( + store, + run.id, + { + ...delivery, + state: "merged", + merge: { + commit: mergeCommit.sha, + tree: mergeCommit.tree, + method, + mergedByLogin, + mergedAt: observation.pullRequest.mergedAt, + defaultBranchHead: observation.defaultBranchHead, + }, + observation: { + mergeChecks: observation.mergeChecks, + observedAt: new Date().toISOString(), + }, + lastErrorCode: null, + }, + "delivery.merge_verified", + { mergeCommit: mergeCommit.sha, method }, + ); + if (run.status !== "merged" && run.status !== "post_merge_verified") { + run = store.transition(run.id, "merged", "delivery.merged"); + } + const checks = checkDecision( + delivery.requiredChecks, + observation.mergeChecks, + ); + if (checks.status === "pending") { + return { run: publicRunRecord(run), delivery }; + } + if (checks.status === "failed") { + delivery = persistDelivery( + store, + run.id, + { + ...delivery, + state: "blocked", + lastErrorCode: "POST_MERGE_CHECKS_FAILED", + }, + "delivery.post_merge_checks_failed", + { failed: checks.failed.length }, + ); + run = setRunBlocker( + store, + run, + "POST_MERGE_CHECKS_FAILED", + "delivery.blocked", + ); + return { run: publicRunRecord(run), delivery }; + } + if (run.status !== "post_merge_verified") { + run = store.transition( + run.id, + "post_merge_verified", + "delivery.post_merge_verified", + ); + } + delivery = persistDelivery( + store, + run.id, + { ...delivery, state: "closed", lastErrorCode: null }, + "delivery.closed", + ); + run = store.transition(run.id, "closed", "run.closed"); + return { run: publicRunRecord(run), delivery }; + } finally { + try { + await lease?.release(); + } finally { + store.close(); + } + } +} diff --git a/src/runtime/github.ts b/src/runtime/github.ts new file mode 100644 index 0000000..221c17f --- /dev/null +++ b/src/runtime/github.ts @@ -0,0 +1,880 @@ +import path from "node:path"; + +import type { MillConfig } from "./inputs.js"; +import { findTrustedExecutable } from "../doctor.js"; +import { ExitCode, MillError } from "../errors.js"; +import { runProcess } from "./process.js"; + +export type ProposeConfig = NonNullable; + +export interface GitHubBinding { + actorLogin: string; + actorId: number; + repositoryNodeId: string; + fullName: string; + cloneUrl: string; + defaultBranch: string; + fork: boolean; +} + +export interface GitHubPullRequest { + number: number; + nodeId: string; + url: string; + state: "open" | "closed"; + draft: boolean; + body: string; + headRef: string; + headSha: string; + baseRef: string; + merged: boolean; + mergeCommitSha: string | null; + mergedByLogin: string | null; + mergedAt: string | null; +} + +export interface GitHubCheck { + name: string; + status: string; + conclusion: string | null; +} + +export interface GitHubReview { + id: string; + actorLogin: string; + state: string; + commitId: string | null; + body: string; + url: string; +} + +export interface GitHubFeedback { + id: string; + actorLogin: string; + priority: "P0" | "P1" | "P2" | "P3" | "unclassified"; + body: string; + path: string | null; + line: number | null; + url: string; + commitId: string; +} + +export interface GitHubCommit { + sha: string; + tree: string; + parents: readonly string[]; +} + +export interface GitHubObservation { + pullRequest: GitHubPullRequest; + branchSha: string | null; + checks: readonly GitHubCheck[]; + mergeChecks: readonly GitHubCheck[]; + reviews: readonly GitHubReview[]; + feedback: readonly GitHubFeedback[]; + defaultBranchHead: string; + mergeCommit: GitHubCommit | null; + mergeIsOnDefaultBranch: boolean; +} + +export interface GitHubAdapter { + inspect(input: { + config: ProposeConfig; + deadlineMs: number; + signal?: AbortSignal; + }): Promise; + readBranch(input: { + config: ProposeConfig; + branch: string; + deadlineMs: number; + signal?: AbortSignal; + }): Promise; + pushExact(input: { + root: string; + config: ProposeConfig; + cloneUrl: string; + branch: string; + candidateCommit: string; + expectedOldCommit: string | null; + deadlineMs: number; + signal?: AbortSignal; + cancellationRequested?: () => boolean; + }): Promise; + findPullRequests(input: { + config: ProposeConfig; + branch: string; + deadlineMs: number; + signal?: AbortSignal; + }): Promise; + createDraftPullRequest(input: { + config: ProposeConfig; + branch: string; + title: string; + body: string; + deadlineMs: number; + signal?: AbortSignal; + cancellationRequested?: () => boolean; + }): Promise; + observe(input: { + config: ProposeConfig; + pullRequestNumber: number; + deadlineMs: number; + signal?: AbortSignal; + }): Promise; +} + +interface ProcessLifecycle { + deadlineMs: number; + signal?: AbortSignal; + cancellationRequested?: () => boolean; +} + +function commandEnvironment(): NodeJS.ProcessEnv { + return { + HOME: process.env.HOME, + XDG_CONFIG_HOME: process.env.XDG_CONFIG_HOME, + GH_CONFIG_DIR: process.env.GH_CONFIG_DIR, + PATH: "/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/homebrew/bin", + LANG: "C", + LC_ALL: "C", + GH_PROMPT_DISABLED: "1", + GH_NO_UPDATE_NOTIFIER: "1", + GIT_CONFIG_GLOBAL: "/dev/null", + GIT_CONFIG_NOSYSTEM: "1", + GIT_NO_REPLACE_OBJECTS: "1", + GIT_OPTIONAL_LOCKS: "0", + GIT_TERMINAL_PROMPT: "0", + GIT_PAGER: "cat", + PAGER: "cat", + }; +} + +function assertSha(value: unknown, label: string): string { + if (typeof value !== "string" || !/^[a-f0-9]{40}$/u.test(value)) { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + `GitHub returned an invalid ${label}.`, + ExitCode.data, + ); + } + return value; +} + +function object(value: unknown, label: string): Record { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + `GitHub returned an invalid ${label}.`, + ExitCode.data, + ); + } + return value as Record; +} + +function text(value: unknown, label: string): string { + if (typeof value !== "string" || value.length === 0) { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + `GitHub returned an invalid ${label}.`, + ExitCode.data, + ); + } + return value; +} + +function integer(value: unknown, label: string): number { + if (!Number.isSafeInteger(value) || (value as number) <= 0) { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + `GitHub returned an invalid ${label}.`, + ExitCode.data, + ); + } + return value as number; +} + +function boolean(value: unknown, label: string): boolean { + if (typeof value !== "boolean") { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + `GitHub returned an invalid ${label}.`, + ExitCode.data, + ); + } + return value; +} + +function parsePullRequest( + value: unknown, + requireMergedFlag: boolean, +): GitHubPullRequest { + const item = object(value, "pull request"); + const head = object(item.head, "pull request head"); + const base = object(item.base, "pull request base"); + const state = item.state; + if (state !== "open" && state !== "closed") { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + "GitHub returned an invalid pull request state.", + ExitCode.data, + ); + } + return { + number: integer(item.number, "pull request number"), + nodeId: text(item.node_id, "pull request node ID"), + url: text(item.html_url, "pull request URL"), + state, + draft: boolean(item.draft, "pull request draft flag"), + body: typeof item.body === "string" ? item.body : "", + headRef: text(head.ref, "pull request head ref"), + headSha: assertSha(head.sha, "pull request head SHA"), + baseRef: text(base.ref, "pull request base ref"), + merged: + item.merged === undefined && !requireMergedFlag + ? false + : boolean(item.merged, "pull request merged flag"), + mergeCommitSha: + item.merge_commit_sha === null + ? null + : assertSha(item.merge_commit_sha, "merge commit SHA"), + mergedByLogin: + item.merged_by === null || item.merged_by === undefined + ? null + : text( + object(item.merged_by, "merge actor").login, + "merge actor login", + ), + mergedAt: + item.merged_at === null || item.merged_at === undefined + ? null + : text(item.merged_at, "merge timestamp"), + }; +} + +function priority(body: string): GitHubFeedback["priority"] { + const match = /(?:\[|\b)(P[0-3])(?:\]|\b)/iu.exec(body); + return ( + (match?.[1]?.toUpperCase() as GitHubFeedback["priority"] | undefined) ?? + "unclassified" + ); +} + +function parseChecks(checkValue: unknown, statusValue: unknown): GitHubCheck[] { + const checksObject = object(checkValue, "check runs"); + const checkRuns = Array.isArray(checksObject.check_runs) + ? checksObject.check_runs + : []; + const statusObject = object(statusValue, "commit statuses"); + const statuses = Array.isArray(statusObject.statuses) + ? statusObject.statuses + : []; + return [ + ...checkRuns.map((raw) => { + const item = object(raw, "check run"); + return { + name: text(item.name, "check name"), + status: text(item.status, "check status"), + conclusion: + item.conclusion === null + ? null + : text(item.conclusion, "check conclusion"), + }; + }), + ...statuses.map((raw) => { + const item = object(raw, "commit status"); + const state = text(item.state, "commit status state"); + return { + name: text(item.context, "commit status context"), + status: state === "pending" ? "in_progress" : "completed", + conclusion: state === "success" ? "success" : state, + }; + }), + ]; +} + +function paginatedArray(value: unknown, label: string): unknown[] { + if (!Array.isArray(value)) { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + `GitHub returned invalid paginated ${label}.`, + ExitCode.data, + ); + } + const items: unknown[] = []; + for (const page of value) { + if (!Array.isArray(page)) { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + `GitHub returned an invalid ${label} page.`, + ExitCode.data, + ); + } + items.push(...(page as unknown[])); + } + return items; +} + +function paginatedObjectCollection( + value: unknown, + property: string, + label: string, +): Record { + if (!Array.isArray(value)) { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + `GitHub returned invalid paginated ${label}.`, + ExitCode.data, + ); + } + const items: unknown[] = []; + for (const rawPage of value) { + const page = object(rawPage, `${label} page`); + const values = page[property]; + if (!Array.isArray(values)) { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + `GitHub returned an invalid ${label} collection.`, + ExitCode.data, + ); + } + items.push(...(values as unknown[])); + } + return { [property]: items }; +} + +class GhGitHubAdapter implements GitHubAdapter { + readonly #root: string; + + constructor(root: string) { + this.#root = root; + } + + async #ghJson( + args: readonly string[], + lifecycle: ProcessLifecycle, + allowNotFound = false, + ): Promise { + const gh = await findTrustedExecutable("gh", this.#root); + if (gh === undefined) { + throw new MillError( + "GH_UNAVAILABLE", + "A trusted gh executable is required for GitHub operations.", + ExitCode.unavailable, + ); + } + const result = await runProcess({ + executable: gh, + args, + cwd: this.#root, + env: commandEnvironment(), + deadlineMs: lifecycle.deadlineMs, + maxOutputBytes: 4 * 1024 * 1024, + ...(lifecycle.signal === undefined ? {} : { signal: lifecycle.signal }), + ...(lifecycle.cancellationRequested === undefined + ? {} + : { cancellationRequested: lifecycle.cancellationRequested }), + }); + if ( + allowNotFound && + result.exitCode !== 0 && + /HTTP 404/iu.test(result.stderr) + ) { + return null; + } + if (result.timedOut || result.cancelled || result.outputExceeded) { + throw new MillError( + result.cancelled + ? "GITHUB_CANCELLED" + : result.timedOut + ? "GITHUB_DEADLINE_EXCEEDED" + : "GITHUB_OUTPUT_BUDGET_EXCEEDED", + "The GitHub operation did not complete within its approved bounds.", + ExitCode.temporary, + ); + } + if (result.exitCode !== 0) { + throw new MillError( + "GITHUB_CALL_FAILED", + "The GitHub operation failed without a verified effect receipt.", + ExitCode.temporary, + { exitCode: result.exitCode }, + ); + } + try { + return JSON.parse(result.stdout) as unknown; + } catch (error) { + throw new MillError( + "INVALID_GITHUB_RESPONSE", + "GitHub returned invalid JSON.", + ExitCode.data, + { cause: String(error) }, + ); + } + } + + async inspect(input: { + config: ProposeConfig; + deadlineMs: number; + signal?: AbortSignal; + }): Promise { + const lifecycle = { + deadlineMs: input.deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }; + const [actorValue, repositoryValue] = await Promise.all([ + this.#ghJson(["api", "--hostname", input.config.host, "user"], lifecycle), + this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + `repos/${input.config.owner}/${input.config.repository}`, + ], + lifecycle, + ), + ]); + const actor = object(actorValue, "actor"); + const repository = object(repositoryValue, "repository"); + return { + actorLogin: text(actor.login, "actor login"), + actorId: integer(actor.id, "actor ID"), + repositoryNodeId: text(repository.node_id, "repository node ID"), + fullName: text(repository.full_name, "repository full name"), + cloneUrl: text(repository.clone_url, "repository clone URL"), + defaultBranch: text(repository.default_branch, "default branch"), + fork: boolean(repository.fork, "repository fork flag"), + }; + } + + async readBranch(input: { + config: ProposeConfig; + branch: string; + deadlineMs: number; + signal?: AbortSignal; + }): Promise { + const value = await this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + `repos/${input.config.owner}/${input.config.repository}/git/ref/heads/${encodeURIComponent(input.branch)}`, + ], + { + deadlineMs: input.deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }, + true, + ); + if (value === null) return null; + return assertSha( + object(object(value, "branch ref").object, "branch object").sha, + "branch SHA", + ); + } + + async pushExact(input: { + root: string; + config: ProposeConfig; + cloneUrl: string; + branch: string; + candidateCommit: string; + expectedOldCommit: string | null; + deadlineMs: number; + signal?: AbortSignal; + cancellationRequested?: () => boolean; + }): Promise { + if (!/^mill\/[A-Za-z0-9._-]+$/u.test(input.branch)) { + throw new MillError( + "INVALID_DELIVERY_BRANCH", + "The delivery branch is outside Mill's fixed branch namespace.", + ExitCode.configuration, + ); + } + const expectedClone = `https://${input.config.host}/${input.config.owner}/${input.config.repository}.git`; + if (input.cloneUrl !== expectedClone) { + throw new MillError( + "GITHUB_REPOSITORY_BINDING_MISMATCH", + "GitHub clone URL does not match the approved repository identity.", + ExitCode.configuration, + ); + } + const [git, gh] = await Promise.all([ + findTrustedExecutable("git", input.root), + findTrustedExecutable("gh", input.root), + ]); + if (git === undefined || gh === undefined) { + throw new MillError( + "SHIPPER_TOOL_UNAVAILABLE", + "Trusted git and gh executables are required for an exact push.", + ExitCode.unavailable, + ); + } + if (!/^[A-Za-z0-9_./ -]+$/u.test(gh)) { + throw new MillError( + "UNSAFE_GH_EXECUTABLE_PATH", + "The gh executable path cannot be represented safely as a Git credential helper.", + ExitCode.configuration, + ); + } + const lease = `--force-with-lease=refs/heads/${input.branch}:${input.expectedOldCommit ?? ""}`; + const result = await runProcess({ + executable: git, + args: [ + "-c", + "core.hooksPath=/dev/null", + "-c", + "core.fsmonitor=false", + "-c", + "credential.helper=", + "-c", + `credential.helper=!"${gh}" auth git-credential`, + "-c", + "credential.useHttpPath=true", + "push", + "--porcelain", + "--no-verify", + lease, + input.cloneUrl, + `${input.candidateCommit}:refs/heads/${input.branch}`, + ], + cwd: input.root, + env: commandEnvironment(), + deadlineMs: input.deadlineMs, + maxOutputBytes: 1024 * 1024, + ...(input.signal === undefined ? {} : { signal: input.signal }), + ...(input.cancellationRequested === undefined + ? {} + : { cancellationRequested: input.cancellationRequested }), + }); + if ( + result.exitCode !== 0 || + result.timedOut || + result.cancelled || + result.outputExceeded + ) { + throw new MillError( + "GITHUB_PUSH_OUTCOME_UNKNOWN", + "The exact Git push did not return a verified receipt; authoritative readback is required.", + ExitCode.temporary, + { exitCode: result.exitCode }, + ); + } + } + + async findPullRequests(input: { + config: ProposeConfig; + branch: string; + deadlineMs: number; + signal?: AbortSignal; + }): Promise { + const query = new URLSearchParams({ + state: "all", + head: `${input.config.owner}:${input.branch}`, + base: input.config.baseBranch, + per_page: "100", + }); + const value = await this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + "--paginate", + "--slurp", + `repos/${input.config.owner}/${input.config.repository}/pulls?${query.toString()}`, + ], + { + deadlineMs: input.deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }, + ); + return paginatedArray(value, "pull request collection").map((item) => + parsePullRequest(item, false), + ); + } + + async createDraftPullRequest(input: { + config: ProposeConfig; + branch: string; + title: string; + body: string; + deadlineMs: number; + signal?: AbortSignal; + cancellationRequested?: () => boolean; + }): Promise { + const value = await this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + "--method", + "POST", + `repos/${input.config.owner}/${input.config.repository}/pulls`, + "--raw-field", + `title=${input.title}`, + "--raw-field", + `head=${input.branch}`, + "--raw-field", + `base=${input.config.baseBranch}`, + "--raw-field", + `body=${input.body}`, + "--field", + "draft=true", + ], + { + deadlineMs: input.deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + ...(input.cancellationRequested === undefined + ? {} + : { cancellationRequested: input.cancellationRequested }), + }, + ); + return parsePullRequest(value, true); + } + + async observe(input: { + config: ProposeConfig; + pullRequestNumber: number; + deadlineMs: number; + signal?: AbortSignal; + }): Promise { + const lifecycle = { + deadlineMs: input.deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }; + const prefix = `repos/${input.config.owner}/${input.config.repository}`; + const pullValue = await this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + `${prefix}/pulls/${input.pullRequestNumber}`, + ], + lifecycle, + ); + const pullRequest = parsePullRequest(pullValue, true); + const [ + branchSha, + checkValue, + statusValue, + reviewsValue, + commentsValue, + defaultRefValue, + ] = await Promise.all([ + this.readBranch({ + config: input.config, + branch: pullRequest.headRef, + deadlineMs: input.deadlineMs, + ...(input.signal === undefined ? {} : { signal: input.signal }), + }), + this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + "--paginate", + "--slurp", + `${prefix}/commits/${pullRequest.headSha}/check-runs?filter=latest&per_page=100`, + ], + lifecycle, + ), + this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + "--paginate", + "--slurp", + `${prefix}/commits/${pullRequest.headSha}/status?per_page=100`, + ], + lifecycle, + ), + this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + "--paginate", + "--slurp", + `${prefix}/pulls/${input.pullRequestNumber}/reviews?per_page=100`, + ], + lifecycle, + ), + this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + "--paginate", + "--slurp", + `${prefix}/pulls/${input.pullRequestNumber}/comments?per_page=100`, + ], + lifecycle, + ), + this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + `${prefix}/git/ref/heads/${encodeURIComponent(input.config.baseBranch)}`, + ], + lifecycle, + ), + ]); + const checks = parseChecks( + paginatedObjectCollection(checkValue, "check_runs", "check runs"), + paginatedObjectCollection(statusValue, "statuses", "commit statuses"), + ); + const reviews = paginatedArray(reviewsValue, "reviews").map( + (raw): GitHubReview => { + const item = object(raw, "review"); + const user = object(item.user, "review actor"); + return { + id: String(integer(item.id, "review ID")), + actorLogin: text(user.login, "review actor login"), + state: text(item.state, "review state").toUpperCase(), + commitId: + item.commit_id === null || item.commit_id === undefined + ? null + : assertSha(item.commit_id, "review commit ID"), + body: typeof item.body === "string" ? item.body : "", + url: text(item.html_url, "review URL"), + }; + }, + ); + const inlineFeedback = paginatedArray(commentsValue, "review comments").map( + (raw): GitHubFeedback => { + const item = object(raw, "review comment"); + const user = object(item.user, "review comment actor"); + const body = typeof item.body === "string" ? item.body : ""; + return { + id: String(integer(item.id, "review comment ID")), + actorLogin: text(user.login, "review comment actor login"), + priority: priority(body), + body, + path: text(item.path, "review comment path"), + line: Number.isSafeInteger(item.line) ? (item.line as number) : null, + url: text(item.html_url, "review comment URL"), + commitId: assertSha(item.commit_id, "review comment commit ID"), + }; + }, + ); + const reviewFeedback = reviews.flatMap((review): GitHubFeedback[] => { + const reviewPriority = priority(review.body); + if ( + review.body.trim().length === 0 || + review.commitId === null || + reviewPriority === "unclassified" + ) { + return []; + } + return [ + { + id: `review-${review.id}`, + actorLogin: review.actorLogin, + priority: reviewPriority, + body: review.body, + path: null, + line: null, + url: review.url, + commitId: review.commitId, + }, + ]; + }); + const feedback = [...reviewFeedback, ...inlineFeedback]; + const defaultBranchHead = assertSha( + object( + object(defaultRefValue, "default branch ref").object, + "default branch object", + ).sha, + "default branch SHA", + ); + let mergeCommit: GitHubCommit | null = null; + let mergeChecks: GitHubCheck[] = []; + let mergeIsOnDefaultBranch = false; + if (pullRequest.merged && pullRequest.mergeCommitSha !== null) { + const [commitValue, compareValue, mergeCheckValue, mergeStatusValue] = + await Promise.all([ + this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + `${prefix}/git/commits/${pullRequest.mergeCommitSha}`, + ], + lifecycle, + ), + this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + `${prefix}/compare/${pullRequest.mergeCommitSha}...${defaultBranchHead}`, + ], + lifecycle, + ), + this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + "--paginate", + "--slurp", + `${prefix}/commits/${pullRequest.mergeCommitSha}/check-runs?filter=latest&per_page=100`, + ], + lifecycle, + ), + this.#ghJson( + [ + "api", + "--hostname", + input.config.host, + "--paginate", + "--slurp", + `${prefix}/commits/${pullRequest.mergeCommitSha}/status?per_page=100`, + ], + lifecycle, + ), + ]); + const commit = object(commitValue, "merge commit"); + const treeObject = object(commit.tree, "merge commit tree"); + const parents = Array.isArray(commit.parents) ? commit.parents : []; + mergeCommit = { + sha: assertSha(commit.sha, "merge commit SHA"), + tree: assertSha(treeObject.sha, "merge tree SHA"), + parents: parents.map((raw) => + assertSha(object(raw, "merge parent").sha, "merge parent SHA"), + ), + }; + const compare = object(compareValue, "default branch comparison"); + mergeIsOnDefaultBranch = + compare.status === "ahead" || compare.status === "identical"; + mergeChecks = parseChecks( + paginatedObjectCollection( + mergeCheckValue, + "check_runs", + "merge check runs", + ), + paginatedObjectCollection( + mergeStatusValue, + "statuses", + "merge commit statuses", + ), + ); + } + return { + pullRequest, + branchSha, + checks, + mergeChecks, + reviews, + feedback, + defaultBranchHead, + mergeCommit, + mergeIsOnDefaultBranch, + }; + } +} + +export function createGitHubAdapter(root: string): GitHubAdapter { + return new GhGitHubAdapter(path.resolve(root)); +} diff --git a/src/runtime/lifecycle.ts b/src/runtime/lifecycle.ts index 3819bb2..076b79b 100644 --- a/src/runtime/lifecycle.ts +++ b/src/runtime/lifecycle.ts @@ -41,6 +41,7 @@ import { } from "./repository.js"; import { acquireWriterLease, + isPurgeSafeRun, isTerminalRun, purgeRepositoryState, publicRunRecord, @@ -189,9 +190,16 @@ function storedManifest(run: RunRecord): ContextManifest { function storedReviewFindings( run: RunRecord, ): ReturnType["findings"] | undefined { - if (run.reviewJson === undefined) return undefined; + const source = + run.blockCode === "REMOTE_REVIEW_FINDINGS" + ? run.remoteFeedbackJson + : run.blockCode === "REVIEW_FINDINGS" || + run.blockCode === "REVIEW_NON_CONVERGENCE" + ? run.reviewJson + : undefined; + if (source === undefined) return undefined; try { - const parsed = reviewResultSchema.safeParse(JSON.parse(run.reviewJson)); + const parsed = reviewResultSchema.safeParse(JSON.parse(source)); if ( !parsed.success || parsed.data.candidateCommit !== run.candidateCommit @@ -204,7 +212,7 @@ function storedReviewFindings( } catch (error) { throw new MillError( "REVIEW_EVIDENCE_INVALID", - "Stored review evidence is invalid or bound to another candidate.", + "Stored local or remote review evidence is invalid or bound to another candidate.", ExitCode.data, { cause: String(error) }, ); @@ -242,7 +250,7 @@ function storedGitControl(run: RunRecord): GitControlSnapshot { } } -async function assertRunBindings( +export async function assertRunBindings( root: string, run: RunRecord, inputs: RuntimeInputs, @@ -300,7 +308,7 @@ function safeBlock(store: StateStore, runId: string, error: MillError): void { const run = store.getRun(runId); if (isTerminalRun(run.status)) return; if (run.status === "blocked") { - store.recordEvent(runId, "run.blocked_again", { code: error.code }); + store.replaceBlocker(runId, error.code, "run.blocker_replaced"); return; } store.transition(runId, "blocked", "run.blocked", { code: error.code }); @@ -317,6 +325,12 @@ function settleFailure( try { const run = store.getRun(runId); if (isTerminalRun(run.status)) return; + if (run.status === "effect_unknown") { + store.recordEvent(runId, "run.reconciliation_required", { + code: error.code, + }); + return; + } if (run.cancelRequested) { store.transition(runId, "cancelled", "run.cancelled", { code: error.code, @@ -831,6 +845,13 @@ export async function resumeRun(input: { } store.setActiveProcess(run.id, null); run = store.getRun(run.id); + if (run.status === "effect_unknown") { + throw new MillError( + "GITHUB_RECONCILIATION_REQUIRED", + "An unknown GitHub effect must be reconciled before cancellation or local resume.", + ExitCode.temporary, + ); + } if (run.cancelRequested && !isTerminalRun(run.status)) { return publicRunRecord( store.transition(run.id, "cancelled", "run.cancelled", { @@ -985,6 +1006,12 @@ export async function cancelRun(input: { if (isTerminalRun(current.status)) { return publicRunRecord(current); } + if (current.status === "effect_unknown") { + store.recordEvent(current.id, "run.cancellation_pending", { + code: "GITHUB_RECONCILIATION_REQUIRED", + }); + return publicRunRecord(current); + } const active = storedActiveProcess(current); if (active !== undefined && processIdentityStatus(active) !== "mismatch") { store.recordEvent(current.id, "run.cancellation_pending", { @@ -1024,7 +1051,7 @@ export async function runStatus(input: { input.runId === undefined ? store.latestRun() : store.getRun(input.runId); if (run === undefined) return {}; let interrupted = false; - let reconciliationRequired = false; + let reconciliationRequired = run.status === "effect_unknown"; const active = storedActiveProcess(run); let controllerAbsent = false; if ( @@ -1080,7 +1107,7 @@ export async function stateBackup(input: { root: string }): Promise { export async function stateRestore(input: { root: string; backupPath: string; -}): Promise { +}): Promise>> { const config = await loadMillConfig(input.root); const commonDirectory = await commonGitDirectory(input.root); const store = await StateStore.open(config.repositoryId, commonDirectory); @@ -1088,7 +1115,7 @@ export async function stateRestore(input: { try { lease = await acquireWriterLease(store); store.close(); - await restoreStateBackup( + return await restoreStateBackup( config.repositoryId, commonDirectory, input.backupPath, @@ -1118,10 +1145,10 @@ export async function statePurge(input: { try { lease = await acquireWriterLease(store); const runs = store.runs(); - if (runs.some((run) => !isTerminalRun(run.status))) { + if (runs.some((run) => !isPurgeSafeRun(run.status))) { throw new MillError( "ACTIVE_RUNS_BLOCK_PURGE", - "All runs must be terminal before state can be purged.", + "All runs must be locally reviewed or terminal before state can be purged.", ExitCode.configuration, ); } diff --git a/src/runtime/repository.ts b/src/runtime/repository.ts index a52fd21..57496b5 100644 --- a/src/runtime/repository.ts +++ b/src/runtime/repository.ts @@ -188,6 +188,40 @@ export async function resolveCommit( return value; } +export async function repositoryRemoteUrl( + root: string, + remoteName: string, +): Promise { + if (!/^[A-Za-z0-9._-]+$/u.test(remoteName)) { + throw new MillError( + "INVALID_REMOTE_NAME", + "The configured Git remote name is invalid.", + ExitCode.configuration, + ); + } + const values = (await git(root, ["remote", "get-url", "--all", remoteName])) + .split("\n") + .map((value) => value.trim()) + .filter((value) => value.length > 0); + if (values.length !== 1) { + throw new MillError( + "AMBIGUOUS_REMOTE_URL", + "The configured Git remote must resolve to exactly one URL.", + ExitCode.configuration, + { remoteName, count: values.length }, + ); + } + const remoteUrl = values[0]; + if (remoteUrl === undefined) { + throw new MillError( + "AMBIGUOUS_REMOTE_URL", + "The configured Git remote did not resolve to a URL.", + ExitCode.configuration, + ); + } + return remoteUrl; +} + async function assertNoDangerousAttributes( root: string, baseCommit: string, diff --git a/src/runtime/state.ts b/src/runtime/state.ts index 41fd05f..5f63970 100644 --- a/src/runtime/state.ts +++ b/src/runtime/state.ts @@ -6,8 +6,10 @@ import { copyFile, lstat, mkdir, + readdir, rename, rm, + writeFile, } from "node:fs/promises"; import { homedir } from "node:os"; import path from "node:path"; @@ -23,6 +25,13 @@ export type RunStatus = | "committed" | "verified" | "reviewed" + | "proposing" + | "effect_unknown" + | "awaiting_ci" + | "awaiting_human" + | "merged" + | "post_merge_verified" + | "closed" | "blocked" | "cancelled" | "failed" @@ -53,6 +62,8 @@ export interface RunRecord { blockCode?: string; validationJson?: string; reviewJson?: string; + deliveryJson?: string; + remoteFeedbackJson?: string; createdAt: string; updatedAt: string; } @@ -65,6 +76,8 @@ export type PublicRunRecord = Omit< | "activeProcessId" | "activeProcessGroup" | "activeProcessIdentity" + | "deliveryJson" + | "remoteFeedbackJson" >; export function publicRunRecord(run: RunRecord): PublicRunRecord { @@ -75,6 +88,8 @@ export function publicRunRecord(run: RunRecord): PublicRunRecord { delete publicRun.activeProcessId; delete publicRun.activeProcessGroup; delete publicRun.activeProcessIdentity; + delete publicRun.deliveryJson; + delete publicRun.remoteFeedbackJson; return publicRun; } @@ -103,33 +118,67 @@ interface RunRow { block_code: string | null; validation_json: string | null; review_json: string | null; + delivery_json: string | null; + remote_feedback_json: string | null; created_at: string; updated_at: string; } -const terminal = new Set([ - "reviewed", - "cancelled", - "failed", - "stale", -]); +const terminal = new Set(["closed", "cancelled", "failed", "stale"]); export function isTerminalRun(status: RunStatus): boolean { return terminal.has(status); } +export function isPurgeSafeRun(status: RunStatus): boolean { + return status === "reviewed" || isTerminalRun(status); +} + const transitions: Readonly> = { approved: ["ready", "blocked", "cancelled", "failed"], ready: ["running", "blocked", "cancelled", "failed", "stale"], running: ["committed", "blocked", "cancelled", "failed", "stale"], committed: ["verified", "blocked", "cancelled", "failed", "stale"], verified: ["reviewed", "blocked", "cancelled", "failed", "stale"], - reviewed: ["stale"], + reviewed: ["proposing", "cancelled", "failed", "stale"], + proposing: [ + "effect_unknown", + "awaiting_ci", + "blocked", + "cancelled", + "failed", + "stale", + ], + effect_unknown: [ + "proposing", + "awaiting_ci", + "blocked", + "cancelled", + "failed", + "stale", + ], + awaiting_ci: [ + "awaiting_human", + "blocked", + "effect_unknown", + "cancelled", + "failed", + "stale", + ], + awaiting_human: ["merged", "blocked", "failed", "stale"], + merged: ["post_merge_verified", "blocked", "failed", "stale"], + post_merge_verified: ["closed", "blocked", "failed", "stale"], + closed: [], blocked: [ "ready", "running", "committed", "verified", + "proposing", + "awaiting_ci", + "awaiting_human", + "merged", + "post_merge_verified", "cancelled", "failed", "stale", @@ -217,6 +266,10 @@ function fromRow(row: RunRow): RunRecord { ? {} : { validationJson: row.validation_json }), ...(row.review_json === null ? {} : { reviewJson: row.review_json }), + ...(row.delivery_json === null ? {} : { deliveryJson: row.delivery_json }), + ...(row.remote_feedback_json === null + ? {} + : { remoteFeedbackJson: row.remote_feedback_json }), createdAt: row.created_at, updatedAt: row.updated_at, }; @@ -288,6 +341,8 @@ export class StateStore { block_code TEXT, validation_json TEXT, review_json TEXT, + delivery_json TEXT, + remote_feedback_json TEXT, created_at TEXT NOT NULL, updated_at TEXT NOT NULL ) STRICT; @@ -319,6 +374,8 @@ export class StateStore { "active_process_id TEXT", "active_process_group INTEGER", "active_process_identity TEXT", + "delivery_json TEXT", + "remote_feedback_json TEXT", ]) { const name = column.split(" ")[0]; if (!runColumns.some((candidate) => candidate.name === name)) { @@ -513,7 +570,8 @@ export class StateStore { this.#database .prepare( `UPDATE runs SET candidate_commit = ?, candidate_tree = ?, - validation_json = NULL, review_json = NULL, status = 'committed', + validation_json = NULL, review_json = NULL, + remote_feedback_json = NULL, status = 'committed', block_code = NULL, updated_at = ? WHERE id = ?`, ) .run(commit, tree, new Date().toISOString(), id); @@ -604,6 +662,64 @@ export class StateStore { return this.getRun(id); } + setDelivery( + id: string, + deliveryJson: string, + eventType: string, + details: Record = {}, + ): RunRecord { + this.#transaction(() => { + this.getRun(id); + this.#database + .prepare( + "UPDATE runs SET delivery_json = ?, updated_at = ? WHERE id = ?", + ) + .run(deliveryJson, new Date().toISOString(), id); + this.#event(id, eventType, details); + }); + return this.getRun(id); + } + + setRemoteFeedback(id: string, feedbackJson: string): RunRecord { + this.#transaction(() => { + this.getRun(id); + this.#database + .prepare( + "UPDATE runs SET remote_feedback_json = ?, updated_at = ? WHERE id = ?", + ) + .run(feedbackJson, new Date().toISOString(), id); + this.#event(id, "remote.feedback_recorded", {}); + }); + return this.getRun(id); + } + + replaceBlocker( + id: string, + code: string, + eventType: string, + details: Record = {}, + ): RunRecord { + this.#transaction(() => { + const current = this.getRun(id); + if (current.status !== "blocked") { + throw new MillError( + "INVALID_RUN_TRANSITION", + `Cannot replace a blocker while run is ${current.status}.`, + ExitCode.configuration, + ); + } + this.#database + .prepare("UPDATE runs SET block_code = ?, updated_at = ? WHERE id = ?") + .run(code, new Date().toISOString(), id); + this.#event(id, eventType, { + from: current.blockCode ?? null, + to: code, + ...details, + }); + }); + return this.getRun(id); + } + beginRepair(id: string): RunRecord { this.#transaction(() => { const current = this.getRun(id); @@ -1006,7 +1122,7 @@ export async function restoreStateBackup( repositoryId: string, commonDirectory: string, backupPath: string, -): Promise { +): Promise { const directory = repositoryStateDirectory(repositoryId, commonDirectory); const resolvedBackup = path.resolve(backupPath); if (!isWithin(directory, resolvedBackup)) { @@ -1026,6 +1142,9 @@ export async function restoreStateBackup( } const databasePath = path.join(directory, "state.sqlite3"); const temporaryPath = path.join(directory, `restore-${randomUUID()}.sqlite3`); + const expectedWorktrees = new Set(); + let quarantineManifest: string | undefined; + const moved: { original: string; quarantined: string }[] = []; try { await copyFile(resolvedBackup, temporaryPath, constants.COPYFILE_EXCL); await chmod(temporaryPath, 0o600); @@ -1050,6 +1169,11 @@ export async function restoreStateBackup( OR (type = 'trigger' AND name IN ('run_events_no_update', 'run_events_no_delete'))`, ) .all() as unknown as { name: string }[]; + const worktrees = candidate + .prepare( + "SELECT worktree_path FROM runs WHERE worktree_path IS NOT NULL", + ) + .all() as unknown as { worktree_path: string }[]; if ( integrity?.integrity_check !== "ok" || version?.value !== "1" || @@ -1057,6 +1181,14 @@ export async function restoreStateBackup( ) { throw new Error("backup integrity, schema version, or objects invalid"); } + const worktreesDirectory = path.join(directory, "worktrees"); + for (const row of worktrees) { + const resolved = path.resolve(row.worktree_path); + if (!isWithin(worktreesDirectory, resolved)) { + throw new Error("backup references a worktree outside Mill state"); + } + expectedWorktrees.add(resolved); + } } catch (error) { throw new MillError( "INVALID_STATE_BACKUP", @@ -1067,14 +1199,86 @@ export async function restoreStateBackup( } finally { candidate?.close(); } + const worktreesDirectory = path.join(directory, "worktrees"); + const entries = await readdir(worktreesDirectory, { withFileTypes: true }); + const orphaned = entries + .map((entry) => ({ + entry, + original: path.join(worktreesDirectory, entry.name), + })) + .filter(({ original }) => !expectedWorktrees.has(path.resolve(original))); + if (orphaned.length > 0) { + for (const { entry } of orphaned) { + if (!entry.isDirectory() || entry.isSymbolicLink()) { + throw new MillError( + "UNSAFE_ORPHANED_WORKTREE", + "Restore found an unclassified entry in the Mill worktree directory.", + ExitCode.configuration, + { name: entry.name }, + ); + } + } + const quarantineId = `restore-${new Date() + .toISOString() + .replaceAll(/[:.]/gu, "-")}-${randomUUID()}`; + const quarantineDirectory = path.join( + directory, + "quarantine", + quarantineId, + ); + await mkdir(quarantineDirectory, { recursive: true, mode: 0o700 }); + await chmod(quarantineDirectory, 0o700); + quarantineManifest = path.join(quarantineDirectory, "manifest.json"); + const planned = orphaned.map(({ entry, original }) => ({ + original, + quarantined: path.join(quarantineDirectory, entry.name), + })); + await writeFile( + quarantineManifest, + `${JSON.stringify( + { + schemaVersion: "1", + repositoryId, + backupPath: resolvedBackup, + protocol: "database_swap_commit_point", + worktrees: planned, + }, + null, + 2, + )}\n`, + { encoding: "utf8", mode: 0o600, flag: "wx" }, + ); + for (const item of planned) { + await rename(item.original, item.quarantined); + moved.push(item); + } + } await rm(`${databasePath}-wal`, { force: true }); await rm(`${databasePath}-shm`, { force: true }); await rename(temporaryPath, databasePath); + return { + quarantinedCount: moved.length, + ...(quarantineManifest === undefined ? {} : { quarantineManifest }), + }; + } catch (error) { + for (const item of moved.toReversed()) { + try { + await rename(item.quarantined, item.original); + } catch { + // The recovery manifest preserves attended recovery evidence. + } + } + throw error; } finally { await rm(temporaryPath, { force: true }); } } +export interface StateRestoreReport { + quarantinedCount: number; + quarantineManifest?: string; +} + export async function purgeRepositoryState( repositoryId: string, commonDirectory: string, diff --git a/src/runtime/verifier.ts b/src/runtime/verifier.ts index 2c9cdd7..d81a978 100644 --- a/src/runtime/verifier.ts +++ b/src/runtime/verifier.ts @@ -1,5 +1,6 @@ import { createHash, randomUUID } from "node:crypto"; -import { realpath, stat } from "node:fs/promises"; +import { chmod, mkdtemp, realpath, rm, stat, symlink } from "node:fs/promises"; +import { tmpdir } from "node:os"; import path from "node:path"; import { findTrustedExecutable } from "../doctor.js"; @@ -173,6 +174,53 @@ async function removeVerifierContainer( } } +async function verifierMountSource(root: string): Promise<{ + source: string; + dispose(): Promise; +}> { + if (!root.includes(",")) { + return { source: root, dispose: () => Promise.resolve() }; + } + const parent = await mkdtemp(path.join(tmpdir(), "mill-bind-")); + await chmod(parent, 0o700); + const source = path.join(parent, "workspace"); + try { + if (source.includes(",")) { + throw new MillError( + "VERIFIER_MOUNT_ALIAS_UNAVAILABLE", + "The trusted temporary directory cannot represent this OCI bind path safely.", + ExitCode.configuration, + ); + } + await symlink(root, source, "dir"); + if ((await realpath(source)) !== root) { + throw new MillError( + "VERIFIER_MOUNT_ALIAS_INVALID", + "The OCI bind alias does not resolve to the exact candidate workspace.", + ExitCode.configuration, + ); + } + return { + source, + async dispose(): Promise { + try { + await rm(parent, { recursive: true }); + } catch (error) { + throw new MillError( + "VERIFIER_MOUNT_ALIAS_CLEANUP_FAILED", + "Mill could not remove its protected OCI bind alias.", + ExitCode.io, + { cause: String(error) }, + ); + } + }, + }; + } catch (error) { + await rm(parent, { recursive: true, force: true }); + throw error; + } +} + export async function verifyDeclaredCommands(input: { root: string; candidateCommit: string; @@ -231,155 +279,160 @@ export async function verifyDeclaredCommands(input: { const uid = process.getuid?.() ?? 1000; const gid = process.getgid?.() ?? 1000; const canonicalRoot = await realpath(input.root); - for (let index = 0; index < input.task.commandIds.length; index += 1) { - const commandId = input.task.commandIds[index]; - if (commandId === undefined) continue; - const cancelled = - input.signal?.aborted === true || - input.cancellationRequested?.() === true; - if (cancelled || Date.now() >= input.deadlineMs) { - const reason = cancelled ? "CANCELLED" : "DEADLINE_EXCEEDED"; - evidence.push( - ...stoppedCommands( - input.config, - input.task.commandIds.slice(index), - reason, - ), + const mount = await verifierMountSource(canonicalRoot); + try { + for (let index = 0; index < input.task.commandIds.length; index += 1) { + const commandId = input.task.commandIds[index]; + if (commandId === undefined) continue; + const cancelled = + input.signal?.aborted === true || + input.cancellationRequested?.() === true; + if (cancelled || Date.now() >= input.deadlineMs) { + const reason = cancelled ? "CANCELLED" : "DEADLINE_EXCEEDED"; + evidence.push( + ...stoppedCommands( + input.config, + input.task.commandIds.slice(index), + reason, + ), + ); + break; + } + const command = input.config.commands[commandId]; + if (command === undefined) { + throw new MillError( + "UNKNOWN_COMMAND_ID", + `Task selects unknown command ID: ${commandId}`, + ExitCode.configuration, + ); + } + if (command.execution !== "oci") { + evidence.push({ + commandId, + required: command.required, + status: "blocked", + exitCode: null, + durationMs: 0, + outputDigest: digestOutput("", ""), + reason: "HOST_EXECUTION_NOT_QUALIFIED", + }); + continue; + } + const commandExecutable = command.argv[0]; + if (commandExecutable === undefined) { + throw new MillError( + "INVALID_COMMAND", + `Command ${commandId} has no executable.`, + ExitCode.configuration, + ); + } + const commandDirectory = await realpath( + path.resolve(canonicalRoot, command.cwd), ); - break; - } - const command = input.config.commands[commandId]; - if (command === undefined) { - throw new MillError( - "UNKNOWN_COMMAND_ID", - `Task selects unknown command ID: ${commandId}`, - ExitCode.configuration, + if ( + !isWithin(canonicalRoot, commandDirectory) || + !(await stat(commandDirectory)).isDirectory() + ) { + throw new MillError( + "INVALID_COMMAND_DIRECTORY", + `Command ${commandId} has an unsafe working directory.`, + ExitCode.configuration, + ); + } + const containerCwd = `/workspace/${path.relative(canonicalRoot, commandDirectory)}`; + const commandDeadline = Math.min( + input.deadlineMs, + Date.now() + command.timeoutSeconds * 1000, ); - } - if (command.execution !== "oci") { + const containerName = `mill-${randomUUID()}`; + let result: ProcessResult; + try { + result = await runProcess({ + executable: docker, + args: [ + "run", + "--name", + containerName, + "--label", + "dev.mill.owner=verifier", + "--network", + "none", + "--read-only", + "--cap-drop", + "ALL", + "--security-opt", + "no-new-privileges", + "--pids-limit", + "128", + "--memory", + "1g", + "--cpus", + "2", + "--tmpfs", + "/tmp:rw,noexec,nosuid,nodev,size=256m", + "--mount", + `type=bind,source=${mount.source},target=/workspace,readonly`, + "--workdir", + containerCwd, + "--user", + `${uid}:${gid}`, + "--env", + "HOME=/tmp", + "--entrypoint", + commandExecutable, + input.config.verifier.image, + ...command.argv.slice(1), + ], + cwd: input.root, + env: { + HOME: process.env.HOME, + PATH: "/usr/bin:/bin:/usr/sbin:/sbin:/opt/homebrew/bin", + LANG: "C", + LC_ALL: "C", + }, + deadlineMs: commandDeadline, + maxOutputBytes: input.maxOutputBytes, + ...(input.signal === undefined ? {} : { signal: input.signal }), + ...(input.onSpawn === undefined ? {} : { onSpawn: input.onSpawn }), + ...(input.onExit === undefined ? {} : { onExit: input.onExit }), + ...(input.cancellationRequested === undefined + ? {} + : { cancellationRequested: input.cancellationRequested }), + }); + } finally { + await removeVerifierContainer(docker, input.root, containerName); + } + const passed = + result.exitCode === 0 && + !result.timedOut && + !result.outputExceeded && + !result.cancelled; evidence.push({ commandId, required: command.required, - status: "blocked", - exitCode: null, - durationMs: 0, - outputDigest: digestOutput("", ""), - reason: "HOST_EXECUTION_NOT_QUALIFIED", - }); - continue; - } - const commandExecutable = command.argv[0]; - if (commandExecutable === undefined) { - throw new MillError( - "INVALID_COMMAND", - `Command ${commandId} has no executable.`, - ExitCode.configuration, - ); - } - const commandDirectory = await realpath( - path.resolve(canonicalRoot, command.cwd), - ); - if ( - !isWithin(canonicalRoot, commandDirectory) || - !(await stat(commandDirectory)).isDirectory() - ) { - throw new MillError( - "INVALID_COMMAND_DIRECTORY", - `Command ${commandId} has an unsafe working directory.`, - ExitCode.configuration, - ); - } - const containerCwd = `/workspace/${path.relative(canonicalRoot, commandDirectory)}`; - const commandDeadline = Math.min( - input.deadlineMs, - Date.now() + command.timeoutSeconds * 1000, - ); - const containerName = `mill-${randomUUID()}`; - let result: ProcessResult; - try { - result = await runProcess({ - executable: docker, - args: [ - "run", - "--name", - containerName, - "--label", - "dev.mill.owner=verifier", - "--network", - "none", - "--read-only", - "--cap-drop", - "ALL", - "--security-opt", - "no-new-privileges", - "--pids-limit", - "128", - "--memory", - "1g", - "--cpus", - "2", - "--tmpfs", - "/tmp:rw,noexec,nosuid,nodev,size=256m", - "--mount", - `type=bind,source=${canonicalRoot},target=/workspace,readonly`, - "--workdir", - containerCwd, - "--user", - `${uid}:${gid}`, - "--env", - "HOME=/tmp", - "--entrypoint", - commandExecutable, - input.config.verifier.image, - ...command.argv.slice(1), - ], - cwd: input.root, - env: { - HOME: process.env.HOME, - PATH: "/usr/bin:/bin:/usr/sbin:/sbin:/opt/homebrew/bin", - LANG: "C", - LC_ALL: "C", - }, - deadlineMs: commandDeadline, - maxOutputBytes: input.maxOutputBytes, - ...(input.signal === undefined ? {} : { signal: input.signal }), - ...(input.onSpawn === undefined ? {} : { onSpawn: input.onSpawn }), - ...(input.onExit === undefined ? {} : { onExit: input.onExit }), - ...(input.cancellationRequested === undefined + status: passed ? "passed" : "failed", + exitCode: result.exitCode, + durationMs: result.durationMs, + outputDigest: digestOutput(result.stdout, result.stderr), + ...(passed ? {} - : { cancellationRequested: input.cancellationRequested }), + : { + reason: result.cancelled + ? "CANCELLED" + : result.timedOut + ? "DEADLINE_EXCEEDED" + : result.outputExceeded + ? "OUTPUT_BUDGET_EXCEEDED" + : "NONZERO_EXIT", + }), }); - } finally { - await removeVerifierContainer(docker, input.root, containerName); } - const passed = - result.exitCode === 0 && - !result.timedOut && - !result.outputExceeded && - !result.cancelled; - evidence.push({ - commandId, - required: command.required, - status: passed ? "passed" : "failed", - exitCode: result.exitCode, - durationMs: result.durationMs, - outputDigest: digestOutput(result.stdout, result.stderr), - ...(passed - ? {} - : { - reason: result.cancelled - ? "CANCELLED" - : result.timedOut - ? "DEADLINE_EXCEEDED" - : result.outputExceeded - ? "OUTPUT_BUDGET_EXCEEDED" - : "NONZERO_EXIT", - }), + return validationEvidence({ + candidateCommit: input.candidateCommit, + verifierImage: input.config.verifier.image, + commands: evidence, }); + } finally { + await mount.dispose(); } - return validationEvidence({ - candidateCommit: input.candidateCommit, - verifierImage: input.config.verifier.image, - commands: evidence, - }); } diff --git a/test/cli-finalize.test.ts b/test/cli-finalize.test.ts new file mode 100644 index 0000000..c81d296 --- /dev/null +++ b/test/cli-finalize.test.ts @@ -0,0 +1,84 @@ +import { mkdir } from "node:fs/promises"; +import path from "node:path"; + +import { describe, expect, it, vi } from "vitest"; + +import type * as Delivery from "../src/runtime/delivery.js"; + +import { temporaryDirectory } from "./helpers.js"; + +const finalizeDraftPr = vi.hoisted(() => vi.fn()); + +vi.mock("../src/runtime/delivery.js", async (importOriginal) => ({ + ...(await importOriginal()), + finalizeDraftPr, +})); + +const { runCli } = await import("../src/cli-program.js"); + +function capture(): { + io: { + stdout: { write(value: string): void }; + stderr: { write(value: string): void }; + }; + stdout: string[]; + stderr: string[]; +} { + const stdout: string[] = []; + const stderr: string[] = []; + return { + io: { + stdout: { write: (value) => void stdout.push(value) }, + stderr: { write: (value) => void stderr.push(value) }, + }, + stdout, + stderr, + }; +} + +describe("finalization CLI result", () => { + it("never reports pending or failed post-merge evidence as success", async () => { + const temporary = await temporaryDirectory("mill-finalize-cli-"); + try { + await mkdir(path.join(temporary.path, ".git")); + for (const [status, blockCode, exitCode, reasonCode] of [ + ["merged", undefined, 75, "POST_MERGE_CHECKS_PENDING"], + ["blocked", "POST_MERGE_CHECKS_FAILED", 78, "POST_MERGE_CHECKS_FAILED"], + ] as const) { + finalizeDraftPr.mockResolvedValueOnce({ + run: { + status, + ...(blockCode === undefined ? {} : { blockCode }), + }, + delivery: {}, + }); + const output = capture(); + expect( + await runCli( + [ + "--json", + "--cwd", + temporary.path, + "pr", + "finalize", + "--task", + "product/tasks/task.yaml", + "--run", + "123e4567-e89b-42d3-a456-426614174000", + ], + output.io, + ), + ).toBe(exitCode); + expect(output.stderr).toEqual([]); + expect(JSON.parse(output.stdout.join(""))).toMatchObject({ + command: "pr.finalize", + ok: false, + status: "blocked", + reasons: [{ code: reasonCode }], + }); + } + } finally { + await temporary.cleanup(); + } + }); +}); diff --git a/test/runtime-boundaries.test.ts b/test/runtime-boundaries.test.ts index 6536dcc..3f1808d 100644 --- a/test/runtime-boundaries.test.ts +++ b/test/runtime-boundaries.test.ts @@ -516,6 +516,27 @@ describe("runtime authority and repository boundaries", () => { } }); + it("verifies repositories whose absolute path contains a comma", async () => { + const fixture = await runtimeFixture({ + repositoryPrefix: "mill-runtime,repo-", + }); + process.env.MILL_DOCKER_PATH = fixture.dockerPath; + try { + const inputs = await loadRuntimeInputs(fixture.root, fixture.taskPath); + const evidence = await verifyDeclaredCommands({ + root: fixture.root, + candidateCommit: "a".repeat(40), + config: inputs.config, + task: inputs.task, + deadlineMs: Date.now() + 30_000, + maxOutputBytes: 1024 * 1024, + }); + expect(evidence.passed).toBe(true); + } finally { + await fixture.cleanup(); + } + }); + it("fails closed when the OCI verifier contract or image is unavailable", async () => { const fixture = await runtimeFixture(); const tools = await temporaryDirectory("mill-verifier-unavailable-"); diff --git a/test/runtime-delivery.test.ts b/test/runtime-delivery.test.ts new file mode 100644 index 0000000..c2cc680 --- /dev/null +++ b/test/runtime-delivery.test.ts @@ -0,0 +1,1219 @@ +import { readFile, writeFile } from "node:fs/promises"; +import path from "node:path"; + +import { afterEach, describe, expect, it } from "vitest"; + +import { MillError, ExitCode } from "../src/errors.js"; +import { + finalizeDraftPr, + observeDraftPr, + openDraftPr, + planDraftPr, + reconcileDraftPr, +} from "../src/runtime/delivery.js"; +import type { + GitHubAdapter, + GitHubBinding, + GitHubCheck, + GitHubCommit, + GitHubFeedback, + GitHubObservation, + GitHubPullRequest, + GitHubReview, + ProposeConfig, +} from "../src/runtime/github.js"; +import { + cancelRun, + qualifyBaseline, + resumeRun, + reviewRun, + runStatus, + startLocalRun, + verifyRun, +} from "../src/runtime/lifecycle.js"; +import { runtimeFixture } from "./runtime-fixture.js"; + +const original = { + state: process.env.MILL_STATE_HOME, + codex: process.env.MILL_CODEX_PATH, + docker: process.env.MILL_DOCKER_PATH, +}; + +afterEach(() => { + if (original.state === undefined) delete process.env.MILL_STATE_HOME; + else process.env.MILL_STATE_HOME = original.state; + if (original.codex === undefined) delete process.env.MILL_CODEX_PATH; + else process.env.MILL_CODEX_PATH = original.codex; + if (original.docker === undefined) delete process.env.MILL_DOCKER_PATH; + else process.env.MILL_DOCKER_PATH = original.docker; +}); + +function activate(fixture: Awaited>): void { + process.env.MILL_STATE_HOME = fixture.stateHome; + process.env.MILL_CODEX_PATH = fixture.codexPath; + process.env.MILL_DOCKER_PATH = fixture.dockerPath; +} + +function completedCheck(conclusion: string, name = "validate"): GitHubCheck { + return { name, status: "completed", conclusion }; +} + +class FakeGitHub implements GitHubAdapter { + binding: GitHubBinding = { + actorLogin: "operator", + actorId: 7, + repositoryNodeId: "R_example", + fullName: "example/app", + cloneUrl: "https://github.com/example/app.git", + defaultBranch: "main", + fork: false, + }; + branchSha: string | null = null; + pullRequest: GitHubPullRequest | null = null; + checks: GitHubCheck[] = []; + mergeChecks: GitHubCheck[] = []; + reviews: GitHubReview[] = []; + feedback: GitHubFeedback[] = []; + mergeCommit: GitHubCommit | null = null; + mergeIsOnDefaultBranch = false; + defaultBranchHead = "d".repeat(40); + pushFailure: "before" | "after" | null = null; + prFailure: "before" | "after" | null = null; + pushCalls = 0; + createCalls = 0; + inspectCalls = 0; + onPush: (() => Promise) | undefined; + attemptedPullRequest: + | { config: ProposeConfig; branch: string; title: string; body: string } + | undefined; + + async inspect(): Promise { + await Promise.resolve(); + this.inspectCalls += 1; + return this.binding; + } + + async readBranch(): Promise { + await Promise.resolve(); + return this.branchSha; + } + + async pushExact(input: { + candidateCommit: string; + expectedOldCommit: string | null; + }): Promise { + await Promise.resolve(); + this.pushCalls += 1; + await this.onPush?.(); + if (input.expectedOldCommit !== this.branchSha) { + throw new MillError( + "FAKE_REMOTE_LEASE_MISMATCH", + "fake expected-old-head mismatch", + ExitCode.configuration, + ); + } + if (this.pushFailure === "before") { + this.pushFailure = null; + throw new MillError( + "FAKE_PUSH_INTERRUPTED", + "fake push interrupted before effect", + ExitCode.temporary, + ); + } + this.branchSha = input.candidateCommit; + if (this.pullRequest !== null) { + this.pullRequest = { + ...this.pullRequest, + headSha: input.candidateCommit, + }; + } + if (this.pushFailure === "after") { + this.pushFailure = null; + throw new MillError( + "FAKE_PUSH_RECEIPT_LOST", + "fake push effect completed before receipt loss", + ExitCode.temporary, + ); + } + } + + async findPullRequests(): Promise { + await Promise.resolve(); + return this.pullRequest === null ? [] : [this.pullRequest]; + } + + #materializePullRequest(): GitHubPullRequest { + const attempted = this.attemptedPullRequest; + if (attempted === undefined || this.branchSha === null) { + throw new Error("fake pull request has no attempted call or branch"); + } + const pullRequest: GitHubPullRequest = { + number: 41, + nodeId: "PR_example", + url: "https://github.com/example/app/pull/41", + state: "open", + draft: true, + body: attempted.body, + headRef: attempted.branch, + headSha: this.branchSha, + baseRef: attempted.config.baseBranch, + merged: false, + mergeCommitSha: null, + mergedByLogin: null, + mergedAt: null, + }; + this.pullRequest = pullRequest; + return pullRequest; + } + + async createDraftPullRequest(input: { + config: ProposeConfig; + branch: string; + title: string; + body: string; + }): Promise { + await Promise.resolve(); + this.createCalls += 1; + this.attemptedPullRequest = input; + if (this.prFailure === "before") { + this.prFailure = null; + throw new MillError( + "FAKE_PR_INTERRUPTED", + "fake pull request interrupted before effect", + ExitCode.temporary, + ); + } + const pullRequest = this.#materializePullRequest(); + if (this.prFailure === "after") { + this.prFailure = null; + throw new MillError( + "FAKE_PR_RECEIPT_LOST", + "fake pull request effect completed before receipt loss", + ExitCode.temporary, + ); + } + return pullRequest; + } + + async observe(): Promise { + await Promise.resolve(); + if (this.pullRequest === null) { + throw new MillError( + "FAKE_PR_MISSING", + "fake pull request is absent", + ExitCode.data, + ); + } + return { + pullRequest: this.pullRequest, + branchSha: this.branchSha, + checks: this.checks, + mergeChecks: this.mergeChecks, + reviews: this.reviews, + feedback: this.feedback, + defaultBranchHead: this.defaultBranchHead, + mergeCommit: this.mergeCommit, + mergeIsOnDefaultBranch: this.mergeIsOnDefaultBranch, + }; + } + + merge(candidateTree: string, parentCount = 1): void { + if (this.pullRequest === null) throw new Error("fake pull request missing"); + const mergeSha = "c".repeat(40); + this.pullRequest = { + ...this.pullRequest, + state: "closed", + draft: false, + merged: true, + mergeCommitSha: mergeSha, + mergedByLogin: "operator", + mergedAt: "2026-09-01T17:00:00.000Z", + }; + this.branchSha = null; + this.mergeCommit = { + sha: mergeSha, + tree: candidateTree, + parents: Array.from({ length: parentCount }, (_value, index) => + String(index + 1).repeat(40), + ), + }; + this.defaultBranchHead = mergeSha; + this.mergeIsOnDefaultBranch = true; + } +} + +async function reviewedFixture( + options: { + githubReviewer?: string; + } = {}, +): Promise<{ + fixture: Awaited>; + runId: string; + candidateCommit: string; + candidateTree: string; +}> { + const fixture = await runtimeFixture({ + propose: true, + ...(options.githubReviewer === undefined + ? {} + : { githubReviewer: options.githubReviewer }), + }); + activate(fixture); + const qualification = await qualifyBaseline({ + root: fixture.root, + taskPath: fixture.taskPath, + }); + if (qualification.approvalDigest === null) { + throw new Error("fake baseline qualification failed"); + } + const started = await startLocalRun({ + root: fixture.root, + taskPath: fixture.taskPath, + approvalDigest: qualification.approvalDigest, + }); + await verifyRun({ + root: fixture.root, + taskPath: fixture.taskPath, + runId: started.run.id, + }); + const reviewed = await reviewRun({ + root: fixture.root, + taskPath: fixture.taskPath, + runId: started.run.id, + }); + if ( + reviewed.run.candidateCommit === undefined || + reviewed.run.candidateTree === undefined + ) { + throw new Error("fake reviewed candidate identity missing"); + } + return { + fixture, + runId: reviewed.run.id, + candidateCommit: reviewed.run.candidateCommit, + candidateTree: reviewed.run.candidateTree, + }; +} + +async function planAndOpen(input: { + fixture: Awaited>; + runId: string; + adapter: FakeGitHub; +}): Promise>> { + const planned = await planDraftPr({ + root: input.fixture.root, + taskPath: input.fixture.taskPath, + runId: input.runId, + adapter: input.adapter, + }); + return openDraftPr({ + root: input.fixture.root, + taskPath: input.fixture.taskPath, + runId: input.runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter: input.adapter, + }); +} + +describe("exact-candidate GitHub draft delivery", () => { + it("requires an exact attended proposal and closes only after merge readback", async () => { + const { fixture, runId, candidateTree } = await reviewedFixture(); + const adapter = new FakeGitHub(); + try { + const planned = await planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(planned.delivery.proposalDigest).toMatch(/^sha256:[a-f0-9]{64}$/u); + expect(adapter).toMatchObject({ pushCalls: 0, createCalls: 0 }); + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: `sha256:${"0".repeat(64)}`, + attended: true, + adapter, + }), + ).rejects.toMatchObject({ code: "DELIVERY_APPROVAL_MISMATCH" }); + expect(adapter).toMatchObject({ pushCalls: 0, createCalls: 0 }); + const opened = await openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }); + expect(opened.run.status).toBe("awaiting_ci"); + expect(adapter).toMatchObject({ pushCalls: 1, createCalls: 1 }); + + const pending = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(pending.run.status).toBe("awaiting_ci"); + adapter.checks = [completedCheck("success")]; + const awaitingHuman = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(awaitingHuman.run.status).toBe("awaiting_human"); + await expect( + finalizeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "HUMAN_MERGE_PENDING" }); + + adapter.merge(candidateTree, 2); + adapter.mergeChecks = [completedCheck("success")]; + await expect( + finalizeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "MERGE_METHOD_NOT_ALLOWED" }); + adapter.merge(candidateTree); + if (adapter.pullRequest === null) throw new Error("fake PR missing"); + adapter.pullRequest = { + ...adapter.pullRequest, + mergedByLogin: "automation-bot", + }; + await expect( + finalizeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "MERGER_NOT_ALLOWED" }); + adapter.pullRequest = { + ...adapter.pullRequest, + mergedByLogin: "operator", + }; + const finalized = await finalizeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(finalized.run.status).toBe("closed"); + expect(finalized.delivery).toMatchObject({ + state: "closed", + merge: { + method: "linear_tree_preserving", + mergedByLogin: "operator", + tree: candidateTree, + }, + }); + } finally { + await fixture.cleanup(); + } + }); + + it("enforces attendance inside the exported mutation boundary", async () => { + const { fixture, runId } = await reviewedFixture(); + const adapter = new FakeGitHub(); + try { + const planned = await planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: false, + adapter, + }), + ).rejects.toMatchObject({ code: "ATTENDED_ACKNOWLEDGEMENT_REQUIRED" }); + expect(adapter).toMatchObject({ + inspectCalls: 1, + pushCalls: 0, + createCalls: 0, + }); + } finally { + await fixture.cleanup(); + } + }); + + it("makes cancellation durable before proposal planning", async () => { + const { fixture, runId } = await reviewedFixture(); + const adapter = new FakeGitHub(); + try { + const cancelled = await cancelRun({ root: fixture.root, runId }); + expect(cancelled).toMatchObject({ + status: "cancelled", + cancelRequested: true, + }); + await expect( + planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "RUN_NOT_REVIEWED" }); + expect(adapter.inspectCalls).toBe(0); + } finally { + await fixture.cleanup(); + } + }); + + it("honors durable cancellation before a subsequent remote effect", async () => { + const { fixture, runId } = await reviewedFixture(); + const adapter = new FakeGitHub(); + try { + const planned = await planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + adapter.onPush = async () => { + const cancellation = await cancelRun({ root: fixture.root, runId }); + expect(cancellation.cancelRequested).toBe(true); + }; + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }), + ).rejects.toMatchObject({ code: "OPERATOR_CANCELLED" }); + expect(adapter).toMatchObject({ + pushCalls: 1, + createCalls: 0, + pullRequest: null, + }); + } finally { + await fixture.cleanup(); + } + }); + + it("reconciles an unknown effect before making cancellation terminal", async () => { + const { fixture, runId, candidateCommit } = await reviewedFixture(); + const adapter = new FakeGitHub(); + adapter.pushFailure = "before"; + try { + const planned = await planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }), + ).rejects.toMatchObject({ code: "FAKE_PUSH_INTERRUPTED" }); + const pending = await cancelRun({ root: fixture.root, runId }); + expect(pending).toMatchObject({ + status: "effect_unknown", + cancelRequested: true, + }); + await expect( + resumeRun({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + }), + ).rejects.toMatchObject({ code: "GITHUB_RECONCILIATION_REQUIRED" }); + await expect( + runStatus({ root: fixture.root, runId }), + ).resolves.toMatchObject({ + run: { status: "effect_unknown", cancelRequested: true }, + reconciliationRequired: true, + }); + adapter.branchSha = candidateCommit; + await expect( + reconcileDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "OPERATOR_CANCELLED" }); + expect(adapter).toMatchObject({ createCalls: 0, pullRequest: null }); + } finally { + await fixture.cleanup(); + } + }); + + it("recovers effects completed before their receipts without duplication", async () => { + const { fixture, runId } = await reviewedFixture(); + const adapter = new FakeGitHub(); + adapter.pushFailure = "after"; + adapter.prFailure = "after"; + try { + const opened = await planAndOpen({ fixture, runId, adapter }); + expect(opened.run.status).toBe("awaiting_ci"); + expect(adapter).toMatchObject({ pushCalls: 1, createCalls: 1 }); + expect(opened.delivery.effects).toEqual( + expect.arrayContaining([ + expect.objectContaining({ kind: "push", status: "verified" }), + expect.objectContaining({ + kind: "pull_request", + status: "verified", + }), + ]), + ); + } finally { + await fixture.cleanup(); + } + }); + + it("authorizes one retry only after readback proves the effect absent", async () => { + const { fixture, runId } = await reviewedFixture(); + const adapter = new FakeGitHub(); + adapter.pushFailure = "before"; + try { + const planned = await planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }), + ).rejects.toMatchObject({ code: "FAKE_PUSH_INTERRUPTED" }); + expect(adapter).toMatchObject({ pushCalls: 1, createCalls: 0 }); + const absent = await reconcileDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(absent.run.status).toBe("proposing"); + expect(absent.delivery.effects).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + kind: "push", + status: "retryable_absent", + attemptCount: 1, + }), + ]), + ); + expect(adapter).toMatchObject({ pushCalls: 1, createCalls: 0 }); + const opened = await openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }); + expect(opened.run.status).toBe("awaiting_ci"); + expect(adapter).toMatchObject({ pushCalls: 2, createCalls: 1 }); + } finally { + await fixture.cleanup(); + } + }); + + it("retries one pull-request call only after readback proves absence", async () => { + const { fixture, runId } = await reviewedFixture(); + const adapter = new FakeGitHub(); + adapter.prFailure = "before"; + try { + const planned = await planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }), + ).rejects.toMatchObject({ code: "FAKE_PR_INTERRUPTED" }); + expect(adapter).toMatchObject({ pushCalls: 1, createCalls: 1 }); + const absent = await reconcileDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(absent.run.status).toBe("proposing"); + const opened = await openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }); + expect(opened.run.status).toBe("awaiting_ci"); + expect(adapter).toMatchObject({ pushCalls: 1, createCalls: 2 }); + } finally { + await fixture.cleanup(); + } + }); + + it("blocks after the single readback-authorized remote retry is exhausted", async () => { + const { fixture, runId } = await reviewedFixture(); + const adapter = new FakeGitHub(); + adapter.pushFailure = "before"; + try { + const planned = await planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }), + ).rejects.toMatchObject({ code: "FAKE_PUSH_INTERRUPTED" }); + await reconcileDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + adapter.pushFailure = "before"; + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }), + ).rejects.toMatchObject({ code: "FAKE_PUSH_INTERRUPTED" }); + const exhausted = await reconcileDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(exhausted.run).toMatchObject({ + status: "blocked", + blockCode: "REMOTE_EFFECT_RETRY_EXHAUSTED", + }); + expect(exhausted.delivery).toMatchObject({ + state: "blocked", + lastErrorCode: "REMOTE_EFFECT_RETRY_EXHAUSTED", + }); + expect(adapter).toMatchObject({ pushCalls: 2, createCalls: 0 }); + } finally { + await fixture.cleanup(); + } + }); + + it("fails closed for identity drift and every non-success required check", async () => { + const { fixture, runId } = await reviewedFixture(); + const changedActor = new FakeGitHub(); + changedActor.binding = { ...changedActor.binding, actorLogin: "intruder" }; + try { + await expect( + planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter: changedActor, + }), + ).rejects.toMatchObject({ code: "GITHUB_BINDING_MISMATCH" }); + expect(changedActor).toMatchObject({ pushCalls: 0, createCalls: 0 }); + + const fork = new FakeGitHub(); + fork.binding = { ...fork.binding, fork: true }; + await expect( + planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter: fork, + }), + ).rejects.toMatchObject({ code: "GITHUB_BINDING_MISMATCH" }); + expect(fork).toMatchObject({ pushCalls: 0, createCalls: 0 }); + + const adapter = new FakeGitHub(); + await planAndOpen({ fixture, runId, adapter }); + for (const conclusion of [ + "failure", + "cancelled", + "neutral", + "skipped", + "timed_out", + ]) { + adapter.checks = [completedCheck(conclusion)]; + const observed = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(observed.run).toMatchObject({ + status: "blocked", + blockCode: "REMOTE_CHECKS_FAILED", + }); + } + adapter.checks = [completedCheck("success"), completedCheck("neutral")]; + const conflict = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(conflict.run.status).toBe("blocked"); + adapter.checks = [completedCheck("success")]; + const passed = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(passed.run.status).toBe("awaiting_human"); + + const configPath = path.join(fixture.root, "mill.yaml"); + await writeFile( + configPath, + (await readFile(configPath, "utf8")).replace( + "allowedMergeMethods: [linear_tree_preserving]", + "allowedMergeMethods: [merge]", + ), + ); + await expect( + observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "DELIVERY_AUTHORITY_DRIFT" }); + } finally { + await fixture.cleanup(); + } + }); + + it("replaces stale blocker identity when remote observations change", async () => { + const { fixture, runId, candidateCommit } = await reviewedFixture({ + githubReviewer: "codex-review", + }); + const adapter = new FakeGitHub(); + try { + await planAndOpen({ fixture, runId, adapter }); + adapter.checks = [completedCheck("failure")]; + const checksBlocked = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(checksBlocked.run.blockCode).toBe("REMOTE_CHECKS_FAILED"); + + adapter.checks = [completedCheck("success")]; + adapter.feedback = [ + { + id: "blocker-change", + actorLogin: "codex-review", + priority: "P1", + body: "[P1] Current-head repair required", + path: "src/value.js", + line: 1, + url: "https://github.com/example/app/pull/41#discussion_blocker-change", + commitId: candidateCommit, + }, + ]; + const reviewBlocked = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(reviewBlocked.run.blockCode).toBe("REMOTE_REVIEW_FINDINGS"); + + adapter.feedback = []; + adapter.checks = [completedCheck("failure")]; + const checksBlockedAgain = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(checksBlockedAgain.run.blockCode).toBe("REMOTE_CHECKS_FAILED"); + await expect( + resumeRun({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + }), + ).rejects.toMatchObject({ code: "RUN_REQUIRES_HUMAN_DISPOSITION" }); + } finally { + await fixture.cleanup(); + } + }); + + it("repairs one aggregated remote review and updates the same pull request", async () => { + const { fixture, runId, candidateCommit } = await reviewedFixture({ + githubReviewer: "codex-review", + }); + const adapter = new FakeGitHub(); + try { + await planAndOpen({ fixture, runId, adapter }); + adapter.checks = [completedCheck("success")]; + adapter.feedback = [ + { + id: "12", + actorLogin: "codex-review", + priority: "P1", + body: "[P1] Use the repaired value", + path: "src/value.js", + line: 1, + url: "https://github.com/example/app/pull/41#discussion_r12", + commitId: candidateCommit, + }, + { + id: "review-13", + actorLogin: "codex-review", + priority: "P2", + body: "[P2] Address the top-level review finding", + path: null, + line: null, + url: "https://github.com/example/app/pull/41#pullrequestreview-13", + commitId: candidateCommit, + }, + ]; + const blocked = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(blocked.run).toMatchObject({ + status: "blocked", + blockCode: "REMOTE_REVIEW_FINDINGS", + }); + + const repaired = await resumeRun({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + }); + expect(repaired.status).toBe("committed"); + expect(repaired.candidateCommit).not.toBe(candidateCommit); + await verifyRun({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + }); + const reviewed = await reviewRun({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + }); + if (reviewed.run.candidateCommit === undefined) { + throw new Error("repaired candidate missing"); + } + adapter.feedback = []; + adapter.reviews = [ + { + id: "21", + actorLogin: "codex-review", + state: "COMMENTED", + commitId: reviewed.run.candidateCommit, + body: "", + url: "https://github.com/example/app/pull/41#pullrequestreview-21", + }, + { + id: "22", + actorLogin: "codex-review", + state: "CHANGES_REQUESTED", + commitId: reviewed.run.candidateCommit, + body: "", + url: "https://github.com/example/app/pull/41#pullrequestreview-22", + }, + ]; + const planned = await planDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + if (adapter.pullRequest === null) throw new Error("fake PR missing"); + const priorPullRequest = adapter.pullRequest; + adapter.branchSha = reviewed.run.candidateCommit; + adapter.pullRequest = { + ...priorPullRequest, + number: 42, + nodeId: "PR_replacement", + headSha: reviewed.run.candidateCommit, + }; + const pushCallsBeforeResume = adapter.pushCalls; + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }), + ).rejects.toMatchObject({ code: "PULL_REQUEST_IDENTITY_MISMATCH" }); + expect(adapter.pushCalls).toBe(pushCallsBeforeResume); + adapter.branchSha = candidateCommit; + adapter.pullRequest = priorPullRequest; + adapter.pushFailure = "before"; + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }), + ).rejects.toMatchObject({ code: "FAKE_PUSH_INTERRUPTED" }); + expect(adapter.pullRequest.headSha).toBe(candidateCommit); + for (const conflictingPullRequest of [ + { ...priorPullRequest, draft: false }, + { ...priorPullRequest, state: "closed" as const, draft: false }, + { + ...priorPullRequest, + state: "closed" as const, + draft: false, + merged: true, + mergeCommitSha: "c".repeat(40), + mergedByLogin: "operator", + mergedAt: "2026-09-01T17:00:00.000Z", + }, + ]) { + adapter.pullRequest = conflictingPullRequest; + await expect( + reconcileDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "PULL_REQUEST_IDENTITY_MISMATCH" }); + await expect( + runStatus({ root: fixture.root, runId }), + ).resolves.toMatchObject({ + run: { status: "effect_unknown" }, + reconciliationRequired: true, + }); + } + adapter.branchSha = reviewed.run.candidateCommit; + adapter.pullRequest = { + ...priorPullRequest, + number: 42, + nodeId: "PR_replacement", + headSha: reviewed.run.candidateCommit, + }; + await expect( + reconcileDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "PULL_REQUEST_IDENTITY_MISMATCH" }); + adapter.branchSha = candidateCommit; + adapter.pullRequest = priorPullRequest; + const reconciled = await reconcileDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(reconciled.run.status).toBe("proposing"); + expect(reconciled.delivery.effects).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + kind: "push", + status: "retryable_absent", + attemptCount: 1, + }), + ]), + ); + adapter.pullRequest = { ...priorPullRequest, draft: false }; + await expect( + openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }), + ).rejects.toMatchObject({ code: "PULL_REQUEST_IDENTITY_MISMATCH" }); + expect(adapter).toMatchObject({ + branchSha: candidateCommit, + pushCalls: 2, + }); + adapter.pullRequest = priorPullRequest; + const reopened = await openDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + approvalDigest: planned.delivery.proposalDigest, + attended: true, + adapter, + }); + expect(reopened.delivery.pullRequest?.number).toBe(41); + expect(adapter).toMatchObject({ pushCalls: 3, createCalls: 1 }); + const changesRequested = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(changesRequested.run.status).toBe("awaiting_ci"); + adapter.reviews.push({ + id: "23", + actorLogin: "codex-review", + state: "COMMENTED", + commitId: reviewed.run.candidateCommit, + body: "", + url: "https://github.com/example/app/pull/41#pullrequestreview-23", + }); + const ready = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(ready.run.status).toBe("awaiting_human"); + } finally { + await fixture.cleanup(); + } + }); + + it("fails closed on PR drift and post-merge evidence until every identity settles", async () => { + const { fixture, runId, candidateCommit, candidateTree } = + await reviewedFixture(); + const adapter = new FakeGitHub(); + try { + await planAndOpen({ fixture, runId, adapter }); + if (adapter.pullRequest === null) throw new Error("fake PR missing"); + adapter.pullRequest = { + ...adapter.pullRequest, + headSha: "f".repeat(40), + }; + await expect( + observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "REMOTE_IDENTITY_DRIFT" }); + adapter.pullRequest = { + ...adapter.pullRequest, + headSha: candidateCommit, + }; + adapter.checks = [ + { name: "validate", status: "in_progress", conclusion: null }, + ]; + const inProgress = await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(inProgress.run.status).toBe("awaiting_ci"); + adapter.checks = [completedCheck("success")]; + await observeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + adapter.pullRequest = { + ...adapter.pullRequest, + state: "closed", + }; + await expect( + finalizeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "PULL_REQUEST_CLOSED_UNMERGED" }); + + adapter.merge("e".repeat(40)); + await expect( + finalizeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }), + ).rejects.toMatchObject({ code: "MERGE_TREE_REVALIDATION_REQUIRED" }); + adapter.merge(candidateTree); + const pending = await finalizeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(pending.run.status).toBe("merged"); + adapter.mergeChecks = [completedCheck("failure")]; + const failed = await finalizeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(failed.run).toMatchObject({ + status: "blocked", + blockCode: "POST_MERGE_CHECKS_FAILED", + }); + adapter.mergeChecks = [completedCheck("success")]; + const closed = await finalizeDraftPr({ + root: fixture.root, + taskPath: fixture.taskPath, + runId, + adapter, + }); + expect(closed.run.status).toBe("closed"); + } finally { + await fixture.cleanup(); + } + }); +}); diff --git a/test/runtime-fixture.ts b/test/runtime-fixture.ts index 4eb2a5c..94ce697 100644 --- a/test/runtime-fixture.ts +++ b/test/runtime-fixture.ts @@ -24,7 +24,13 @@ async function git(root: string, args: readonly string[]): Promise { } export async function runtimeFixture( - options: { reviewRepair?: boolean; retryCount?: number } = {}, + options: { + reviewRepair?: boolean; + retryCount?: number; + repositoryPrefix?: string; + propose?: boolean; + githubReviewer?: string; + } = {}, ): Promise<{ root: string; stateHome: string; @@ -34,7 +40,9 @@ export async function runtimeFixture( dockerPath: string; cleanup(): Promise; }> { - const repository = await temporaryDirectory("mill-runtime-repo-"); + const repository = await temporaryDirectory( + options.repositoryPrefix ?? "mill-runtime-repo-", + ); const state = await temporaryDirectory("mill-runtime-state-"); const tools = await temporaryDirectory("mill-runtime-tools-"); const root = repository.path; @@ -58,17 +66,43 @@ export async function runtimeFixture( 'import assert from "node:assert/strict";\nimport test from "node:test";\nimport { value } from "../src/value.js";\ntest("value stays positive", () => assert.ok(value > 0));\n', ), ]); + const reviewMode = + options.githubReviewer === undefined ? "local_only" : "github_required"; + const reviewers = + options.githubReviewer === undefined ? "[]" : `[${options.githubReviewer}]`; + const proposalConfiguration = + options.propose === true + ? `propose: + forge: github + host: github.com + owner: example + repository: app + repositoryNodeId: R_example + remoteName: origin + baseBranch: main + branchPrefix: mill/ + allowedActors: [operator] + allowedMergerLogins: [operator] + requiredChecks: [validate] + reviewPolicy: + mode: ${reviewMode} + requiredReviewerLogins: ${reviewers} + allowedMergeMethods: [linear_tree_preserving] + approvalTtlSeconds: 900 + pollTimeoutSeconds: 30 +` + : ""; await writeFile( path.join(root, "mill.yaml"), `schemaVersion: "1" repositoryId: "11111111-1111-4111-8111-111111111111" -trustCeiling: build +trustCeiling: ${options.propose === true ? "propose" : "build"} sensitivePaths: - .env verifier: image: "node@sha256:ba849c60be29959425b8734d57b8b4b7d56f98edd9504c9af091d5281095a71e" network: none -commands: +${proposalConfiguration}commands: test: argv: ["node", "--test"] cwd: "." @@ -120,6 +154,12 @@ budget: `, ); await git(root, ["init", "--initial-branch=main"]); + await git(root, [ + "remote", + "add", + "origin", + "https://github.com/example/app.git", + ]); await git(root, ["add", "."]); await git(root, [ "commit", diff --git a/test/runtime-github.test.ts b/test/runtime-github.test.ts new file mode 100644 index 0000000..c7611e0 --- /dev/null +++ b/test/runtime-github.test.ts @@ -0,0 +1,400 @@ +import { chmod, readFile, writeFile } from "node:fs/promises"; +import path from "node:path"; + +import { afterEach, describe, expect, it } from "vitest"; + +import { + createGitHubAdapter, + type ProposeConfig, +} from "../src/runtime/github.js"; +import { temporaryDirectory } from "./helpers.js"; + +const originalGh = process.env.MILL_GH_PATH; +const originalGit = process.env.MILL_GIT_PATH; + +afterEach(() => { + if (originalGh === undefined) delete process.env.MILL_GH_PATH; + else process.env.MILL_GH_PATH = originalGh; + if (originalGit === undefined) delete process.env.MILL_GIT_PATH; + else process.env.MILL_GIT_PATH = originalGit; +}); + +const sha = "a".repeat(40); +const config: ProposeConfig = { + forge: "github", + host: "github.com", + owner: "example", + repository: "app", + repositoryNodeId: "R_example", + remoteName: "origin", + baseBranch: "main", + branchPrefix: "mill/", + allowedActors: ["operator"], + allowedMergerLogins: ["operator"], + requiredChecks: ["validate"], + reviewPolicy: { + mode: "github_required", + requiredReviewerLogins: ["codex-review"], + }, + allowedMergeMethods: ["linear_tree_preserving"], + approvalTtlSeconds: 900, + pollTimeoutSeconds: 30, +}; + +describe("GitHub CLI adapter", () => { + it("uses bounded pagination and parses exact-head provider evidence", async () => { + const repository = await temporaryDirectory("mill-github-repository-"); + const tools = await temporaryDirectory("mill-github-tools-"); + const gh = path.join(tools.path, "gh"); + try { + await writeFile( + gh, + `#!${process.execPath} +import {appendFileSync} from "node:fs"; +appendFileSync(new URL("./calls.log",import.meta.url),JSON.stringify(process.argv.slice(2))+"\\n"); +const args=process.argv.slice(2);const endpoint=args.find((value)=>value.startsWith("repos/"))??args.at(-1)??""; +const pull={number:41,node_id:"PR_example",html_url:"https://github.com/example/app/pull/41",state:"open",draft:true,body:"",head:{ref:"mill/task",sha:"${sha}"},base:{ref:"main"},merged:false,merge_commit_sha:null,merged_by:null,merged_at:null}; +const listedPull={...pull};delete listedPull.merged;delete listedPull.merged_by;delete listedPull.merged_at; +if(endpoint==="user")console.log(JSON.stringify({login:"operator",id:7})); +else if(endpoint==="repos/example/app")console.log(JSON.stringify({node_id:"R_example",full_name:"example/app",clone_url:"https://github.com/example/app.git",default_branch:"main",fork:false})); +else if(endpoint.includes("/git/ref/heads/missing")){console.error("HTTP 404");process.exit(1)} +else if(endpoint.includes("/git/ref/heads/"))console.log(JSON.stringify({object:{sha:"${sha}"}})); +else if(endpoint.includes("/pulls?"))console.log(JSON.stringify([[listedPull]])); +else if(args.includes("--method")&&endpoint==="repos/example/app/pulls")console.log(JSON.stringify(pull)); +else if(endpoint.endsWith("/pulls/41"))console.log(JSON.stringify(pull)); +else if(endpoint.includes("/check-runs"))console.log(JSON.stringify([{check_runs:[{name:"validate",status:"completed",conclusion:"success"}]}])); +else if(endpoint.includes("/status?"))console.log(JSON.stringify([{statuses:[{state:"pending",context:"legacy"}]}])) +else if(endpoint.includes("/reviews?"))console.log(JSON.stringify([[{id:11,user:{login:"codex-review"},state:"COMMENTED",commit_id:"${sha}",body:"[P1] top-level finding",html_url:"https://github.com/example/app/pull/41#pullrequestreview-11"}]])); +else if(endpoint.includes("/comments?"))console.log(JSON.stringify([[{id:12,user:{login:"codex-review"},body:"[P2] clarify edge case",path:"src/index.ts",line:4,html_url:"https://github.com/example/app/pull/41#discussion_r12",commit_id:"${sha}"}]])); +else process.exit(2); +`, + { mode: 0o755 }, + ); + await chmod(gh, 0o755); + process.env.MILL_GH_PATH = gh; + const adapter = createGitHubAdapter(repository.path); + await expect( + adapter.inspect({ config, deadlineMs: Date.now() + 10_000 }), + ).resolves.toMatchObject({ + actorLogin: "operator", + repositoryNodeId: "R_example", + fullName: "example/app", + }); + await expect( + adapter.findPullRequests({ + config, + branch: "mill/task", + deadlineMs: Date.now() + 10_000, + }), + ).resolves.toHaveLength(1); + await expect( + adapter.readBranch({ + config, + branch: "missing", + deadlineMs: Date.now() + 10_000, + }), + ).resolves.toBeNull(); + await expect( + adapter.createDraftPullRequest({ + config, + branch: "mill/task", + title: "@/definitely-not-a-provider-input-file", + body: "body", + deadlineMs: Date.now() + 10_000, + }), + ).resolves.toMatchObject({ number: 41, draft: true }); + const observation = await adapter.observe({ + config, + pullRequestNumber: 41, + deadlineMs: Date.now() + 10_000, + }); + expect(observation).toMatchObject({ + branchSha: sha, + checks: [ + { name: "validate", status: "completed", conclusion: "success" }, + { name: "legacy", status: "in_progress", conclusion: "pending" }, + ], + reviews: [ + { + id: "11", + actorLogin: "codex-review", + state: "COMMENTED", + commitId: sha, + body: "[P1] top-level finding", + }, + ], + feedback: [ + { priority: "P1", commitId: sha, path: null }, + { priority: "P2", commitId: sha, path: "src/index.ts" }, + ], + }); + const calls = await readFile(path.join(tools.path, "calls.log"), "utf8"); + const paginated = calls + .trim() + .split("\n") + .map((line) => JSON.parse(line) as string[]) + .filter((args) => args.includes("--paginate")); + expect(paginated).not.toHaveLength(0); + expect(paginated.every((args) => args.includes("--slurp"))).toBe(true); + const createCall = calls + .trim() + .split("\n") + .map((line) => JSON.parse(line) as string[]) + .find( + (args) => + args.includes("--method") && + args.includes("repos/example/app/pulls"), + ); + expect(createCall).toBeDefined(); + expect( + createCall?.filter((value) => value === "--raw-field"), + ).toHaveLength(4); + expect(createCall).toEqual( + expect.arrayContaining([ + "title=@/definitely-not-a-provider-input-file", + "head=mill/task", + "base=main", + "body=body", + "--field", + "draft=true", + ]), + ); + + const git = path.join(tools.path, "git"); + await writeFile( + git, + `#!${process.execPath}\nimport {writeFileSync} from "node:fs";writeFileSync(new URL("./git-call.json",import.meta.url),JSON.stringify(process.argv.slice(2)));process.exit(0);\n`, + { mode: 0o755 }, + ); + await chmod(git, 0o755); + process.env.MILL_GIT_PATH = git; + await expect( + adapter.pushExact({ + root: repository.path, + config, + cloneUrl: "https://github.com/example/app.git", + branch: "mill/task", + candidateCommit: sha, + expectedOldCommit: null, + deadlineMs: Date.now() + 10_000, + }), + ).resolves.toBeUndefined(); + const gitCall = JSON.parse( + await readFile(path.join(tools.path, "git-call.json"), "utf8"), + ) as string[]; + expect(gitCall).toContain("--force-with-lease=refs/heads/mill/task:"); + await expect( + adapter.pushExact({ + root: repository.path, + config, + cloneUrl: "https://github.com/example/app.git", + branch: "feature/task", + candidateCommit: sha, + expectedOldCommit: null, + deadlineMs: Date.now() + 10_000, + }), + ).rejects.toMatchObject({ code: "INVALID_DELIVERY_BRANCH" }); + await expect( + adapter.pushExact({ + root: repository.path, + config, + cloneUrl: "https://github.com/lookalike/app.git", + branch: "mill/task", + candidateCommit: sha, + expectedOldCommit: null, + deadlineMs: Date.now() + 10_000, + }), + ).rejects.toMatchObject({ + code: "GITHUB_REPOSITORY_BINDING_MISMATCH", + }); + process.env.MILL_GIT_PATH = path.join(tools.path, "missing-git"); + await expect( + adapter.pushExact({ + root: repository.path, + config, + cloneUrl: "https://github.com/example/app.git", + branch: "mill/task", + candidateCommit: sha, + expectedOldCommit: null, + deadlineMs: Date.now() + 10_000, + }), + ).rejects.toMatchObject({ code: "SHIPPER_TOOL_UNAVAILABLE" }); + await writeFile( + git, + `#!${process.execPath}\nsetInterval(()=>{},1000);\n`, + { mode: 0o755 }, + ); + await chmod(git, 0o755); + process.env.MILL_GIT_PATH = git; + let cancellationPolls = 0; + await expect( + adapter.pushExact({ + root: repository.path, + config, + cloneUrl: "https://github.com/example/app.git", + branch: "mill/task", + candidateCommit: sha, + expectedOldCommit: null, + deadlineMs: Date.now() + 10_000, + cancellationRequested: () => ++cancellationPolls >= 2, + }), + ).rejects.toMatchObject({ code: "GITHUB_PUSH_OUTCOME_UNKNOWN" }); + expect(cancellationPolls).toBeGreaterThanOrEqual(2); + await writeFile(git, `#!${process.execPath}\nprocess.exit(9);\n`, { + mode: 0o755, + }); + await chmod(git, 0o755); + process.env.MILL_GIT_PATH = git; + await expect( + adapter.pushExact({ + root: repository.path, + config, + cloneUrl: "https://github.com/example/app.git", + branch: "mill/task", + candidateCommit: sha, + expectedOldCommit: null, + deadlineMs: Date.now() + 10_000, + }), + ).rejects.toMatchObject({ code: "GITHUB_PUSH_OUTCOME_UNKNOWN" }); + } finally { + await Promise.all([repository.cleanup(), tools.cleanup()]); + } + }); + + it("reads exact merge identity, containment, and post-merge checks", async () => { + const repository = await temporaryDirectory("mill-github-merged-"); + const tools = await temporaryDirectory("mill-github-merged-tools-"); + const gh = path.join(tools.path, "gh"); + const mergeSha = "c".repeat(40); + const tree = "b".repeat(40); + try { + await writeFile( + gh, + `#!${process.execPath} +const args=process.argv.slice(2);const endpoint=args.find((value)=>value.startsWith("repos/"))??args.at(-1)??""; +const pull={number:41,node_id:"PR_example",html_url:"https://github.com/example/app/pull/41",state:"closed",draft:false,body:"marker",head:{ref:"mill/task",sha:"${sha}"},base:{ref:"main"},merged:true,merge_commit_sha:"${mergeSha}",merged_by:{login:"operator"},merged_at:"2026-09-01T17:00:00.000Z"}; +if(endpoint.endsWith("/pulls/41"))console.log(JSON.stringify(pull)); +else if(endpoint.includes("/git/ref/heads/mill")){console.error("HTTP 404");process.exit(1)} +else if(endpoint.includes("/git/ref/heads/main"))console.log(JSON.stringify({object:{sha:"${mergeSha}"}})); +else if(endpoint.includes("/check-runs"))console.log(JSON.stringify([{check_runs:[{name:"validate",status:"completed",conclusion:"success"}]}])); +else if(endpoint.includes("/status?"))console.log(JSON.stringify([{statuses:[]}])) +else if(endpoint.includes("/reviews?"))console.log(JSON.stringify([[]])); +else if(endpoint.includes("/comments?"))console.log(JSON.stringify([[]])); +else if(endpoint.includes("/git/commits/"))console.log(JSON.stringify({sha:"${mergeSha}",tree:{sha:"${tree}"},parents:[{sha:"${"d".repeat(40)}"}]})); +else if(endpoint.includes("/compare/"))console.log(JSON.stringify({status:"identical"})); +else process.exit(2); +`, + { mode: 0o755 }, + ); + await chmod(gh, 0o755); + process.env.MILL_GH_PATH = gh; + const observation = await createGitHubAdapter(repository.path).observe({ + config, + pullRequestNumber: 41, + deadlineMs: Date.now() + 10_000, + }); + expect(observation).toMatchObject({ + branchSha: null, + defaultBranchHead: mergeSha, + pullRequest: { mergedByLogin: "operator" }, + mergeCommit: { sha: mergeSha, tree, parents: ["d".repeat(40)] }, + mergeIsOnDefaultBranch: true, + mergeChecks: [{ name: "validate", conclusion: "success" }], + }); + } finally { + await Promise.all([repository.cleanup(), tools.cleanup()]); + } + }); + + it("fails closed on malformed, incomplete, and failed provider responses", async () => { + const repository = await temporaryDirectory("mill-github-invalid-"); + const tools = await temporaryDirectory("mill-github-invalid-tools-"); + const gh = path.join(tools.path, "gh"); + const mode = path.join(tools.path, "mode"); + try { + await writeFile( + gh, + `#!${process.execPath} +import {readFileSync} from "node:fs"; +const mode=readFileSync(new URL("./mode",import.meta.url),"utf8").trim(); +if(mode==="call-failed")process.exit(3);if(mode==="invalid-json"){console.log("{");process.exit(0)} +const args=process.argv.slice(2);const endpoint=args.find((value)=>value.startsWith("repos/"))??args.at(-1)??""; +const pull={number:41,node_id:"PR_example",html_url:"https://github.com/example/app/pull/41",state:mode==="bad-pull-state"?"unexpected":"open",draft:mode==="bad-draft"?"true":true,body:"marker",head:{ref:"mill/task",sha:"${sha}"},base:{ref:"main"},merged:false,merge_commit_sha:null,merged_by:null,merged_at:null}; +const listedPull={...pull};if(mode==="bad-merged")listedPull.merged="false";else delete listedPull.merged;delete listedPull.merged_by;delete listedPull.merged_at; +if(endpoint==="user")console.log(JSON.stringify({login:mode==="bad-login"?"":"operator",id:mode==="bad-actor"?0:7})); +else if(endpoint==="repos/example/app")console.log(JSON.stringify(mode==="bad-repo"?[]:{node_id:"R_example",full_name:"example/app",clone_url:"https://github.com/example/app.git",default_branch:"main",fork:mode==="bad-fork"?"false":false})); +else if(endpoint.includes("/git/ref/heads/"))console.log(JSON.stringify({object:{sha:mode==="bad-sha"?"bad":"${sha}"}})); +else if(endpoint.includes("/pulls?"))console.log(JSON.stringify(mode==="bad-pages"?{}:mode==="bad-page"?[{}]:[[listedPull]])); +else if(endpoint.endsWith("/pulls/41"))console.log(JSON.stringify(pull)); +else if(endpoint.includes("/check-runs"))console.log(JSON.stringify(mode==="bad-check-pages"?{}:mode==="bad-check-page"?[{}]:[{check_runs:[]}])) +else if(endpoint.includes("/status?"))console.log(JSON.stringify([{statuses:[]}])) +else if(endpoint.includes("/reviews?"))console.log(JSON.stringify([[]])); +else if(endpoint.includes("/comments?"))console.log(JSON.stringify([[]])); +else process.exit(2); +`, + { mode: 0o755 }, + ); + await chmod(gh, 0o755); + process.env.MILL_GH_PATH = gh; + const adapter = createGitHubAdapter(repository.path); + const inspect = () => + adapter.inspect({ config, deadlineMs: Date.now() + 10_000 }); + for (const invalidMode of [ + "bad-actor", + "bad-login", + "bad-repo", + "bad-fork", + ]) { + await writeFile(mode, invalidMode); + await expect(inspect()).rejects.toMatchObject({ + code: "INVALID_GITHUB_RESPONSE", + }); + } + await writeFile(mode, "bad-sha"); + await expect( + adapter.readBranch({ + config, + branch: "mill/task", + deadlineMs: Date.now() + 10_000, + }), + ).rejects.toMatchObject({ code: "INVALID_GITHUB_RESPONSE" }); + for (const invalidMode of [ + "bad-pull-state", + "bad-draft", + "bad-merged", + "bad-pages", + "bad-page", + ]) { + await writeFile(mode, invalidMode); + await expect( + adapter.findPullRequests({ + config, + branch: "mill/task", + deadlineMs: Date.now() + 10_000, + }), + ).rejects.toMatchObject({ code: "INVALID_GITHUB_RESPONSE" }); + } + for (const invalidMode of ["bad-check-pages", "bad-check-page"]) { + await writeFile(mode, invalidMode); + await expect( + adapter.observe({ + config, + pullRequestNumber: 41, + deadlineMs: Date.now() + 10_000, + }), + ).rejects.toMatchObject({ code: "INVALID_GITHUB_RESPONSE" }); + } + for (const invalidMode of ["call-failed", "invalid-json"]) { + await writeFile(mode, invalidMode); + await expect(inspect()).rejects.toMatchObject({ + code: + invalidMode === "call-failed" + ? "GITHUB_CALL_FAILED" + : "INVALID_GITHUB_RESPONSE", + }); + } + } finally { + await Promise.all([repository.cleanup(), tools.cleanup()]); + } + }); +}); diff --git a/test/runtime-state.test.ts b/test/runtime-state.test.ts index 529694f..06d4538 100644 --- a/test/runtime-state.test.ts +++ b/test/runtime-state.test.ts @@ -1,6 +1,13 @@ import { spawn } from "node:child_process"; import { randomUUID } from "node:crypto"; -import { access, stat, symlink, writeFile } from "node:fs/promises"; +import { + access, + mkdir, + readFile, + stat, + symlink, + writeFile, +} from "node:fs/promises"; import { once } from "node:events"; import path from "node:path"; @@ -356,6 +363,46 @@ describe("operational state", () => { } }); + it("quarantines worktrees absent from an older restored backup", async () => { + const temporary = await temporaryDirectory("mill-state-restore-orphan-"); + process.env.MILL_STATE_HOME = temporary.path; + const repositoryId = "11111111-1111-4111-8111-111111111111"; + const store = await StateStore.open(repositoryId, temporary.path); + const backup = await store.backup(); + const newerWorktree = path.join(store.worktreesDirectory, "newer-run"); + await mkdir(newerWorktree); + store.close(); + try { + const result = await restoreStateBackup( + repositoryId, + temporary.path, + backup, + ); + expect(result.quarantinedCount).toBe(1); + expect(result.quarantineManifest).toBeDefined(); + await expect(access(newerWorktree)).rejects.toMatchObject({ + code: "ENOENT", + }); + const manifestPath = result.quarantineManifest; + expect(manifestPath).toBeDefined(); + if (manifestPath === undefined) throw new Error("manifest missing"); + const manifest = JSON.parse(await readFile(manifestPath, "utf8")) as { + protocol: string; + worktrees: { original: string; quarantined: string }[]; + }; + expect(manifest.protocol).toBe("database_swap_commit_point"); + expect(manifest.worktrees).toHaveLength(1); + expect(manifest.worktrees[0]?.original).toBe(newerWorktree); + const quarantined = manifest.worktrees[0]?.quarantined; + expect(quarantined).toBeDefined(); + if (quarantined === undefined) throw new Error("worktree missing"); + const quarantineStat = await stat(quarantined); + expect(quarantineStat.mode).toEqual(expect.any(Number)); + } finally { + await temporary.cleanup(); + } + }); + it("uses an OS-released SQLite lease and fails closed for corrupt lease state", async () => { const temporary = await temporaryDirectory("mill-state-recovery-"); process.env.MILL_STATE_HOME = temporary.path; diff --git a/test/schemas.test.ts b/test/schemas.test.ts index 3079160..064e592 100644 --- a/test/schemas.test.ts +++ b/test/schemas.test.ts @@ -81,6 +81,44 @@ const samples = { }, }, }, + deliveryRecord: { + schemaVersion: "1", + runId: "123e4567-e89b-42d3-a456-426614174000", + deliveryKey: digest, + proposalDigest: digest, + approvalExpiresAt: "2026-09-01T12:15:00.000Z", + state: "planned", + target: { + forge: "github", + host: "github.com", + owner: "example", + repository: "app", + repositoryNodeId: "R_example", + cloneUrl: "https://github.com/example/app.git", + remoteName: "origin", + baseBranch: "main", + actorLogin: "operator", + actorId: 1, + }, + branchName: "mill/task-123e4567", + candidateCommit: "a".repeat(40), + candidateTree: "b".repeat(40), + requiredChecks: ["validate"], + reviewPolicy: { + mode: "local_only", + requiredReviewerLogins: [], + }, + allowedMergerLogins: ["operator"], + allowedMergeMethods: ["linear_tree_preserving"], + effects: [], + remoteHeadCommit: null, + pullRequest: null, + observation: null, + merge: null, + lastErrorCode: null, + createdAt: "2026-09-01T12:00:00.000Z", + updatedAt: "2026-09-01T12:00:00.000Z", + }, millLock: { schemaVersion: "1", mill: { package: "@davidahmann/mill", version: "0.0.0-development" }, @@ -158,6 +196,7 @@ const schemaFiles = { scenarioSet: "scenario-set.schema.json", outcomePlan: "outcome-plan.schema.json", millConfig: "mill-config.schema.json", + deliveryRecord: "delivery-record.schema.json", millLock: "mill-lock.schema.json", taskPacket: "task-packet.schema.json", contextManifest: "context-manifest.schema.json", @@ -180,6 +219,7 @@ describe("compact schemas", () => { return false; } }); + ajv.addFormat("date-time", (value) => Number.isFinite(Date.parse(value))); ajv.addFormat("email", /^[^\s@]+@[^\s@]+$/u); for (const kind of Object.keys( schemaFiles, @@ -290,6 +330,92 @@ describe("compact schemas", () => { ); }); + it("requires an exact GitHub proposal boundary at the propose trust ceiling", async () => { + const ajv = new Ajv2020({ allErrors: true, strict: true }); + ajv.addFormat( + "uuid", + /^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/iu, + ); + ajv.addFormat("uri", (value) => { + try { + void new URL(value); + return true; + } catch { + return false; + } + }); + ajv.addFormat("date-time", (value) => Number.isFinite(Date.parse(value))); + const validate = ajv.compile( + JSON.parse( + await readFile(path.join("schemas", "mill-config.schema.json"), "utf8"), + ), + ); + const base = { + ...samples.millConfig, + trustCeiling: "propose" as const, + }; + expect(validate(base)).toBe(false); + expect(contractSchemas.millConfig.safeParse(base).success).toBe(false); + const localReview = { + ...base, + propose: { + forge: "github", + host: "github.com", + owner: "example", + repository: "app", + repositoryNodeId: "R_example", + remoteName: "origin", + baseBranch: "main", + branchPrefix: "mill/", + allowedActors: ["operator"], + allowedMergerLogins: ["operator"], + requiredChecks: ["validate"], + reviewPolicy: { + mode: "local_only", + requiredReviewerLogins: [], + }, + allowedMergeMethods: ["linear_tree_preserving"], + }, + } as const; + expect(validate(localReview)).toBe(true); + expect(contractSchemas.millConfig.safeParse(localReview).success).toBe( + true, + ); + const emptyRequiredReview = { + ...localReview, + propose: { + ...localReview.propose, + reviewPolicy: { + mode: "github_required", + requiredReviewerLogins: [], + }, + }, + } as const; + expect(validate(emptyRequiredReview)).toBe(false); + expect( + contractSchemas.millConfig.safeParse(emptyRequiredReview).success, + ).toBe(false); + const deliveryValidate = ajv.compile( + JSON.parse( + await readFile( + path.join("schemas", "delivery-record.schema.json"), + "utf8", + ), + ), + ); + const invalidDeliveryReview = { + ...samples.deliveryRecord, + reviewPolicy: { + mode: "github_required", + requiredReviewerLogins: [], + }, + } as const; + expect(deliveryValidate(invalidDeliveryReview)).toBe(false); + expect( + contractSchemas.deliveryRecord.safeParse(invalidDeliveryReview).success, + ).toBe(false); + }); + it("rejects option-like and whitespace-bearing Git base references", async () => { const ajv = new Ajv2020({ allErrors: true, strict: true }); ajv.addFormat("email", /^[^\s@]+@[^\s@]+$/u);