Skip to content

fix(ci): stop dependency-index drift blocking weekly podcast dispatch - #832

Merged
jmservera merged 4 commits into
mainfrom
fix/w42-deterministic-lock
Oct 12, 2026
Merged

jmservera merged 4 commits into
mainfrom
fix/w42-deterministic-lock

Conversation

@jmservera

Copy link
Copy Markdown
Owner

Production cause

W42 crawl run 38208941107 succeeded, but sync run 38209680833 failed because required Lockfile resolved latest PyPI releases rather than the reviewed lock. The only diff was jsonpointer/pydantic/pydantic-core. This left generated-data PR #831 blocked and auto-dispatch 38209757756 skipped.

Durable fix

Use Python/pip dry resolution of requirements plus all exact lock pins and require the complete resolved graph to equal the lock. Reject conflicting requirements, extra/unpinned dependencies, missing packages, malformed and duplicate pins. No uv, no bypass, all required checks/audit retained. Future compatible index releases alone cannot block publication.

Validation

Actual 27-pin pip dry resolution succeeded. Seven regression tests plus test_pipeline passed. Ruff check/format passed. Local Checkov: 184 passed / 0 failed. Local zizmor 1.25.2 no findings (hosted pinned 1.27.0 pending). Operational recovery documented.

Recovery after review/merge

Rerun sync-publish-to-main.yml, inspect exact-identity W42 auto-dispatch; if manual trigger needed use week 2026-W42 and original crawl run 38208941107. Do not duplicate an accepted publication. Production provider verification still pending for W42. Required independent pipeline/security review and latest-head Copilot review before merge.

Prevent newer compatible PyPI releases from blocking the weekly sync and podcast dispatch while rejecting incompatible or incomplete locks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab74df71-7f3c-4a63-a324-7f030dd5e719
Copilot AI balanced review requested due to automatic review settings October 12, 2026 14:12
Comment thread scripts/check_dependency_lock.py Fixed
Comment thread scripts/check_dependency_lock.py Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Pip’s captured diagnostic output must remain visible when the required lockfile gate fails.

1 open finding
What changed in this PR

Replaces index-sensitive lock regeneration with exact-pin compatibility validation to prevent weekly publishing failures.

Changes:

  • Adds pip dry-run graph validation and regression tests.
  • Updates CI to use the new validator.
  • Documents sync recovery procedures.
File Description
scripts/​check_dependency_lock.py Implements exact dependency graph validation.
tests/​test_dependency_lock.py Tests accepted and rejected lock states.
.github/​workflows/​ci.yml Replaces uv regeneration with graph validation.
docs/​deployment/​publish-sync-github-app.md Documents troubleshooting and recovery.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/check_dependency_lock.py Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab74df71-7f3c-4a63-a324-7f030dd5e719

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The production sync still installs ranged requirements instead of the reviewed lock.

2 open findings

🧠 Review effort: Balanced

Comment thread .github/workflows/ci.yml
…gnostics

Address Copilot review: pin weekly sync dependencies and its cache key; retain pip stderr for actionable conflict and network diagnostics.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab74df71-7f3c-4a63-a324-7f030dd5e719
Copilot AI balanced review requested due to automatic review settings October 12, 2026 14:21
@jmservera jmservera added the squad Squad triage inbox — Lead will assign to a member label Oct 12, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The required dependency audit still resolves requirements.txt, leaving publication exposed to live-index drift.

0 open findings

2 resolved since last review
Previously missed (1)

In code that hasn't changed since last review

Medium severity Audit the lockfile instead of resolving ranged requirements

.github/​workflows/​ci.yml:84

The required dependency audit above still runs python -m pip_audit -r requirements.txt, so it independently resolves the ranged requirements against the live index instead of auditing this reviewed lock. A compatible future release can therefore still change that required check's graph (or make its resolution fail), contradicting the stated guarantee that index releases alone cannot block publishing. Point the audit at requirements.lock as well.

🧠 Review effort: Balanced

🔒 - Generated by Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ab74df71-7f3c-4a63-a324-7f030dd5e719
Copilot AI balanced review requested due to automatic review settings October 12, 2026 14:52
@jmservera

Copy link
Copy Markdown
Owner Author

Independent Frank correction pushed as c6ad320 for Rusty re-review. Requirements are now the only pip roots and requirements.lock supplies constraints; full closure must match exactly, rejecting orphan pins. The network-free real-pip defusedxml/six regression fails against the rejected -r/-r algorithm and passes with the correction. CI audits deployed requirements.lock with --strict --disable-pip --no-deps; no vulnerability exemptions. Weekly sync still installs exact lock pins. Validation: 59 targeted dependency/workflow tests passed, actual 27-pin resolution passed, strict deployed-pin audit found no known vulnerabilities, Ruff passed, Checkov 3.2.533 passed 224 checks with zero failures, pinned Zizmor 1.27.0 found no blocking findings on both workflows. Prior latest-head Copilot audit finding is addressed. No merge, publication, or production recovery performed. Awaiting fresh latest-head Copilot review and CI; Rusty retains approval ownership.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Production workflow and dependency-security changes still require the mandated independent pipeline and security review.

0 open findings

🧠 Review effort: Balanced

@jmservera

Copy link
Copy Markdown
Owner Author

Corrected-head verification complete for c6ad320: all 18 reported CI/security/site checks passed. Automatic Copilot review finished on this exact SHA at 2026-10-12T14:56:13Z with 0 open findings; all existing review threads remain resolved (no unrelated threads changed). Copilot explicitly retains the repository-mandated independent pipeline/security review requirement; this is ready for Rusty re-review, not authorization to merge. Basher did not participate in this independent revision. Worktree is clean; validation scratch artifacts removed. No merge or publication performed.

@jmservera

Copy link
Copy Markdown
Owner Author

URL — named AI pipeline reviewer: APPROVE for head c6ad320.

Bounded pipeline review completed: requirements.txt supplies the resolver roots; requirements.lock supplies exact constraints; full normalized closure equality rejects missing, changed, and orphan pins without upgrading reviewed versions. The real-resolver orphan regression is included. Sync installation and its pip cache use requirements.lock. CI audits those pins with --strict --disable-pip --no-deps; audit failures remain blocking. Required check names, production gates, triggers, permissions, and merge checks are not weakened by this diff.

Verified hosted CI for this head (GitHub PR merge checkout): Lockfile validated 27 exact pins; strict audit reported no known vulnerabilities; all 19 dependency-lock tests passed within 1957 passing Python tests. All 18 reported checks succeeded; latest Copilot review has zero open findings and no unresolved threads.

No high-confidence pipeline blockers. This is URL AI pipeline approval only, not human signoff, security approval, functional approval, merge authorization, or publication verification. Required independent security review and human signoff remain separate. No implementation, uv execution, or merge performed.

@jmservera

Copy link
Copy Markdown
Owner Author

Hermes — AI-assisted mandated workflow safety review: APPROVE for head c6ad320. No high-confidence blocking findings. Requirements-rooted pip resolution uses lock constraints and exact closure comparison; missing/orphan/conflicting pins fail. Sync installs the reviewed lock. Dependency audit remains blocking with --strict --disable-pip --no-deps on actual pins, without vulnerability exclusions. Fixed argv/no shell; pip stderr remains visible; no new secret exposure or permissions expansion. Verified 18/18 checks successful, latest-head Copilot review completed, all threads resolved, and four targeted local tests passed. This is an AI review, not human signoff or merge/publication authorization.

@jmservera
jmservera merged commit c5ebb7c into main Oct 12, 2026
19 checks passed
@jmservera
jmservera deleted the fix/w42-deterministic-lock branch October 12, 2026 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

squad Squad triage inbox — Lead will assign to a member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants