Skip to content

feat(ci): stage catalog sync and auto-fix with review skill - #825

Open
Yimin-Jin wants to merge 2 commits into
template/devfrom
yimin/review-sample-catalog-skill
Open

Yimin-Jin wants to merge 2 commits into
template/devfrom
yimin/review-sample-catalog-skill

Conversation

@Yimin-Jin

@Yimin-Jin Yimin-Jin commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Keep one Sync Sample Catalog workflow and one job, with separate scan, metadata, grouping, Details, write, Draft PR, and final skill-review steps.
  • Automatically apply bounded prose corrections: at most two repair passes and one fresh final verification; unresolved findings fail the workflow while preserving the Draft PR. No manual corrections are required on the successful path, and approval/merge remain human-controlled.
  • Run pinned Copilot CLI 1.0.88 in a read-only, network-isolated container with only view/grep/glob/skill tools. The host retains GitHub/Azure credentials and exposes only a bounded inference proxy.
  • Validate field scope, pinned-source quotations, catalog invariants, and trusted tests before appending a commit parented to the captured PR head with a non-force ref update. Refuse reruns that would overwrite an existing run branch.
  • Use the same review-sample-catalog skill for CI and maintainer review. The catalog snapshot itself is unchanged.

Validation

  • Local Node regression suite: 123/123 passing.
  • Workflow YAML, single-job ordering, skill links, syntax/diagnostics, and git diff checks passed.
  • Offline Docker/CLI smoke test is wired into the existing validation step; GitHub runner verification is in progress because the local Docker daemon is unavailable.
  • A live Azure model review and automatic data-PR publication have not yet been verified end to end.

Guardrails

  • No agent shell, edits, GitHub tools, source execution, or raw write credentials.
  • Maximum three agent passes, 40 model requests and 300k reported tokens per pass, 16k output tokens/request, 12 minutes/pass.
  • Only affected card prose and new template name/description may change; existing identities, membership, Patterns and unaffected metadata/Details are protected.
  • Evidence-reference validation is deterministic, but semantic judgment remains model-based; blocked reports must never be treated as approval.

Add an on-demand repository skill for the agreed CI-generated Draft PR followed by human-led AI review. Define snapshot and structural checks, per-implementation semantic standards, minimal authorized data fixes, validation and release-promotion boundaries. Reference current code rather than hard-coded counts or versions; do not change the sync workflow.
Split incremental generation into resumable stages and append a sandboxed Copilot skill review to the same Draft PR workflow. Keep repository and model credentials outside the agent, constrain prose patches and pinned-source evidence, and publish via non-force Git ref updates after trusted validation. Add offline container smoke coverage and 123 passing regression tests.
@Yimin-Jin Yimin-Jin changed the title docs(skills): define source-grounded sample catalog review feat(ci): stage catalog sync and auto-fix with review skill Sep 24, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant