Merge upstream

.agents/skills/minimizing-ty-ecosystem-changes/SKILL.md

@@ -5,117 +5,89 @@ description: Use when a user says "minimize this ty ecosystem change", "reproduc
 
 # Minimizing Ty Ecosystem Changes
 
-Use this skill when asked to reproduce a ty ecosystem change, investigate a behavior difference in a primer project, or minimize a reproducer from a ty ecosystem project.
+## Invariants
 
-## Building ty
+1. Use the exact Ruff revisions, PR config, dependency cutoff, mypy-primer revision, and project Python version from the Actions run.
+2. Reproduce the reported project difference before explaining it or writing a smaller example.
+3. Treat copied binaries and config as read-only, and verify every reduction against both binaries.
 
-If a primary agent supplies paths to a copied merge-base ty binary, copied PR ty binary, and copied PR ecosystem config that it freshly prepared at the start of the current task, reuse those artifacts as read-only inputs. Do not rebuild ty, switch refs in the Ruff checkout, or overwrite the supplied artifacts. Use a separate transient ecosystem-project directory from any concurrent subagents. If a supplied artifact is missing, stop and report that to the primary agent instead of mutating the Ruff checkout.
+Start each investigation from fresh artifacts. Do not trust retained memories, previous minimizations, current upstream project state, or the helper script's default lockfile.
 
-Otherwise, prepare the artifacts as follows.
+## Collect Exact-Run Metadata
 
-Build ty on both the PR branch and the PR's merge base by running `CARGO_PROFILE_PROFILING_DEBUG=line-tables-only cargo build --package ty --profile profiling` from the repository root on both refs, matching ecosystem CI. Copy each built executable to a stable path before checking out the other ref; otherwise, the second build overwrites `target/profiling/ty`.
+Run the bundled helper with the Actions run ID or URL and every affected mypy-primer project name:
 
-From the PR branch, also copy `.github/ty-ecosystem.toml` to a stable path before checking out another ref. Use that copied file for every comparison, matching ecosystem CI's behavior of reusing the PR config.
+```bash
+scripts/collect_ty_ecosystem_run_metadata.py \
+  <actions-run> <project-name>... \
+  --output target/ty-ecosystem-run.json
+```
+
+The manifest contains the analyzed Ruff revisions, Actions `EXCLUDE_NEWER`, ecosystem-analyzer and mypy-primer revisions, and each project's CI Python version. Stop if the helper cannot determine a unique value; never substitute a comment timestamp or local default.
+
+## Prepare ty
 
-Before checking out another ref, inspect the working tree. If `git status --short` shows any staged, unstaged, or untracked changes, abort immediately and ask the user how to proceed.
+If a primary agent supplied freshly copied base and PR binaries plus the PR ecosystem config, verify the paths exist and reuse them. Do not rebuild, switch Ruff refs, or overwrite the shared artifacts.
 
-For example:
+Otherwise, require a clean working tree, copy `.github/ty-ecosystem.toml` from the PR revision, and build ty on the manifest's merge base and PR revision:
+
+Fetch the PR revision explicitly because pull-request runs usually use a synthetic GitHub merge commit that a normal clone does not contain:
 
 ```bash
 set -euo pipefail
 
-if [[ -n "$(git status --short)" ]]; then
-    git status --short
-    echo "working tree is not clean; stop and ask the user how to proceed" >&2
-    exit 1
-fi
-
+test -z "$(git status --short)" || { git status --short; exit 1; }
+git fetch origin <pr-revision>
 mkdir -p target/ty-ecosystem-bins
-cp .github/ty-ecosystem.toml target/ty-ecosystem-bins/ty-ecosystem.toml
-export TY_CONFIG_FILE="$PWD/target/ty-ecosystem-bins/ty-ecosystem.toml"
 export CARGO_PROFILE_PROFILING_DEBUG=line-tables-only
 
 git checkout <merge-base>
 cargo build --package ty --profile profiling
 cp target/profiling/ty target/ty-ecosystem-bins/ty-base
 
-git checkout <pr-branch-or-sha>
+git checkout <pr-revision>
+cp .github/ty-ecosystem.toml target/ty-ecosystem-bins/ty-ecosystem.toml
 cargo build --package ty --profile profiling
 cp target/profiling/ty target/ty-ecosystem-bins/ty-pr
 ```
 
-On Windows, adapt the shell commands and executable paths as necessary.
-
-You should only need to build ty twice at the start of the investigation. Reuse the copied executables when determining if the behavior difference still reproduces.
+## Reproduce
 
-## Reproduce First
-
-From the repository root, export the ecosystem config that was copied from the PR branch, then use the following script to clone the ecosystem project, check out the relevant revision, and install its dependencies into `.venv`. Keep `TY_CONFIG_FILE` set when running ty comparisons; this applies the same rule configuration as ecosystem CI without overwriting the user's ty config. Shell exports may not persist between agent tool calls, so re-export `TY_CONFIG_FILE` in every new shell before invoking either ty binary. The examples below use the default artifact paths; substitute supplied artifact paths when a primary agent provides them.
-
-When preparing artifacts yourself, if `target/ty-ecosystem-bins/ty-ecosystem.toml` does not exist yet, check out the PR branch and copy `.github/ty-ecosystem.toml` from there before continuing. Do not copy `.github/ty-ecosystem.toml` while on the merge base, because ecosystem CI compares both binaries with the PR branch's config.
+Create a unique temporary directory for each project. Read its Python version and the pinned mypy-primer revision from the manifest, then bypass the adjacent script lockfile:
 
 ```bash
-set -euo pipefail
-
-export TY_CONFIG_FILE="$PWD/target/ty-ecosystem-bins/ty-ecosystem.toml"
-test -f "$TY_CONFIG_FILE" || { echo "missing $TY_CONFIG_FILE" >&2; exit 1; }
-uv run --no-project scripts/setup_primer_project.py <project-name> <some-temp-dir> --revision <report-revision> --exclude-newer <report-timestamp>
+uv run \
+  --python <project-python> \
+  --with "mypy-primer @ git+https://github.com/hauntsaninja/mypy_primer@<mypy-primer-revision>" \
+  --no-project \
+  python scripts/setup_primer_project.py \
+  <project-name> <temporary-directory> \
+  --revision <report-project-revision> \
+  --exclude-newer <EXCLUDE_NEWER>
 ```
 
-ALWAYS confirm the behavior difference reproduces before minimizing or explaining it.
-ALWAYS use this script before attempting to reproduce the change.
-NEVER try to confirm the behaviour difference without first setting up the project's virtual environment using this script.
-
-When comparing the copied ty binaries, use the project-specific ty command printed by `setup_primer_project.py`. That command includes the project's mypy-primer metadata such as `paths` and any custom `ty_cmd`, matching ecosystem CI. Keep `TY_CONFIG_FILE` exported in the same shell that runs the comparison, and set `ty_binary` to each copied binary before running the printed command:
+Use absolute paths and re-export `TY_CONFIG_FILE` in every new shell before running either binary:
 
 ```bash
 export TY_CONFIG_FILE="$PWD/target/ty-ecosystem-bins/ty-ecosystem.toml"
-test -f "$TY_CONFIG_FILE" || { echo "missing $TY_CONFIG_FILE" >&2; exit 1; }
-
-project_dir="$PWD/<some-temp-dir>"
+test -f "$TY_CONFIG_FILE"
+project_dir="$PWD/<temporary-directory>"
 ty_base="$PWD/target/ty-ecosystem-bins/ty-base"
 ty_pr="$PWD/target/ty-ecosystem-bins/ty-pr"
 
 cd "$project_dir"
 ty_binary="$ty_base"
-<project-specific ty command printed by setup_primer_project.py>
+<project-specific command printed by setup_primer_project.py>
 ty_binary="$ty_pr"
-<project-specific ty command printed by setup_primer_project.py>
+<project-specific command printed by setup_primer_project.py>
 ```
 
-If the ecosystem report gives an exact project revision, pass it to the script with `--revision`. If the report gives a timestamp, pass it to the script with `--exclude-newer`. Do not install dependencies before checking out the report revision. Do not use current upstream or the current mypy-primer revision as evidence for a historical report when the report provides a pinned revision. If running `ecosystem-analyzer` directly, also pass the report timestamp as `--exclude-newer`, matching ecosystem CI.
+Confirm the detailed report's difference exactly, including duplicate diagnostics when present.
 
 ## Minimize
 
-When asked to minimize an ecosystem change, start from the reproduced project. Reduce the Python code until only the smallest code needed to demonstrate the behavior difference remains. DO NOT look at conversation or analysis in other PR comments: your analysis should start from a clean slate. Even if the cause looks obvious, DO NOT skip ahead to an explanation or a hand-written reproducer. Use a rigorous, systematic process and verify after each reduction that the behavior difference still reproduces.
-
-An ideal minimized reproducer:
-
-- Is a single file that is as small as possible.
-- Has as few third-party imports as possible, ideally none.
-- Uses smaller third-party packages with fewer dependencies when third-party imports are unavoidable. For example, prefer an import from `numpy` over one from `pandas`.
-- Has as few first-party and standard-library imports as possible.
-- Keeps imports from modules with highly special-cased symbols only when they are necessary (such modules may include `typing`, `abc`, `enum`, `types`, or `typing_extensions`).
-- Uses an absolute minimum of "advanced"/complex typing or language features. For example, if the original code
-  uses a walrus operator, but the behavior difference still reproduces without it, remove the walrus operator. If the original code uses a `Protocol` from `typing_extensions`, but the behavior difference still reproduces without it, remove the `Protocol`.
-
-Use a systematic loop. You do not need to apply every tool in every iteration, but keep looping until none of the available minimization tools can reduce the reproducer further while preserving the behavior difference. Your clones of ecosystem projects and/or their dependencies should be treated as transient artifacts of the investigation: they should be deleted after the minimization is complete, and you SHOULD NOT ask for permission to modify them during the minimization process. If the user has asked for an ecosystem change to be minimized, ANY of the changes below should be understood as automatically having been approved by the user.
-
-Available minimization tools include:
-
-1. Remove files that are not needed for the difference.
-2. Strip imports, definitions, and statements from the remaining files.
-3. Cut and paste definitions from one file into another file to reduce first-party imports.
-4. Copy installed third-party dependency files from the project's `.venv` into the project as first-party files so that you can convert third-party imports into first-party imports. If you must clone a dependency instead, use the exact installed version rather than the dependency's current default branch.
-5. Inline the relevant parts of stdlib definitions from ty's vendored typeshed stubs into first-party code to reduce stdlib imports. ty's vendored typeshed stubs are the single source of truth for ty regarding the Python standard library, and they are found in `./crates/ty_vendored`.
-
-After applying a minimization tool, re-run the comparison between the PR branch and the PR's merge base.
-Keep a reduction only when the behavior difference still reproduces.
-
-DO NOT stop looping until no further reductions could be applied without making the behavior difference disappear.
-
-Before retaining the final reproducer, perform an import audit: attempt to delete every import, inline third-party definitions where possible, and explain why each surviving third-party import is necessary.
+Reduce the reproduced project iteratively, using the base-versus-PR output as the oracle after every change. Prefer a single file, no third-party imports, and the least complex code that preserves the difference. For nontrivial reductions, follow [references/advanced-minimization.md](references/advanced-minimization.md).
 
-If the minimized behavior change is a diagnostic change, the final minimized Python snippet MUST include the full diagnostic message and error code on both branches, as a comment above or on the relevant line of code.
+## Return
 
-ANY reference to a line number in an external project MUST use a permalink in the form `[project file.py:123](permalink)`. Referring to an external line number without a permalink is unacceptable.
+Provide the original permalinked report entry, exact base and PR behavior, minimal code, full diagnostic messages and error codes, and the manifest/commands needed to reproduce it. When called from the summary workflow, return import-audit and reduction notes separately from report-ready Markdown.

.agents/skills/minimizing-ty-ecosystem-changes/references/advanced-minimization.md

@@ -0,0 +1,26 @@
+# Advanced Minimization
+
+Use this reference after the reported difference reproduces against the copied base and PR binaries.
+
+## Target
+
+Prefer a single-file reproducer with no third-party imports, few definitions, and the least complex typing or language features that still demonstrate the difference. Keep special modules such as `typing`, `abc`, `enum`, `types`, and `typing_extensions` only when removing them changes the behavior.
+
+## Reduction Loop
+
+Work systematically from the reproduced project. Do not skip ahead to an explanation, hand-written reproducer, or a guessed subset of relevant code. Follow the stages below in order and exhaust each stage before advancing. Try one controlled reduction at a time, run both copied ty binaries after every change, and keep the reduction only if the original difference remains. After every successful reduction, restart at step 1 because it may make earlier reductions possible.
+
+1. Delete unrelated files.
+2. Remove imports, definitions, decorators, annotations, statements, and branches.
+3. Inline first-party definitions into the reproducer.
+4. For each required third-party dependency, copy the entire installed dependency into the source tree as first-party code, including every package directory and module it provides. Do this before attempting to minimize any part of the dependency. Adjust imports, verify that the difference still reproduces with the complete copy, and only then begin deleting files or definitions from it. Never start by copying only apparently relevant files or definitions. If cloning a dependency is unavoidable, use the exact installed revision or version and copy the complete dependency into the source tree before reducing it.
+5. Inline the relevant standard-library definitions from `crates/ty_vendored`, which is ty's source of truth for stdlib types.
+6. Replace complex constructs with simpler equivalents, such as removing a walrus expression or replacing a protocol when the difference survives.
+
+Repeat the full loop until an exhaustive pass through every stage finds no further reduction that preserves the difference. Do not stop merely because the likely cause is understood or the reproducer is already small.
+
+## Final Audit
+
+Attempt to remove every remaining import and inline every remaining third-party definition. Record why any surviving import is essential. Keep these notes as working evidence; the caller decides whether they belong in its final artifact.
+
+Delete transient project and dependency copies after the investigation.