Skip to content

fix(release-validation): untrack operator fleet file, add boxes.example.toml (#2271) - #2300

Merged
thinmintdev merged 2 commits into
mainfrom
fix/untrack-release-validation-boxes
Oct 5, 2026
Merged

thinmintdev merged 2 commits into
mainfrom
fix/untrack-release-validation-boxes

Conversation

@thinmintdev

Copy link
Copy Markdown
Contributor

What changed

  • tests/release-validation/boxes.toml is removed from the index and added to .gitignore (the working copy stays on the operator's disk).
  • New tests/release-validation/boxes.example.toml: same schema (hypervisor, ssh_key, ssh_user, every [boxes.<name>] table and key), placeholder values only (RFC 5737 192.0.2.x, ~/.ssh/<your-key>, generic hostnames), field-meaning comments kept, operator narrative dropped.
  • tests/release-validation/README.md:18 now says the file is gitignored and created by copying the example; docs/reference/support-matrix.mdx and the box-role comment in .claude/workflows/rc-validate.js:42 point at the example. Historical reports are untouched.
  • tests/release/test_kit_ct151_reset_gotchas.py read the real file (lines 17-22); it now reads boxes.example.toml, which carries the generic ct151 post-rollback checklist the test pins. Added two guards: the example parses, has the schema keys and no RFC 1918 address or real ~/.ssh/ key; and boxes.toml is not tracked (git ls-files, skipped outside a git checkout).

Why

CLAUDE.md forbids host names, LAN addresses and operator-local paths in tracked files; the old file published the lab topology. History scrubbing (step 3 of the issue) is left to the operator.

Verification

  • With boxes.toml force-added, pytest tests/release/test_kit_ct151_reset_gotchas.py gives 1 failed, 5 passed (the untracked guard fails); after untracking, 6 passed.
  • ls-files tests/release-validation no longer lists boxes.toml; the new .gitignore line matches it.
  • uv run ruff format --check src tests: 1304 files already formatted. uv run ruff check src tests: All checks passed.
  • uv run pytest tests/ -q -x -m "not integration": 4195 passed, 4 skipped, 1 xfailed, then stopped on tests/cli/test_doctor.py::test_preflight_ports_soft_mode_downgrades_to_warning, which fails in this root sandbox independent of this change. A rerun with that test deselected was clean through ~69% before hitting the time limit.

Closes #2271

🤖 Generated with Claude Code

https://claude.ai/code/session_017bDysVfpfD2VTJfPcQwCcM


Generated by Claude Code

…ple.toml (#2271)

tests/release-validation/boxes.toml described the operator's private lab
(guest LAN addresses, hypervisor address, ssh key path), which CLAUDE.md
forbids in tracked files. Gitignore it, drop it from the index, and commit
a placeholder boxes.example.toml with the same schema. The ct151 reset
checklist test now reads the example, and two guards assert the example
carries no RFC 1918 address or real key path and that boxes.toml stays
untracked.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017bDysVfpfD2VTJfPcQwCcM
Signed-off-by: Claude <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@thinmintdev thinmintdev left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review of 8bcf2c7. Verdict: APPROVE. (Posted as a comment review because GitHub does not let the PR author account approve its own PR.)

What I verified, with evidence

  1. The reported path is out of the index. git ls-files tests/release-validation no longer lists boxes.toml. git check-ignore -v matches it at .gitignore:157. I parsed both files with tomllib and compared the full set of dotted keys between the removed file (origin/main) and boxes.example.toml. Neither file has a key the other lacks, and all 53 keys match. That covers the top-level hypervisor, ssh_key and ssh_user, all four [boxes.*] tables, and every key inside them.
  2. Leak check is clean. I took every IP address, the hypervisor address, the ssh key path, the key file name, any home-directory path, and each box hostname, ip and api value from the removed file. I left out two hostnames that are just the product name. None of those values appears in any added line of the diff. None appears anywhere in the example, README.md, docs/reference/support-matrix.mdx, .claude/workflows/rc-validate.js, the test file or .gitignore. None of those files contains an RFC 1918 address. The only ~/.ssh/ path is the <your-key> placeholder. The example uses RFC 5737 192.0.2.x addresses and *-example hostnames.
  3. The new guards fail without the fix.
    • I force-added a dummy tests/release-validation/boxes.toml. test_operator_fleet_file_is_not_tracked then failed (1 failed, 5 passed). After git rm --cached and cleanup, all 6 pass.
    • I planted a 192.168.x address in the example. test_example_fleet_file_has_schema_and_no_private_details failed. It also failed when I replaced the key placeholder with a real-looking key name. I restored the file afterwards.
  4. The retarget did not weaken the contract. The only change to the four existing tests is the BOXES path at tests/release/test_kit_ct151_reset_gotchas.py:22. The DELETES fact, the exact dev0/dev3 re-add command, pct set 151 --nameserver and apt install -y curl jq are all still pinned, and the example carries them in [boxes.ct151-cpu-fresh].notes. One thing is unavoidably lost: the operator's untracked copy is no longer checked, because it isn't in the repo any more.
  5. No scope creep. The rc-validate.js:42 change is a comment only. The support-matrix.mdx:22 change is a path pointer only. The diff does not touch CHANGELOG.md, .github/, migrations or infrastructure.
  6. Lint, tests and sign-off pass. ruff format --check src tests reports 1304 files already formatted. ruff check src tests reports all checks passed. pytest tests/release -q gives 143 passed. The commit has a Signed-off-by trailer.

Not blocking (follow-up issue suggested)

The same lab subnet and the same ssh key name still appear in about 24 other tracked files that this PR does not touch, for example scripts/fresh-test-ct.sh:22,27, several tests/api and tests/agents fixtures, and the historical tests/release-validation/reports/*.md. Issue #2271 only covers boxes.toml, so these belong in a separate issue. The new guard also checks only for RFC 1918 addresses and key paths, not hostnames, which is fine for a template.


Generated by Claude Code

@thinmintdev
thinmintdev merged commit 4bb47a3 into main Oct 5, 2026
11 checks passed
@thinmintdev
thinmintdev deleted the fix/untrack-release-validation-boxes branch October 5, 2026 03:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-agent PRD is fully scoped and ready for an AFK agent to pick up

Projects

None yet

Development

Successfully merging this pull request may close these issues.

repo: tests/release-validation/boxes.toml is tracked and carries lab addresses, a hypervisor address and an ssh key path

2 participants