Skip to content

Trim Cluster A's vendored deployment/upgrade battery - #1288

Merged
mkarlesky merged 1 commit into
next_versionfrom
test/cluster-a-battery-trim
Sep 14, 2026
Merged

mkarlesky merged 1 commit into
next_versionfrom
test/cluster-a-battery-trim

Conversation

@mkarlesky

Copy link
Copy Markdown
Member

Summary

PR 3 of 5 in the system-test redesign (Stage 2 plan, following the "System Test Duplication Assessment" report's Finding 1 -- the largest single lever, an estimated 17.0% of measured system-test time / ~76 min of full-matrix CI compute per run).

deployment_as_gem_spec.rb is untouched and stays the canonical full CommonSystemTestCases battery (52 examples) -- the one place install-mode-agnostic behavior (preprocessing, Partials, defines, Unity features, crash handling, verbosity, test filtering) runs, under the default (most common real-world) install mode.

Changes

  • deployment_as_vendor_spec.rb: 82 → 13 examples. Each of its three ceedling new scaffold variants (--local --docs; --local --docs --gitsupport; --local alone) keeps only its own distinguishing scaffolding-specific checks (does the right vendor/docs/git structure exist) plus one test_project_success smoke proving that variant's vendored install actually builds and runs something for real. Drops version reporting (redundant with the gem spec) and the full preprocessing/Partials/defines/Unity/crash/verbosity/filtering battery -- pure build/runtime logic already proven once, install-mode-agnostic.
  • upgrade_as_vendor_spec.rb: 32 → 10 examples. Keeps its only unique payload (can_upgrade_projects/can_upgrade_projects_with_no_test_support_folder -- the only tests that invoke ceedling upgrade at all -- and cannot_upgrade_non_existing_project, the only coverage of the failure path) plus a lean pre/post test_project_success pair bracketing the upgrade, dropping the repeated 8-item generic battery that ran identically before and after upgrade today.

Cluster A total: 166 → 75 examples (55% reduction).

Coverage retention

Verified by comparing test_case symbol sets across the three files: every symbol dropped from vendor/upgrade is either vendor/upgrade-specific (kept in its own file) or still present in the untouched deployment_as_gem_spec.rb. comm -13 between the two symbol sets confirms exactly the 7 expected vendor/upgrade-specific symbols, no unexpected orphans -- nothing silently lost coverage.

Verification

  • All 75 examples pass together, host (macOS) and the madsciencelab-plugins Docker image (Linux).
  • 4 GDB-related failures seen on host are confirmed pre-existing and environmental: reproduced identically against the unmodified deployment_as_gem_spec.rb on a clean git stash (macOS gdb codesigning issue -- "please check gdb is codesigned - see taskgated(8)"), unrelated to this change. Clean (0 failures) on Linux, where that macOS-specific quirk doesn't apply.
  • Full-suite validation (all 16 CI matrix jobs) is CI's job per the Stage 2 plan's process -- not duplicated locally.

This is PR 3 of 5; the plan calls for a go-ahead before starting PR 4 (mixin tier split).

🤖 Generated with Claude Code

PR 3 of the system-test redesign (Stage 2 plan, Finding 1 -- the largest
lever, 17.0% of measured system-test time). deployment_as_gem_spec.rb is
untouched and stays the canonical full CommonSystemTestCases battery (52
examples, all install-mode-agnostic behavior: preprocessing, Partials,
defines, Unity features, crash handling, verbosity, test filtering).

deployment_as_vendor_spec.rb (82 -> 13 examples): each of its three
`ceedling new` scaffold variants (--local --docs; --local --docs
--gitsupport; --local alone) keeps only its own distinguishing
scaffolding-specific checks (does the right vendor/docs/git structure
exist) plus one test_project_success smoke proving that variant's
vendored install actually builds and runs something for real -- file
presence alone wouldn't catch a broken vendored $LOAD_PATH. Drops
version reporting (redundant with the gem spec's own coverage) and the
full preprocessing/Partials/defines/Unity/crash/verbosity/filtering
battery (47 examples across the three blocks -- pure build/runtime logic
already proven once, install-mode-agnostic).

upgrade_as_vendor_spec.rb (32 -> 10 examples): keeps its only unique
payload -- can_upgrade_projects/can_upgrade_projects_with_no_test_support_folder
(the only tests that invoke `ceedling upgrade` and assert success) and
cannot_upgrade_non_existing_project (the only coverage of the upgrade
failure path) -- plus a lean pre/post test_project_success pair bracketing
the upgrade itself, dropping the repeated 8-item generic battery that ran
identically before *and* after upgrade today.

Verified by comparing test_case symbol sets: every symbol dropped from
vendor/upgrade is either vendor/upgrade-specific (kept) or still present
in the untouched deployment_as_gem_spec.rb (comm -13 confirms exactly 7
expected vendor/upgrade-specific symbols, no orphans).

Cluster A total: 166 -> 75 examples (55% reduction). All 75 pass together,
host and the madsciencelab-plugins Docker image (Linux) -- 4 GDB-related
failures on host are confirmed pre-existing/environmental (macOS gdb
codesigning, "please check gdb is codesigned - see taskgated(8)"),
reproduced identically against the unmodified deployment_as_gem_spec.rb on
a clean stash, unrelated to this change; clean on Linux where that
macOS-specific gdb quirk doesn't apply.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mkarlesky
mkarlesky merged commit 10b60be into next_version Sep 14, 2026
19 checks passed
@mkarlesky
mkarlesky deleted the test/cluster-a-battery-trim branch September 14, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant