Skip to content

feat(extension): add default_testsuite_as_full to neutralize partial-run for defaultTestSuite-resolved selection#237

Merged
wadakatu merged 2 commits into
mainfrom
feat/default-testsuite-as-full
May 13, 2026
Merged

feat(extension): add default_testsuite_as_full to neutralize partial-run for defaultTestSuite-resolved selection#237
wadakatu merged 2 commits into
mainfrom
feat/default-testsuite-as-full

Conversation

@wadakatu
Copy link
Copy Markdown
Collaborator

Summary

OpenApiCoverageExtension に opt-in パラメータ default_testsuite_as_full を追加。true かつ Configuration::includeTestSuites()Configuration::defaultTestSuite() と完全一致するときだけ --testsuite 由来の partial 判定を解除し、strict_required ゲート等が defaultTestSuite ベースの canonical full run でも到達可能になるようにする。

Why

phpunit.xmldefaultTestSuite を宣言しているプロジェクトでは、引数なしの phpunit 実行でも PHPUnit が Configuration::includeTestSuites() にその名前を埋める。結果として #221 の partial 判定が常に立ち、strict_required ゲート (#225) はユーザーの default workflow からは決して評価されない (常に [OpenAPI Strict Required] NOTE: skipped on partial runs ... のみ)。

複数 suite を切っているプロジェクト (make test で Stripe 統合だけ別 job に分離する等) は defaultTestSuite を外せず、phpunit-override 等の二重 config を強いられる状況だった。

#236 で議論した 3 案のうち短期推奨の opt-in パラメータ案 (A) を採用。--testsuite signal のみを打ち消す scoped escape hatch にすることで、--filter / path args 等の他 partial signal は引き続き保護する。

Fixes #236

Verification

  • 失敗テストを先に追加 (tests/Unit/Internal/PartialRunDecisionTest.php の新規 5 ケース) し、PartialRunDecision::fromSignals / OpenApiCoverageExtension::detectPartialRun を実装して green に。

  • composer ci (PHPUnit 1715 / 4029 assertions + PHPStan level 6 + PHP-CS-Fixer dry) すべて pass。

  • 手元の studio-api 構成での再現は本 PR では未実施 (本リポジトリの単体テストに反映済み)。Reviewer 側で必要なら issue feat(extension): treat defaultTestSuite-resolved selection as full run for strict_required / coverage gates #236 の再現プロジェクトで default_testsuite_as_full=true を設定し、[OpenAPI Strict Required] NOTE: skipped on partial runs が出なくなることを確認できます。

  • composer test passes

  • composer stan passes

  • composer cs-check passes

Notes for reviewers

  • PartialRunDecision::fromSignals は internal クラスなので、optional 引数 2 つの追加は SemVer 上問題なし (UPGRADING.md の Frozen に該当しない)。既存呼び出しは引数の名前付きで指定されているため、追加位置 (末尾) で破壊しないことを確認済み。
  • default_testsuite_as_full=true を立てると output_file 等の persistent 書き込みも実行される。defaultTestSuite に含まれない suite (例: 別 job で動かす Integration suite) が抜けた状態のレポートが書かれる点はユーザー側のトレードオフ。docs/ci.md の新節と docs/strict-required.md の Partial-run skipping 節に明記。
  • issue feat(extension): treat defaultTestSuite-resolved selection as full run for strict_required / coverage gates #236 の中期案 (C: PartialRunDecision を output-persistence と gate で 2 系統に分離) は本 PR の範囲外。需要があれば follow-up issue を起票。
  • Configuration::hasDefaultTestSuite() / defaultTestSuite() は PHPUnit 11/12/13 すべてに存在するため readTestSuiteList() のような dynamic dispatch は不要。hasDefaultTestSuite() の guard は NoDefaultTestSuiteException の throw を避けるために必須。

wadakatu added 2 commits May 13, 2026 16:27
…run for defaultTestSuite-resolved selection

When `phpunit.xml` declares `defaultTestSuite`, PHPUnit fills
`Configuration::includeTestSuites()` with that name even on a bare
`phpunit` run. The #221 partial-run detection therefore marks every
such run as partial, which (since #225) silently skips the
`strict_required` gate with no way to reach it from the user's
default workflow.

Add an opt-in `default_testsuite_as_full` parameter (default `false`)
that neutralises the `--testsuite` partial signal — but only that
signal, and only when `includeTestSuites === [defaultTestSuite]`.
Other partial signals (`--filter`, path args, `--group`, …) keep
their verdict, so the escape hatch cannot silently relax unrelated
selections. Output-persistence skipping (the #221 guarantee) is
re-enabled together with the gate, which is the user-chosen
trade-off documented in `docs/ci.md`.

Closes #236
…te_as_full opt-in

Review follow-ups for #237:

- WARN when `default_testsuite_as_full=true` but `defaultTestSuite` is
  absent or empty. Without this, the opt-in is silently inert and users
  can't tell why their `strict_required` gate still skips. Gated on the
  opt-in itself so non-opt-in callers stay silent.
- New `OpenApiCoverageExtensionBootstrapTest` cases pin the bootstrap
  wiring end-to-end: opt-in + missing default → WARN, opt-in + empty
  default → WARN, opt-in + matching default → no WARN, no opt-in even
  with matching default → no WARN. The subscriber's persistent-write
  WARN is already exercised by `CoverageReportSubscriberPartialRunTest`.
- ASCII-only ci.md heading for the opt-in section so its anchor
  (`#default_testsuite_as_full-opt-in`) is robust across GitHub renderer
  versions; setup.md / strict-required.md links re-pointed accordingly.
@wadakatu
Copy link
Copy Markdown
Collaborator Author

Follow-up commit (1bff7c0)

Applied review findings (multi-agent review surfaced 1 critical + 5 important + several suggestions; addressed critical & all important):

  • Silent failure (IMPORTANT-1/-2): readDefaultTestSuite() now WARNs when default_testsuite_as_full=true but defaultTestSuite is absent or empty. Gated on the opt-in itself — non-opt-in callers stay silent.
  • Integration coverage (C1): 4 new cases in OpenApiCoverageExtensionBootstrapTest pin the bootstrap wiring end-to-end (opt-in × {missing / empty / matching default} × {opt-in / no opt-in}). The subscriber's persistent-write WARN remains covered by CoverageReportSubscriberPartialRunTest.
  • Doc anchors (comment-analyzer Important 1/2): ASCII-only ci.md heading (default_testsuite_as_full opt-in) so anchors are robust across GitHub renderer versions; setup.md / strict-required.md links re-pointed accordingly.

composer ci passes locally: 1719 tests / 4035 assertions, PHPStan level 6, PHP-CS-Fixer dry.

Suggestions (anchor robustness alternatives, additional --exclude-testsuite matrix expansion, resolveBooleanFlag invalid-value pinning) intentionally deferred to follow-up — they are nice-to-haves on a path that's already covered by the new integration tests + the unit --filter cross-signal test.

@wadakatu wadakatu merged commit ce72f9e into main May 13, 2026
16 checks passed
@wadakatu wadakatu deleted the feat/default-testsuite-as-full branch May 13, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(extension): treat defaultTestSuite-resolved selection as full run for strict_required / coverage gates

1 participant