Skip to content

test(scripts): add platform support-declaration consistency tests - #24

Merged
phodal merged 1 commit into
QoderAI:mainfrom
creayma-del:test/support-declaration-consistency
Jul 30, 2026
Merged

test(scripts): add platform support-declaration consistency tests#24
phodal merged 1 commit into
QoderAI:mainfrom
creayma-del:test/support-declaration-consistency

Conversation

@creayma-del

@creayma-del creayma-del commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a contract test that keeps the platform support declarations named by the
roadmap Definition of Done in agreement: session-analysis CLI help and platform
gate, the exported SESSION_ANALYSIS_HELP, the agent-customize provider
registry, createAnalyzer rejection, harness analyze help and platform gate,
the asset-baseline provider gate, and the host adapter matrix. Any host added
or dropped in one owner without the others now fails npm test.

To let the test assert the report platform gate's whitelist (not just that it
rejects unknown input), the reportPlatform gate's rejection error now names
the full supported set, matching the session-analysis and asset-baseline gates.

Why

  • Issue/Story: roadmap.md TODO A-06 (P0) — "Add support-declaration
    consistency tests."
  • User or maintainer outcome: the supported platform set (qoder, codex,
    claude, cursor, qwen, copilot) is declared independently in at least
    five places. Nothing asserted they agree, so adding a host in one owner
    silently left CLI help, error messages, and docs advertising different
    platform sets (this actually happened when qwen and later copilot
    landed). This test closes that gap.

Traceability and Scope

  • Spec/ADR, if applicable: docs/specs/2026-07-28-a06-support-declaration-consistency.md
  • Acceptance criteria addressed: AC-1 (every surface's declared list equals the
    canonical set), AC-2 (public routes only: root facade CLI, the
    session-analysis/index.mjs and agent-customize/providers/index.mjs import
    surfaces, shipped docs), AC-3 (unsupported input fails closed naming the full
    set on each gated route), AC-4 (npm test discovers the test automatically).
  • Canonical owners changed: scripts/harness-analysis/report-run.mjs
    reportPlatform whitelist extracted to a file-local REPORT_PLATFORMS
    constant; its rejection error now names the supported set. Each capability
    still owns its own declaration. Also adds test/support-declarations.test.mjs,
    the spec, a CHANGELOG.md entry, and checks the A-06 roadmap box.
  • Explicit non-goals: no change to which platforms are supported; no shared
    runtime constant across modules; ANALYZE_HELP keeps its own prose list
    (asserted separately by the test); no prose-wording assertions beyond the
    declared platform lists; other commands with their own platform/provider
    lists (coding-agent-practices inventory / asset-integrity,
    agent-customize, agent-lint, evidence-bundle, task-loop-source,
    selection-profile) stay out of scope.

Change Type

  • Feature
  • Bug fix
  • Tests only
  • Refactor with no intended behavior change
  • Documentation/community
  • Dependency, packaging, or infrastructure

Test and Review Evidence

Check Result
node --test test/support-declarations.test.mjs 6/6 pass
node --test test/doc-link-graph.test.mjs 6/6 pass
node --test test/harness-report-run.test.mjs test/harness-report-quality.test.mjs test/harness-report-render-cli.test.mjs test/harness-report-source.test.mjs test/report-source-review.test.mjs 137 pass / 3 fail
Same 5 report suites on git stash (changes removed) identical 3 failures (render command Canvas suites), pre-existing on main and unrelated to this change
CHANGELOG.md Unreleased entry added for the report-gate error-message change
Rebased onto latest main (copilot host) conflict in reportPlatform resolved by keeping the constant/error-list approach and adding copilot; canonical set updated to 6 platforms; all 6 consistency tests pass

Manual or visual evidence: n/a (no visual surface).

Risk and Recovery

  • Compatibility and cross-platform impact: the only runtime change appends
    Supported platforms: ... to the reportPlatform rejection message; the
    unsupported Harness report platform: <name> prefix and the
    UNSUPPORTED_REPORT_PLATFORM error code are unchanged, and no other code or
    test depends on the old message format. Test uses process.execPath +
    spawnSync (no shell), so it runs on Windows, macOS, and Linux.
  • Package, plugin, schema, or generated-file impact: none.
  • Rollback or recovery path: revert the single commit; nothing depends on the
    new test file, the spec, or the extracted constant.
  • Residual risk or unverified boundary: the test extracts lists from current
    help/error phrasing (--platform <...>, Supported platforms: ... .);
    rewording those messages requires updating the extraction patterns. Platform
    declarations in the out-of-scope commands listed under non-goals can still
    drift and are intentionally not covered.

AI Involvement

  • Level: Generated (agent-authored; co-author recorded in the commit trailer).
  • Human review and validation: maintainer reviewed the analysis, diff, and spec
    wording, approved each step; a code-review pass caught and fixed a spec
    statement that overstated help/constant sharing; the branch was rebased onto
    the latest main and the reportPlatform conflict resolved manually; the
    listed test commands were executed locally with results reported verbatim
    above.

Checklist

  • I followed AGENTS.md, CONTRIBUTING.md, and the relevant canonical-owner guidance.
  • The change is focused and does not include unrelated local or generated state.
  • Tests and documentation match the behavior actually delivered.
  • Markdown links were checked when documentation moved or changed.
  • Cross-platform behavior was considered for Windows, macOS, and Linux.
  • Package/runtime verification was run when shipped files or dependencies changed.
  • User-facing or compatibility changes are recorded in CHANGELOG.md.
  • I have the right to contribute this work under the repository's MIT License.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a contract-style Node test to keep the declared supported host/platform set consistent across key public surfaces (CLI help + gates, exported session-analysis help, agent-customize provider registry, asset-baseline provider gate, and the adapters documentation matrix), and records the work in the roadmap and a supporting spec.

Changes:

  • Add test/support-declarations.test.mjs to assert a single canonical supported-platform set matches what multiple surfaces declare.
  • Mark roadmap item A-06 as done.
  • Add a spec documenting intent, scope, and acceptance criteria for the consistency test.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
test/support-declarations.test.mjs New contract test asserting platform declarations across CLI/help/registries/docs stay in sync
roadmap.md Checks off A-06 to reflect completion of the roadmap item
docs/specs/2026-07-28-a06-support-declaration-consistency.md Adds spec describing the consistency-test contract and acceptance criteria

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/support-declarations.test.mjs
Comment thread docs/specs/2026-07-28-a06-support-declaration-consistency.md
@creayma-del
creayma-del force-pushed the test/support-declaration-consistency branch 2 times, most recently from e3377de to 35bd93f Compare July 30, 2026 02:49
The supported platform set (qoder, codex, claude, cursor, qwen, copilot)
is declared independently in CLI help, the agent-customize provider
registry, the session-analysis platform loader, the Harness report
platform gate, the asset-baseline provider gate, and the host adapter
matrix. Nothing kept those declarations in agreement, so adding or
dropping a host in one owner silently left the others stale.

Add test/support-declarations.test.mjs, which derives the declared list
from each surface (help output, registry keys, fail-closed error
messages, and documented module paths) through public routes only and
asserts set equality against one canonical list.

To let the contract test assert the report platform gate's whitelist
rather than only that it rejects unknown input, extract the reportPlatform
whitelist to a file-local REPORT_PLATFORMS constant and have its rejection
error name the full supported set, matching the session-analysis and
asset-baseline gates. This is the only runtime change; the error prefix
and error.code are unchanged. This closes roadmap P0 item A-06.

Story: roadmap.md TODO A-06
Spec: docs/specs/2026-07-28-a06-support-declaration-consistency.md
Test: node --test test/support-declarations.test.mjs (6 pass)

Co-authored-by: QoderAI (Qwen 3.8 Max) <qoder_ai@qoder.com>
@creayma-del
creayma-del force-pushed the test/support-declaration-consistency branch from 35bd93f to 0750bc8 Compare July 30, 2026 03:26
@phodal
phodal merged commit 3004766 into QoderAI:main Jul 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants