Skip to content

Commit d372c07

Browse files
committed
Run the extension-tagged conformance scenarios in CI as baselined known failures
The tasks-* scenarios are extension-tagged: they never match a --spec-version filter and sit on the harness's pending list, so none of the existing server legs can select them — they were silently absent from CI with no skip line and no waiver. server-sse-polling (pending, removed at 2026-07-28) was equally unreachable. Add a bare --suite all server leg, which selects every scenario shipped with the pinned harness, and baseline the nine runnable tasks-* scenarios in expected-failures.yml. The stale-entry rule turns the baseline into a forced burn-down list as tasks support lands. server-sse-polling turns out to already pass. tasks-status-notifications is deliberately not baselined: the harness skips it unconditionally, and a baseline entry for a scenario with no failing checks is itself flagged stale.
1 parent 8d0f928 commit d372c07

2 files changed

Lines changed: 36 additions & 1 deletion

File tree

.github/actions/conformance/expected-failures.yml

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,4 +12,23 @@
1212

1313
client: []
1414

15-
server: []
15+
server:
16+
# SEP-2663 (io.modelcontextprotocol/tasks): the SDK does not implement the
17+
# tasks extension yet. These extension-tagged scenarios are selected only by
18+
# the bare `--suite all` leg — extension scenarios never match a
19+
# --spec-version filter and the active/draft suites exclude them — so these
20+
# entries are inert for the other legs that read this file.
21+
#
22+
# `tasks-status-notifications` is intentionally NOT listed: the harness
23+
# skips it unconditionally (pending its rewrite against subscriptions/
24+
# listen), and a baseline entry for a scenario with no failing checks is
25+
# flagged stale.
26+
- tasks-lifecycle
27+
- tasks-capability-negotiation
28+
- tasks-wire-fields
29+
- tasks-request-state-removal
30+
- tasks-mrtr-input
31+
- tasks-request-headers
32+
- tasks-dispatch-and-envelope
33+
- tasks-required-task-error
34+
- tasks-mrtr-composition

.github/workflows/conformance.yml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,22 @@ jobs:
7575
--suite all
7676
--spec-version 2026-07-28
7777
--expected-failures ./.github/actions/conformance/expected-failures.2026-07-28.yml
78+
- name: Run server conformance (all suite, extension scenarios)
79+
# A bare `--suite all` (no --spec-version) selects every scenario
80+
# shipped with the pinned harness — including the extension-tagged
81+
# tasks-* scenarios and pending-listed ones like server-sse-polling,
82+
# which no other leg reaches (extension scenarios never match a
83+
# --spec-version filter, and the pending list keeps them out of the
84+
# active suite). Running the full set keeps unimplemented surfaces
85+
# visible as baselined known failures in expected-failures.yml instead
86+
# of silent exclusions, and stays robust to scenarios moving between
87+
# harness suite lists across pin bumps. `--suite pending` would cover
88+
# the same union slightly faster; the full set is preferred for the
89+
# self-contained run and for parity with typescript-sdk's CI.
90+
run: >-
91+
./.github/actions/conformance/run-server.sh
92+
--suite all
93+
--expected-failures ./.github/actions/conformance/expected-failures.yml
7894
7995
client-conformance:
8096
runs-on: ubuntu-latest

0 commit comments

Comments
 (0)