Skip to content

Commit cf25635

Browse files
committed
ci: add a weekly newest-dependencies canary
Users installing `mcp` get the newest release of every dependency the day it ships, but PR CI only tests uv.lock and the floors, so an upstream release that breaks the SDK is currently noticed by users first (median 11 days across past incidents). Running "highest" on every PR was tried and removed (#1869) because a half-uploaded release turns unrelated PRs red; the weekly lock-bump PR that replaced it (#1874) never got merged. This adds a scheduled workflow instead. Every Monday it re-resolves the runtime closure of mcp[cli,rich] to the newest versions the specifiers allow (test tooling stays at uv.lock, releases younger than a day are ignored), runs the suite on ubuntu 3.10/3.14 and windows 3.14, and keeps a single tracking issue in sync: opened and assigned when newest-allowed breaks, refreshed while it stays broken, closed once it passes again. Each cell re-runs failures serially and once more with deprecation warnings demoted, so the issue says whether users are actually broken or a dependency merely deprecated something. The report lists what changed since the last green run, what is held below latest by someone else's cap, and the exact command to reproduce the resolution. It never runs on pull requests and never opens a PR adding a ceiling; the issue carries the runbook and a maintainer decides. No-Verification-Needed: CI-only change (workflow + scripts/ci); scripts exercised locally, workflow verified by a branch run
1 parent 5285e93 commit cf25635

5 files changed

Lines changed: 793 additions & 0 deletions

File tree

Lines changed: 316 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,316 @@
1+
name: Dependency canary
2+
3+
# Weekly: re-resolve the runtime dependencies of `mcp[cli,rich]` to the newest
4+
# versions our (floors-only) specifiers allow, ignoring uv.lock, run the test
5+
# suite against them, and keep ONE tracking issue in sync with the result —
6+
# opened (and assigned) when newest-allowed breaks, refreshed weekly while it
7+
# stays broken, closed automatically once it passes again.
8+
#
9+
# Why this exists: users who `pip install mcp` get the newest release of every
10+
# dependency the day it ships, while PR CI only ever sees uv.lock (`locked`) and
11+
# the floors (`lowest-direct`). This is deliberately NOT part of PR CI, so an
12+
# upstream release can never turn an unrelated PR red; the price is up to a
13+
# week of latency, which the incident history says is fine.
14+
#
15+
# What it does not do, on purpose:
16+
# - open a PR adding a ceiling. A cap only helps once released, resolvers
17+
# route around retroactive caps by picking an older uncapped mcp, and the
18+
# bot cannot tell which package (or interaction) is at fault. The issue's
19+
# "What to do" section is the runbook; a human decides.
20+
# - float test tooling (pytest, ruff, pyright, coverage, ...). Those stay at
21+
# uv.lock so a pytest major cannot masquerade as an SDK break; Dependabot
22+
# owns moving them.
23+
# - test pre-releases on the schedule. `workflow_dispatch` with
24+
# `prerelease: true` does that on demand and never files an issue.
25+
# - bisect. The issue lists what changed since the last green run (usually
26+
# one to three packages) and the one-line command to pin a suspect back.
27+
#
28+
# Known blind spots: Python 3.11-3.13 and macOS are not run; runtime deps that a
29+
# *dev* dependency caps (e.g. logfire pins opentelemetry-sdk, which pins
30+
# opentelemetry-api) cannot reach their newest release here — the issue's "Not
31+
# tested at their newest release" section lists them each run. Dependency
32+
# groups outside `default-groups` (translate, codegen) are stripped before
33+
# resolving so their caps (anthropic: pydantic<3) do not apply.
34+
#
35+
# Notifications: assignees get the issue traffic. GitHub additionally e-mails
36+
# scheduled-run failures only to whoever last edited the `cron:` line below.
37+
38+
on:
39+
schedule:
40+
- cron: "23 5 * * 1" # Mondays 05:23 UTC
41+
workflow_dispatch:
42+
inputs:
43+
prerelease:
44+
description: "Also consider pre-releases (investigative run; never files an issue)"
45+
type: boolean
46+
default: false
47+
file-issue:
48+
description: "Create/update/close the tracking issue exactly as a scheduled run would"
49+
type: boolean
50+
default: false
51+
# TEMPORARY while this workflow is under review: exercise it on the PR branch.
52+
# Report-only (push runs never touch issues). Remove before merging.
53+
push:
54+
branches: ["ci/dependency-canary"]
55+
56+
permissions: {}
57+
58+
concurrency:
59+
group: ${{ github.workflow }}-${{ github.ref }}
60+
cancel-in-progress: true
61+
62+
env:
63+
COLUMNS: 150
64+
UV_VERSION: "0.11.33"
65+
# Releases younger than this are invisible to the run: skips half-uploaded
66+
# releases (the ruff 0.14.12 incident that got the per-PR "highest" leg
67+
# removed in #1869) and same-day yanks. A weekly job loses nothing by it.
68+
CANARY_LAG: "24 hours"
69+
CANARY_LABEL: dependency-canary
70+
CANARY_ASSIGNEES: "maxisbey,Kludex"
71+
72+
jobs:
73+
resolve:
74+
# Don't run the schedule on forks.
75+
if: github.repository == 'modelcontextprotocol/python-sdk'
76+
runs-on: ubuntu-latest
77+
timeout-minutes: 15
78+
permissions:
79+
contents: read
80+
actions: read # `gh run list`: find the last green scheduled run to diff against
81+
outputs:
82+
cutoff: ${{ steps.cutoffs.outputs.cutoff }}
83+
steps:
84+
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
85+
with:
86+
persist-credentials: false
87+
88+
# setup-uv's manifest fetch is a single request with a hard 5s timeout
89+
# (astral-sh/setup-uv#869); retry once. Drop when upstream adds a retry.
90+
- name: Install uv
91+
id: setup-uv
92+
continue-on-error: true
93+
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
94+
with:
95+
enable-cache: false
96+
version: ${{ env.UV_VERSION }}
97+
98+
- name: Install uv (retry)
99+
if: steps.setup-uv.outcome == 'failure'
100+
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
101+
with:
102+
enable-cache: false
103+
version: ${{ env.UV_VERSION }}
104+
105+
- name: Compute cutoffs
106+
id: cutoffs
107+
env:
108+
GH_TOKEN: ${{ github.token }}
109+
run: |
110+
mkdir -p canary-resolve
111+
cutoff=$(date -u -d "-$CANARY_LAG" +%Y-%m-%dT%H:%M:%SZ)
112+
echo "cutoff=$cutoff" >>"$GITHUB_OUTPUT"
113+
echo "$cutoff" >canary-resolve/cutoff.txt
114+
uv self version >canary-resolve/uv-version.txt
115+
# Baseline = what the last green *scheduled* run saw (its start time minus the same lag).
116+
last_green=$(gh run list --repo "$GITHUB_REPOSITORY" --workflow dependency-canary.yml \
117+
--branch main --event schedule --status success --limit 1 --json startedAt --jq '.[0].startedAt // empty')
118+
if [ -n "$last_green" ]; then
119+
date -u -d "$last_green -$CANARY_LAG" +%Y-%m-%dT%H:%M:%SZ >canary-resolve/baseline.txt
120+
else
121+
: >canary-resolve/baseline.txt
122+
fi
123+
echo "cutoff=$cutoff baseline=$(cat canary-resolve/baseline.txt)"
124+
125+
- name: Work out what to float
126+
run: |
127+
# The runtime closure of mcp[cli,rich] as currently locked: exactly the set
128+
# `pip install "mcp[cli,rich]"` pulls in. New transitive deps that a newer
129+
# release introduces have no lock entry and so resolve to newest anyway.
130+
uv export --frozen --no-default-groups --all-extras --no-emit-workspace \
131+
--no-hashes --no-header --no-annotate | sed -E 's/[=; @].*//' | sort -u >canary-resolve/closure.txt
132+
echo "Floating $(wc -l <canary-resolve/closure.txt) packages:"; tr '\n' ' ' <canary-resolve/closure.txt; echo
133+
# Dependency groups that `uv sync` does not install still constrain the
134+
# resolution (uv.lock is universal). Strip the non-default ones so e.g. the
135+
# translate group's `anthropic` cannot hold pydantic below a new major.
136+
python3 - <<'EOF' >canary-resolve/strip.sh
137+
import re, tomllib
138+
project = tomllib.load(open("pyproject.toml", "rb"))
139+
keep = set(project.get("tool", {}).get("uv", {}).get("default-groups", []))
140+
for group, deps in project.get("dependency-groups", {}).items():
141+
names = [re.match(r"[A-Za-z0-9._-]+", d).group(0) for d in deps if isinstance(d, str)]
142+
if group not in keep and names:
143+
print("uv remove --frozen --group", group, *names)
144+
EOF
145+
cat canary-resolve/strip.sh
146+
bash -e canary-resolve/strip.sh
147+
148+
- name: Resolve newest allowed versions
149+
env:
150+
PRERELEASE: ${{ inputs.prerelease && 'allow' || '' }}
151+
run: |
152+
set -o pipefail
153+
args=(--exclude-newer "$(cat canary-resolve/cutoff.txt)")
154+
if [ -n "$PRERELEASE" ]; then args+=(--prerelease "$PRERELEASE"); fi
155+
while read -r pkg; do args+=(-P "$pkg"); done <canary-resolve/closure.txt
156+
cp uv.lock canary-resolve/committed.lock
157+
158+
# Reconstruct what the last green run resolved (same command, its cutoff) so the
159+
# report can list only what moved since. Best effort: a failure here just drops that table.
160+
baseline=$(cat canary-resolve/baseline.txt)
161+
if [ -n "$baseline" ]; then
162+
echo "::group::Baseline resolution as of the last green run ($baseline)"
163+
base_args=("${args[@]}")
164+
base_args[1]=$baseline
165+
if uv lock "${base_args[@]}" 2>&1 | tee canary-resolve/baseline.log; then
166+
cp uv.lock canary-resolve/baseline.lock
167+
else
168+
echo "::warning::could not re-resolve the last-green baseline; the report will only diff against uv.lock"
169+
: >canary-resolve/baseline.txt
170+
fi
171+
cp canary-resolve/committed.lock uv.lock
172+
echo "::endgroup::"
173+
fi
174+
175+
uv lock "${args[@]}" 2>&1 | tee canary-resolve/lock.log
176+
cp uv.lock canary-resolve/uv.lock
177+
178+
- name: Summarise what moved
179+
run: |
180+
python3 scripts/ci/canary_lock_diff.py canary-resolve/committed.lock uv.lock \
181+
--old-label "uv.lock" --new-label "this run" --suspects canary-resolve/suspects-vs-lock.txt >canary-resolve/vs-lock.md
182+
if [ -f canary-resolve/baseline.lock ]; then
183+
python3 scripts/ci/canary_lock_diff.py canary-resolve/baseline.lock uv.lock \
184+
--old-label "last green" --new-label "this run" --suspects canary-resolve/suspects-since-green.txt >canary-resolve/since-green.md
185+
fi
186+
# Direct runtime deps that could not reach their newest release (capped by something else in the resolution).
187+
uv tree --frozen --outdated --depth 1 --package mcp >canary-resolve/tree.txt 2>/dev/null || true
188+
{
189+
echo "| Package | Resolved | Latest |"
190+
echo "| --- | --- | --- |"
191+
sed -nE 's/^[^A-Za-z0-9]*([A-Za-z0-9._-]+)(\[[^]]*\])? v([^ ]+)( \(extra: [^)]*\))? \(latest: v([^)]+)\)$/| \1 | \3 | \5 |/p' canary-resolve/tree.txt
192+
} >canary-resolve/held-back.md
193+
if [ "$(wc -l <canary-resolve/held-back.md)" -le 2 ]; then : >canary-resolve/held-back.md; fi
194+
{
195+
echo "## Resolution (cutoff $(cat canary-resolve/cutoff.txt))"
196+
echo
197+
if [ -s canary-resolve/since-green.md ]; then echo "### Since last green ($(cat canary-resolve/baseline.txt))"; cat canary-resolve/since-green.md; echo; fi
198+
echo "### vs uv.lock"; cat canary-resolve/vs-lock.md; echo
199+
if [ -s canary-resolve/held-back.md ]; then echo "### Held back below latest"; cat canary-resolve/held-back.md; fi
200+
} >>"$GITHUB_STEP_SUMMARY"
201+
202+
- name: Upload resolution
203+
if: always()
204+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
205+
with:
206+
name: canary-resolve
207+
path: canary-resolve/
208+
retention-days: 90
209+
if-no-files-found: error
210+
211+
test:
212+
name: test (${{ matrix.cell }})
213+
needs: resolve
214+
runs-on: ${{ matrix.os }}
215+
timeout-minutes: 20
216+
permissions:
217+
contents: read
218+
strategy:
219+
fail-fast: false
220+
matrix:
221+
include:
222+
# Oldest and newest supported Python bracket the marker forks in the
223+
# lock (deps drop 3.10 first; 3.14 gets wheels last). Windows/newest is
224+
# where a fresh release most often lacks a wheel, and pywin32 lives there.
225+
- { cell: ubuntu-3.10, os: ubuntu-latest, python: "3.10" }
226+
- {
227+
cell: ubuntu-3.14,
228+
os: ubuntu-latest,
229+
python: "3.14",
230+
smoke: "1",
231+
pyright: "1",
232+
}
233+
- { cell: windows-3.14, os: windows-latest, python: "3.14" }
234+
steps:
235+
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
236+
with:
237+
persist-credentials: false
238+
239+
- name: Install uv
240+
id: setup-uv
241+
continue-on-error: true
242+
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
243+
with:
244+
enable-cache: false
245+
version: ${{ env.UV_VERSION }}
246+
247+
- name: Install uv (retry)
248+
if: steps.setup-uv.outcome == 'failure'
249+
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
250+
with:
251+
enable-cache: false
252+
version: ${{ env.UV_VERSION }}
253+
254+
- name: Fetch the resolved lock
255+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
256+
with:
257+
name: canary-resolve
258+
path: canary-resolve
259+
260+
- name: Install and test
261+
shell: bash
262+
env:
263+
CANARY_CELL: ${{ matrix.cell }}
264+
CANARY_PYTHON: ${{ matrix.python }}
265+
CANARY_PYRIGHT: ${{ matrix.pyright }}
266+
# Same switches as PR CI: real stdio/uvicorn subprocess smoke tests on one
267+
# cell, and PEP 597 EncodingWarnings surfaced (as an env var so xdist workers inherit it).
268+
MCP_EXAMPLES_SMOKE: ${{ matrix.smoke }}
269+
PYTHONWARNDEFAULTENCODING: "1"
270+
run: |
271+
cp canary-resolve/uv.lock uv.lock
272+
# The lock was resolved with these groups stripped; keep pyproject consistent so --frozen holds.
273+
bash -e canary-resolve/strip.sh
274+
bash scripts/ci/canary_cell.sh
275+
276+
- name: Upload cell result
277+
if: always()
278+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
279+
with:
280+
name: canary-cell-${{ matrix.cell }}
281+
path: |
282+
canary-out/status
283+
canary-out/cell.md
284+
retention-days: 30
285+
if-no-files-found: error
286+
287+
report:
288+
needs: [resolve, test]
289+
# always(): a red resolve/test job is exactly when this must run.
290+
if: always() && github.repository == 'modelcontextprotocol/python-sdk' && needs.resolve.result != 'cancelled' && needs.resolve.result != 'skipped'
291+
runs-on: ubuntu-latest
292+
timeout-minutes: 10
293+
permissions:
294+
contents: read # checkout, for scripts/ci/canary_report.sh
295+
issues: write # open / refresh / close the tracking issue
296+
steps:
297+
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
298+
with:
299+
persist-credentials: false
300+
sparse-checkout: scripts/ci
301+
302+
- name: Collect results
303+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
304+
with:
305+
pattern: canary-*
306+
path: artifacts
307+
308+
- name: Report
309+
env:
310+
GH_TOKEN: ${{ github.token }}
311+
CANARY_ARTIFACTS: artifacts
312+
CANARY_RESOLVE_RESULT: ${{ needs.resolve.result }}
313+
CANARY_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
314+
# Only the schedule (or an explicit dispatch asking for it) touches issues, and never a pre-release run.
315+
CANARY_FILE_ISSUES: ${{ !inputs.prerelease && (github.event_name == 'schedule' || inputs.file-issue) && 'true' || 'false' }}
316+
run: bash scripts/ci/canary_report.sh

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -183,3 +183,8 @@ results/
183183

184184
# conformance CI local runs
185185
conformance-results/
186+
187+
# dependency canary local runs (scripts/ci/canary_*.sh)
188+
canary-out/
189+
canary-resolve/
190+
canary-body.md

0 commit comments

Comments
 (0)