Skip to content

docs: 让 CI 一节与分片后的实际形状一致,并让 --shard 只有一个含义 #162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Sunrisepeak merged 1 commit into from
Aug 6, 2026
+182 −30
Merged

docs: 让 CI 一节与分片后的实际形状一致,并让 --shard 只有一个含义 #162

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 74 additions & 10 deletions docs/repository-and-schema.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,15 @@ tests/examples/<member>/ one test project per library (a workspace member; <
tests/check_mirror_urls.lua lint: GLOBAL+CN table completeness, and that CN points at mcpp-res
tests/check_package_name.lua lint: identity shape (name is a single atomic segment, hierarchy belongs to namespace)
tests/list_cn_urls.lua extracts the CN urls for mirror-cn-reachable
tests/run_members.sh runs workspace members one at a time and times each. The entry point used both by CI
and locally; see "Running workspace members locally" below
tests/plan_shards.lua assigns members to shards from measured times. Called once by the `select` job, and by
run_members.sh --shard, so both produce the same split
tests/member-timings.tsv the measured per-member wall-clock plan_shards.lua reads. Refreshed deliberately from
the member-timings artifact rather than written back on every run
README.md index overview and contribution entry point (README.zh-CN.md is the Chinese version)
.github/workflows/validate.yml CI: lint / mirror-cn-reachable / workspace (a 3-platform matrix)
.github/workflows/validate.yml CI: lint / mirror-cn-reachable / select / workspace (sharded per platform on a full
run) / timings
.agents/docs/<date>-*.md the design-document convention
docs/ contributor reference documentation (this directory; docs/zh/ holds the Chinese version)
tools/gtc the gitcode CLI, see cn-mirror.md
Expand Down Expand Up @@ -150,28 +157,65 @@ locally with `mcpp xpkg parse pkgs/<x>/<name>.lua`.
## CI behavior (validate.yml)

- Triggers: a PR (touching `pkgs/**/*.lua`, `tests/**`, either README, `mcpp.toml`, `index.toml` or this workflow),
a push to main, the nightly cron, and manual dispatch.
a push to main, the nightly cron, and manual dispatch. Dispatch takes a `cache` input — `global`, the default, or
`local`. Under `local` every member rebuilds every dependency from scratch, which isolates a member's own cost from
what it inherited from the members that ran before it; that is the condition per-member times should be compared
under, and it is also far slower.
- `env.MCPP_VERSION` is the mcpp version every job uses; local verification should match it.
- `lint` (always runs): lua syntax via `loadfile(f,'t')`; `spec=`/`name=`/`xpm=` must be present; leading-v versions
are rejected; runs `check_mirror_urls.lua`; runs `check_package_name.lua` (identity shape, see "Package identity"
above); then runs `mcpp xpkg parse` over every descriptor with the mcpp version CI pins (strict — an unknown key
fails). `xpkg parse` in mcpp >= 0.0.106 enforces the identity shape itself, which makes the lua lint an earlier and
cheaper redundant gate.
- `mirror-cn-reachable` (always runs): `curl`s each CN url; all must return 200.
- `workspace (linux|macos|windows)`: the whole test surface is one mcpp workspace and the **only build/run channel** —
there is no shell-driven exception (the public module packages imgui/ffmpeg/opencv/tinyhttps are ordinary members
too, resolving from the checkout through a member-level `[indices] default = { path = "../../.." }`,
mcpp >= 0.0.97; members consuming `compat` inherit the root-level declaration, see "Index redirection" above).
- `select`: decides the entire plan once and emits it to the runners as data. Three questions are answered here
rather than on each runner — which members run, how many shards each platform gets, and which members land on
which shard.
- Selective member testing: on a PR, `git diff` maps changed files to the affected members
(`pkgs/<x>/<lib>.lua` → members whose mcpp.toml references `<lib>`; `tests/examples/<m>/**` → member `<m>`), and
only those run through `mcpp test -p <member>`; global changes — the workflow itself, the non-member part of the
workspace manifest, `tools/` and so on — go to a full `mcpp test --workspace`. push/nightly/dispatch are always
full runs.
(`pkgs/<x>/<lib>.lua` → members whose mcpp.toml references `<lib>`; `tests/examples/<m>/**` → member `<m>`).
A change that can affect everything selects the full workspace instead: a non-PR event, this workflow file, a
non-member edit to the workspace manifest, or a shared test script. Documentation-only and `tools/`-only changes
select nothing.
- Sharding applies to full runs only — a selective run is one job per platform. The shard count per platform is
that platform's **measured runner concurrency** (linux 3, macos 1, windows 2) rather than a round number.
Wall-clock is `ceil(shards / concurrency) × slowest-shard`, so shards beyond the concurrency remove no work and
each still pays its own checkout, mcpp download and cache restore. At concurrency 1, splitting macOS is strictly
slower than not splitting it. Re-measure with:
`gh api repos/<owner>/<repo>/actions/runs/<id>/jobs --paginate --jq '[.jobs[]|select(.status=="in_progress")]|length'`
- The assignment itself comes from `tests/plan_shards.lua`. It reads `tests/member-timings.tsv` and packs
longest-first onto the least-loaded shard; ties break toward the shard already holding members with overlapping
dependencies, because shards share no build cache and a dependency landing on two shards is built twice. A
member with no recorded time is charged the median, so a newly added member is assumed neither free nor huge.
Against round-robin on the real workspace the slowest linux shard falls from 4158s to 3706s and the spread from
47% to 15%. One floor no split can beat remains: the single slowest member, `grpc-module` at 1701s.
- Planning runs here, on linux, because it needs lua: windows has no apt or brew, and Homebrew installs `lua`
rather than `lua5.4`.
- `workspace (<platform> <shard>/<count>)`: the whole test surface is one mcpp workspace and the **only build/run
channel** — there is no shell-driven exception (the public module packages imgui/ffmpeg/opencv/tinyhttps are
ordinary members too, resolving from the checkout through a member-level `[indices] default = { path = "../../.." }`,
mcpp >= 0.0.97; members consuming `compat` inherit the root-level declaration, see "Index redirection" above). The
shard suffix appears only where the platform is actually split.
- Members run through `tests/run_members.sh`, the same script used locally. A timing table that exists only in CI
cannot be consulted while deciding what to optimise, and a local harness that differs from CI measures something
else.
- The package build cache is global, which is mcpp's default. The step formerly set `MCPP_BUILD_CACHE: local` to
work around mcpp#344, in which one cache entry could hold two object layouts; mcpp 2026.8.3.4 keyed the cache per
package with the consumer-dependent layout included, so the reason no longer holds. Keeping the bypass was
expensive: under `local`, 59 members that largely share abseil, protobuf and opencv rebuilt each of them from
scratch, and a full linux run reached 2h30m — past the timeout, so it produced no result at all.
- The `~/.mcpp/registry` cache carries the toolchains and the already-built compat packages, so a repeat run is
incremental and fast. Its key is computed once, in a step of its own, from `git ls-files -s` over the tracked
inputs — never with `hashFiles()`, which globs the working tree and would re-hash the multi-GB build output under
`tests/examples/*/target` when actions/cache re-evaluates the key in its post (save) step (that blew past the
runner's 120s template-evaluation cap on windows).
- The published index is refreshed before testing. Most members resolve everything from the checkout, but a member
redirecting a namespace other than `compat` takes the rest from the published index, whose snapshot is whatever
the pinned mcpp release vendored — older than main by construction, and never moved by anything else in the run.
- `timings`: merges the per-shard timing artifacts into one ranking per platform in the run summary, and publishes
the combined table as the `member-timings` artifact. Sharding otherwise hides where the time goes, since each
runner reports only its own slice. The table is not committed automatically: a number that rewrites itself on
every run makes every diff noisy and silently absorbs a one-off slow runner. Refresh `tests/member-timings.tsv`
from that artifact when the numbers have actually moved.

## Reproducing lint locally (equivalent to the CI lint job)

Expand All @@ -187,6 +231,26 @@ done
[ $fail -eq 0 ] && echo "ALL LINT PASS"
```

## Running workspace members locally

`tests/run_members.sh` is the entry point CI uses, so a local run measures the same thing under the same split:

```bash
bash tests/run_members.sh --all # every member
bash tests/run_members.sh opencv-module protobuf # named members
bash tests/run_members.sh --all --shard 1/3 # exactly what CI's linux shard 1 runs
bash tests/run_members.sh --all --shard 0/2 --platform windows
bash tests/run_members.sh --all --cache local # bypass the package build cache
```

Shard indices are 0-based, and `--shard` delegates to `tests/plan_shards.lua` — the script the `select` job calls —
so shard N locally holds the members shard N holds in CI. Without lua on `PATH` it falls back to round-robin and
says so. `--platform` chooses which column of `tests/member-timings.tsv` to read and defaults to the host.

`MCPP` selects the binary (default: `mcpp` on `PATH`); `MCPP_TIMINGS` names a file to append
`<seconds>\t<member>\t<ok|FAIL>` rows to. The exit status is non-zero if any member failed, and the timing table
prints either way — a run worth diagnosing is exactly the one where the times matter.

## After the merge

`publish-artifact.yml` republishes the mcpp-index artifact and moves the pointer automatically once the change lands
Expand Down
69 changes: 61 additions & 8 deletions docs/zh/repository-and-schema.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,13 @@ tests/examples/<member>/ 每库测试工程(workspace 成员;<member> 为包
tests/check_mirror_urls.lua lint:GLOBAL+CN 表完整性,以及 CN 指向 mcpp-res
tests/check_package_name.lua lint:身份形态(name 为单一原子段,层级归 namespace)
tests/list_cn_urls.lua 抽取 CN url,供 mirror-cn-reachable 使用
tests/run_members.sh 逐个运行 workspace 成员并计时。CI 与本地共用的入口,见下文「本地运行 workspace 成员」
tests/plan_shards.lua 依实测耗时将成员分配到各分片。由 `select` job 调用一次,`run_members.sh --shard`
亦调用同一脚本,故两处得到同一划分
tests/member-timings.tsv plan_shards.lua 读取的逐成员实测墙钟。由 member-timings artifact 择时手工刷新,
而非每次运行自动回写
README.md 索引说明与贡献入口(英文;中文版为 README.zh-CN.md)
.github/workflows/validate.yml CI:lint / mirror-cn-reachable / workspace(3 平台矩阵)
.github/workflows/validate.yml CI:lint / mirror-cn-reachable / select / workspace(全量时按平台分片)/ timings
.agents/docs/<date>-*.md 设计文档惯例
docs/ 贡献者参考文档(英文);docs/zh/ 为中文版(本目录)
tools/gtc gitcode CLI,见 cn-mirror.md
Expand Down Expand Up @@ -115,25 +120,54 @@ mcpp 跑 `xpkg parse`(strict:未知键即失败),所以需要更新文法/键的
## CI 行为(validate.yml)

- 触发条件:PR(改动 `pkgs/**/*.lua`、`tests/**`、两份 README 之一、`mcpp.toml`、`index.toml` 或本 workflow)、
push 至 main、nightly cron、手动触发。
push 至 main、nightly cron、手动触发。手动触发接受 `cache` 输入 —— `global`(默认)或 `local`。`local` 下
每个成员均从零重建其全部依赖,从而把成员自身的开销与它从先前成员处继承到的部分隔离开;逐成员耗时应在此
条件下比较,代价是显著更慢。
- `env.MCPP_VERSION` 为全部 job 使用的 mcpp 版本,本地验证应与之对齐。
- `lint`(始终运行):lua 语法 `loadfile(f,'t')`;须含 `spec=`/`name=`/`xpm=`;禁止前导 v 版本;执行
`check_mirror_urls.lua`;执行 `check_package_name.lua`(身份形态,见上文「包身份」);再用 CI pin 的
mcpp 对每个描述符跑 `mcpp xpkg parse`(strict,未知键即失败)。mcpp ≥ 0.0.106 的 `xpkg parse` 自身
也强制身份形态,lua lint 因此是更早、更便宜的冗余闸门。
- `mirror-cn-reachable`(始终运行):逐个 `curl` CN url,均须返回 200。
- `workspace (linux|macos|windows)`:整个测试面就是一个 mcpp workspace,**唯一的构建/运行通道**——
- `select`:一次性决定整个计划,并以数据形式下发给各 runner。三个问题在此处而非各 runner 上回答 ——
哪些成员要跑、每个平台分几片、哪些成员落在哪一片。
- 选择性成员测试:PR 时由 `git diff` 将改动文件映射到受影响成员
(`pkgs/<x>/<lib>.lua` → mcpp.toml 引用 `<lib>` 的成员;`tests/examples/<m>/**` → 成员 `<m>`)。
可能影响全部成员的改动则选中整个 workspace:非 PR 事件、本 workflow 文件、workspace 清单的非成员
部分、共享测试脚本。仅文档与仅 `tools/` 的改动不选中任何成员。
- 分片仅用于全量运行,选择性运行为每平台一个 job。每平台的分片数取该平台**实测的 runner 并发度**
(linux 3、macos 1、windows 2),而非取整数。墙钟为 `ceil(分片数 / 并发度) × 最慢分片`,故超出并发度
的分片不减少任何工作量,却各自仍要付出 checkout、mcpp 下载与缓存恢复的固定开销;并发度为 1 时,
对 macOS 分片严格慢于不分片。重新测量:
`gh api repos/<owner>/<repo>/actions/runs/<id>/jobs --paginate --jq '[.jobs[]|select(.status=="in_progress")]|length'`
- 分配本身由 `tests/plan_shards.lua` 给出:读取 `tests/member-timings.tsv`,按耗时降序依次放入当前负载
最小的分片;负载接近时优先选择已含有共同依赖成员的分片 —— 分片之间不共享构建缓存,同一依赖落在两片
上就要构建两次。无实测记录的成员按中位数计价,故新增成员既不被假定为零成本,也不被假定为极重。
在真实 workspace 上与轮转法相比,linux 最慢分片由 4158s 降至 3706s,离散度由 47% 降至 15%。有一条
任何划分都无法突破的下界:最慢的单个成员,`grpc-module` 为 1701s。
- 规划在此 linux job 内进行,因为它需要 lua:windows 上既无 apt 亦无 brew,而 Homebrew 安装的是 `lua`
而非 `lua5.4`。
- `workspace (<平台> <分片>/<总数>)`:整个测试面就是一个 mcpp workspace,**唯一的构建/运行通道**——
没有任何 shell 驱动的例外(公开模块包 imgui/ffmpeg/opencv/tinyhttps 也是普通成员,经成员级
`[indices] default = { path = "../../.." }` 从 checkout 解析,mcpp ≥ 0.0.97;消费 `compat` 的
成员则继承根级声明,见上文「索引重定向」)。
- 选择性成员测试:PR 时由 `git diff` 将改动文件映射到受影响成员
(`pkgs/<x>/<lib>.lua` → mcpp.toml 引用 `<lib>` 的成员;`tests/examples/<m>/**` → 成员 `<m>`),
仅 `mcpp test -p <member>` 这些成员;workflow 本身、workspace 清单非成员部分、`tools/` 等
全局性改动 → `mcpp test --workspace` 全量。push/nightly/dispatch 恒为全量。
成员则继承根级声明,见上文「索引重定向」)。分片后缀仅在该平台确实被拆分时出现。
- 成员经 `tests/run_members.sh` 运行,该脚本亦即本地入口。只存在于 CI 的耗时表无法在决定优化对象时被
参考,而与 CI 不同的本地测量装置度量的是另一回事。
- 包构建缓存取全局,即 mcpp 的默认值。该步骤此前设 `MCPP_BUILD_CACHE: local` 以规避 mcpp#344 —— 同一
缓存条目可能持有两种对象布局;mcpp 2026.8.3.4 起缓存按包计键并纳入随消费方而变的布局,该理由已不成立。
保留该旁路的代价是实际的:`local` 之下,59 个大量共享 abseil、protobuf、opencv 的成员会各自从零重建
这些依赖,linux 全量运行达到 2h30m —— 超出超时上限,因而根本得不到结果。
- `~/.mcpp/registry` 缓存携带工具链与已构建的 compat 包,重复运行增量很快。其 key 由独立步骤经
`git ls-files -s` 就已跟踪的输入算出一次,而**不用** `hashFiles()`:后者按工作树 glob,而
actions/cache 会在 post(save)步骤重新求值 key,届时 `tests/examples/*/target` 下的数 GB 构建产物
也会被一并哈希 —— windows 上曾因此撞破 runner 的 120 秒模板求值上限。
- 测试前刷新已发布索引。多数成员的全部依赖都从本 checkout 解析,但重定向了 `compat` 之外命名空间的成员
会从已发布索引取其余部分,而该快照是所 pin 的 mcpp 发行版随附的那一份 —— 按构造即早于 main,且运行中
没有任何其他环节会推进它。
- `timings`:将各分片的耗时 artifact 合并为每平台一份排名写入 run summary,并以 `member-timings` artifact
发布合并后的表。否则分片会掩盖时间的去向 —— 每个 runner 只报告自己那一片。该表不自动提交:每次运行都
改写自身的数字会让每份 diff 都充满噪声,并会悄悄吸收一次偶发的慢 runner。数字确实发生变化时,再据该
artifact 刷新 `tests/member-timings.tsv`。

## 本地 lint 复现(等价于 CI lint job)

Expand All @@ -149,6 +183,25 @@ done
[ $fail -eq 0 ] && echo "ALL LINT PASS"
```

## 本地运行 workspace 成员

`tests/run_members.sh` 即 CI 使用的入口,故本地运行度量的是同一对象、同一划分:

```bash
bash tests/run_members.sh --all # 全部成员
bash tests/run_members.sh opencv-module protobuf # 指定成员
bash tests/run_members.sh --all --shard 1/3 # 与 CI 的 linux 分片 1 完全一致
bash tests/run_members.sh --all --shard 0/2 --platform windows
bash tests/run_members.sh --all --cache local # 绕过包构建缓存
```

分片下标自 0 起。`--shard` 委托给 `tests/plan_shards.lua`,即 `select` job 所调用的同一脚本,因此本地的第 N
片与 CI 的第 N 片持有相同成员;`PATH` 上没有 lua 时退回轮转法,并在输出中说明。`--platform` 选择读取
`tests/member-timings.tsv` 的哪一列,缺省为宿主平台。

`MCPP` 指定所用二进制(缺省为 `PATH` 上的 `mcpp`);`MCPP_TIMINGS` 指定追加 `<秒>\t<成员>\t<ok|FAIL>` 行的
文件。任一成员失败则退出码非零,而耗时表两种情况下均会打印 —— 值得诊断的运行恰恰就是耗时重要的那一次。

## 合并后

`publish-artifact.yml` 在合并至 `main` 后自动重新发布 mcpp-index artifact 并移动指针,无需发布新的 mcpp 版本。
Expand Down
Loading
Loading