Skip to content

docs: 让 CI 一节与分片后的实际形状一致,并让 --shard 只有一个含义 - #162

Open
Sunrisepeak wants to merge 1 commit into
mainfrom
docs/ci-sharding-current
Open

docs: 让 CI 一节与分片后的实际形状一致,并让 --shard 只有一个含义#162
Sunrisepeak wants to merge 1 commit into
mainfrom
docs/ci-sharding-current

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

为什么

docs/repository-and-schema.md 的 CI 一节仍在描述三平台矩阵与每平台一条 mcpp test --workspace#155 已把它换成:select job 一次性决定整个计划、每平台分片数取自实测 runner 并发度、按实测耗时装箱分配、以及一个对成员排名的 timings job。这些都没有写进文档。

改了什么

文档(中英双份)

  • select job:选择性成员测试、分片仅用于全量、分片数=实测并发度(linux 3 / macos 1 / windows 2)及其理由(墙钟为 ceil(分片数/并发度) × 最慢分片,超出并发度的分片不减工作量只加固定开销)
  • plan_shards.lua 的判据:降序装箱 + 依赖亲和作平局判据 + 缺测值按中位数计价;与轮转法的实测对比(最慢分片 4158s → 3706s,离散度 47% → 15%);以及任何划分都突破不了的下界(grpc-module 1701s)
  • timings job:合并排名写入 run summary,member-timings artifact 择时手工刷新而非自动回写,以及为什么不自动提交
  • 手动触发的 cache 输入,以及全局包构建缓存为何回归(mcpp#344 的理由在 2026.8.3.4 被消除;保留旁路会让 linux 全量运行达到 2h30m 而超时,即根本得不到结果)
  • 测试前刷新已发布索引这一步
  • 新增「本地运行 workspace 成员」一节
  • 仓库布局补上 run_members.sh / plan_shards.lua / member-timings.tsv

tests/run_members.sh

--shard 此前自行用轮转法划分,而 CI 走 plan_shards.lua 的实测装箱 —— 同一个问题两套算法,而该脚本头部恰恰声称「CI 与本地是同一套」。现在 --shard 委托给 plan_shards.lua,本地第 N 片与 CI 第 N 片持有相同成员;PATH 上没有 lua 时退回轮转法,并在输出中明说。新增 --platform 选择读取 member-timings.tsv 的哪一列,缺省为宿主平台。

验证

$ bash tests/run_members.sh --all --shard 1/3
shard 1/3 (measured split for linux) -> 18 member(s)

首个成员 protobuf-protoc,与 lua5.4 tests/plan_shards.lua linux 1 3 直接输出一致。

无 lua 的回退路径也实测过(以只含必需工具、不含 lua 的 PATH 运行):

shard 1/3 (round-robin — no lua on PATH, so times were not consulted) -> 20 member(s)

改动 tests/*.sh 会触发全量运行,因此本 PR 的 CI 同时是对所文档化的分片机制本身的一次端到端验证。

…one thing

The CI section still described a three-platform matrix and a single
`mcpp test --workspace` per platform. #155 replaced that with a `select` job
that decides the whole plan once, per-platform shard counts taken from measured
runner concurrency, measured-time bin packing, and a `timings` job that ranks
members. None of it was written down.

Also documented: the `cache` dispatch input, why the global package build cache
came back (mcpp#344's reason was removed in 2026.8.3.4, and the bypass made the
full linux run exceed its timeout), and the pre-test index refresh.

`run_members.sh --shard` computed its own split by round-robin while CI used
`plan_shards.lua`'s measured packing — two algorithms answering one question,
in a script whose header claims local and CI measure the same thing. --shard
now delegates to plan_shards.lua, so shard N locally holds the members shard N
holds in CI; round-robin stays as the fallback where lua is absent and says so
in its output. --platform selects which column of member-timings.tsv to read
and defaults to the host.

Both language versions updated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant