docs: 让 CI 一节与分片后的实际形状一致,并让 --shard 只有一个含义 - #162
Open
Sunrisepeak wants to merge 1 commit into
Open
Conversation
…one thing The CI section still described a three-platform matrix and a single `mcpp test --workspace` per platform. #155 replaced that with a `select` job that decides the whole plan once, per-platform shard counts taken from measured runner concurrency, measured-time bin packing, and a `timings` job that ranks members. None of it was written down. Also documented: the `cache` dispatch input, why the global package build cache came back (mcpp#344's reason was removed in 2026.8.3.4, and the bypass made the full linux run exceed its timeout), and the pre-test index refresh. `run_members.sh --shard` computed its own split by round-robin while CI used `plan_shards.lua`'s measured packing — two algorithms answering one question, in a script whose header claims local and CI measure the same thing. --shard now delegates to plan_shards.lua, so shard N locally holds the members shard N holds in CI; round-robin stays as the fallback where lua is absent and says so in its output. --platform selects which column of member-timings.tsv to read and defaults to the host. Both language versions updated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
为什么
docs/repository-and-schema.md的 CI 一节仍在描述三平台矩阵与每平台一条mcpp test --workspace。#155 已把它换成:selectjob 一次性决定整个计划、每平台分片数取自实测 runner 并发度、按实测耗时装箱分配、以及一个对成员排名的timingsjob。这些都没有写进文档。改了什么
文档(中英双份)
selectjob:选择性成员测试、分片仅用于全量、分片数=实测并发度(linux 3 / macos 1 / windows 2)及其理由(墙钟为ceil(分片数/并发度) × 最慢分片,超出并发度的分片不减工作量只加固定开销)plan_shards.lua的判据:降序装箱 + 依赖亲和作平局判据 + 缺测值按中位数计价;与轮转法的实测对比(最慢分片 4158s → 3706s,离散度 47% → 15%);以及任何划分都突破不了的下界(grpc-module1701s)timingsjob:合并排名写入 run summary,member-timingsartifact 择时手工刷新而非自动回写,以及为什么不自动提交cache输入,以及全局包构建缓存为何回归(mcpp#344 的理由在 2026.8.3.4 被消除;保留旁路会让 linux 全量运行达到 2h30m 而超时,即根本得不到结果)run_members.sh/plan_shards.lua/member-timings.tsvtests/run_members.sh--shard此前自行用轮转法划分,而 CI 走plan_shards.lua的实测装箱 —— 同一个问题两套算法,而该脚本头部恰恰声称「CI 与本地是同一套」。现在--shard委托给plan_shards.lua,本地第 N 片与 CI 第 N 片持有相同成员;PATH上没有 lua 时退回轮转法,并在输出中明说。新增--platform选择读取member-timings.tsv的哪一列,缺省为宿主平台。验证
首个成员
protobuf-protoc,与lua5.4 tests/plan_shards.lua linux 1 3直接输出一致。无 lua 的回退路径也实测过(以只含必需工具、不含 lua 的 PATH 运行):
改动
tests/*.sh会触发全量运行,因此本 PR 的 CI 同时是对所文档化的分片机制本身的一次端到端验证。