Skip to content

Commit 915b89a

Browse files
committed
fix(bench): take the bmi_schedule arm out of CI, and quote the default configuration
L2 的正确性缺陷第四次尝试后仍未修好,所以: * CI 的引擎展开里去掉 `mcpp[schedule=on]` —— 它是 opt-in、默认关闭,用户拿到的 东西不受影响,但 **CI 不应该去测一个构建不起来的配置**。放回去是一行,门槛写在 workflow 注释里:§8 的复现六个场景全绿。 * 两个 README 的表格改回引用**默认配置**(cold 79.54s · 1.2x, touch-hub 0.40s · 208x,edit-comment 0.38s · 218x),并注明 schedule 能把 cold 降到 35.4s 但因缺陷未修而不引用。中文 README 补上 edit-comment 一行。 进展与坑记录在 §8b:导入者读不到 BMI 的那条**已经修好**(旧设计在 spawn 之前就把 BMI rename 走,实测消失 ~208ms;改成复制+按文件身份判断发布)。仍未修好的是 `touch-leaf`/`edit-body` 的**链接**失败 —— 方向是「BMI 的 restat 抑制不能连带 抑制这个单元自己的 object 边」。 ⚠️ 中途两个坑值得单独记:`file_size` 失败返回 `(uintmax_t)-1`,我在检查 `ec` 之前就写进了结构体;`copy_file` 盖的是拷贝时刻的 mtime,不显式带过去就把 restat 抑制整个废掉 —— 而那次**六个格子全绿**,只有数字(touch-hub 12.61s vs 冷构建 12.37s)能看出来。
1 parent 6739b7f commit 915b89a

5 files changed

Lines changed: 141 additions & 23 deletions

File tree

.agents/docs/2026-08-13-build-optimization-status.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -487,3 +487,41 @@ P1689 扫描的产出上;第一版发射(未提交)会让 `mcpp build` **段错
487487
* 修法方向:要么让 BMI 在整个重建期间保持可读(先编译到临时路径、成功后再原子
488488
替换,而不是先把旧的挪走),要么让导入者的边真正等到 BMI **重新发布**之后。
489489
前者更像是对的 —— 「先移走再重建」本身就在制造一个不存在的中间态。
490+
491+
492+
## 8b. 第四次尝试之后:把 L2 从 CI 里撤出,并说明现状
493+
494+
**已修好的两件事(独立成立,与下面那条无关):**
495+
496+
* `compile_release_at_bmi``read_rc` 分支返回成功却不 `settle_bmi` ——
497+
上一份 BMI 停在 `.bak`,而且那个单元的**等价性检查从未运行**,级联抑制对最便宜
498+
的单元静默关闭。
499+
* 编译失败时不再把单元留在「完全没有 BMI」的状态。
500+
501+
**「导入者读不到 BMI」已经修好。** 原设计在 spawn 编译器**之前**就把旧 BMI
502+
`rename` 走,于是模块在磁盘上有约 208ms 没有 BMI(实测)。改成**复制**一份到
503+
`.bak`、原件留在原地,并用「文件身份(size+mtime)发生变化」而不是「文件存在」
504+
来判断发布。`failed to read compiled module` 不再出现。
505+
506+
⚠️ 中间踩的两个坑,都值得记住:
507+
* `std::filesystem::file_size(p, ec)` 失败时返回 `(uintmax_t)-1`。我在检查 `ec`
508+
**之前**就把它写进结构体,于是「文件不存在」与默认构造的哨兵不相等 —— phase 1
509+
第一次轮询就认为「变了」,在编译器产出任何东西之前返回,所有 object 边报
510+
`no compiler was started … phase 1 did not run`
511+
* `copy_file` 给副本盖的是**拷贝时刻**的 mtime。而 `settle_bmi` 恢复这份副本正是
512+
为了让 mtime **不前进**、让 ninja 的 restat 掐断级联。不显式把原 mtime 带过去,
513+
恢复反而把 mtime 推前 —— `touch-hub` 变成 12.61s(冷构建 12.37s),
514+
**六个格子全报 `ok`**。状态列抓不到这个,只有数字能。
515+
516+
**仍然没修好的:** `touch-leaf` / `edit-body` 现在挂在**链接**上 ——
517+
`undefined reference to unit_19_value@fx.unit_19()`。方向应当是:BMI 的 restat
518+
抑制不能连带抑制**这个单元自己的 object 边** —— 它的源码确实变了,object 确实
519+
必须重建。BMI 不变(导入者不必重建)与 object 必须重建,是两件事。
520+
521+
**因此 CI 里暂时不跑 `+schedule=on` 这条臂。** 它是 opt-in、默认关闭,用户拿到的
522+
东西不受影响;但 CI 不应该去测一个构建不起来的配置。放回去是一行,门槛是 §8 的
523+
复现全绿。
524+
525+
**这条 bug 我连错四次**(误诊 settle_bmi、哨兵不匹配、mtime 没带过去、以及现在的
526+
object 边)。记在这里是因为下一个人应该从「object 边与 BMI 边的 restat 语义不同」
527+
开始,而不是从头再猜一遍。

.github/workflows/bench.yml

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -357,7 +357,17 @@ jobs:
357357
IFS=',' read -ra want <<< '${{ matrix.engines }}'
358358
for e in "${want[@]}"; do
359359
if [ "$e" = "mcpp" ]; then
360-
e="mcpp=$MCPP_UNDER_TEST,mcpp[schedule=on]=$MCPP_UNDER_TEST,mcpp"
360+
# ⚠️ THE `mcpp[schedule=on]` ARM IS DELIBERATELY ABSENT.
361+
#
362+
# `[build] bmi_schedule = "on"` has an unresolved correctness bug
363+
# on incremental rebuilds — see
364+
# .agents/docs/2026-08-13-build-optimization-status.md §8. It is
365+
# opt-in and off by default, so nothing users get is affected, but
366+
# CI must not measure a configuration that does not build.
367+
#
368+
# Putting it back is one line, and the §8 reproduction is the gate:
369+
# all six fixture scenarios green at --runs 2 before it returns.
370+
e="mcpp=$MCPP_UNDER_TEST,mcpp"
361371
fi
362372
engines="${engines:+$engines,}$e"
363373
done

README.md

Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -312,25 +312,24 @@ Each cell is the median wall-clock and how many times faster it is than cmake.
312312

313313
| scenario | what changed | **mcpp** | cmake | xmake |
314314
|---|---|---|---|---|
315-
| `cold` | nothing built yet | **35.43s** · 2.6x | 92.33s · 1.0x | 90.30s · 1.0x |
315+
| `cold` | nothing built yet | **79.54s** · 1.2x | 92.33s · 1.0x | 90.30s · 1.0x |
316316
| `noop` | nothing at all | **0.16s** · 1.8x | 0.28s · 1.0x | 0.38s · 0.7x |
317-
| `touch-hub` | mtime on a widely-imported interface, content unchanged | **0.22s** · 379x | 83.39s · 1.0x | 82.07s · 1.0x |
318-
| `edit-body` | a real edit inside a function body | **30.17s** · 2.8x | 85.64s · 1.0x | 84.61s · 1.0x |
319-
| `edit-comment` | a comment added to a widely-imported interface | **0.18s** · 461x | 82.96s · 1.0x | 82.73s · 1.0x |
317+
| `touch-hub` | mtime on a widely-imported interface, content unchanged | **0.40s** · 208x | 83.39s · 1.0x | 82.07s · 1.0x |
318+
| `edit-body` | a real edit inside a function body | **76.24s** · 1.1x | 85.64s · 1.0x | 84.61s · 1.0x |
319+
| `edit-comment` | a comment added to a widely-imported interface | **0.38s** · 218x | 82.96s · 1.0x | 82.73s · 1.0x |
320320

321-
> ⚠️ `bmi_schedule` is opt-in and currently has a **known correctness bug** on
322-
> incremental rebuilds (see [`bench/README.md`](bench/README.md)); the `cold`
323-
> figure here was measured with that defect present.
324-
325-
<sub>mcpp with `[build] bmi_schedule = "on"`. Linux x86_64 · i9-13900K ·
326-
gcc 16.1.0 · n=1 · pinned workload `a749e9f`.</sub>
321+
<sub>mcpp in its DEFAULT configuration. Linux x86_64 · i9-13900K · gcc 16.1.0 ·
322+
n=1 · pinned workload `a749e9f`. The opt-in `[build] bmi_schedule = "on"` takes
323+
`cold` to 35.4s, but it has an unresolved correctness bug on incremental
324+
rebuilds and is therefore not quoted here — see
325+
[`bench/README.md`](bench/README.md).</sub>
327326

328327
* **`touch-hub` and `edit-comment` are where the day goes.** cmake and xmake
329328
decide by timestamp and rebuild everything downstream; mcpp compares the BMI
330329
the compiler just produced against the previous one, and when the interface
331330
did not change it skips the cascade entirely.
332331
* **`edit-body` is the control.** There the interface really did change, so the
333-
cascade is owed — mcpp is 2.8x rather than 400x, and an engine that were
332+
cascade is owed — mcpp is 1.1x rather than 200x, and an engine that were
334333
faster would have skipped work it owed.
335334
* **Cold builds** come down to one 26-deep chain of module interfaces. `mcpp`
336335
publishes each BMI as soon as it exists and moves code generation off the

README.zh-CN.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -307,13 +307,14 @@ import mcpplibs.cmdline;
307307

308308
| 场景 | 改了什么 | **mcpp** | cmake | xmake |
309309
|---|---|---|---|---|
310-
| `cold` | 什么都还没构建 | **79.5s** | 92.3s | 90.3s |
311-
| `noop` | 什么都没改 | **0.16s** | 0.28s | 0.38s |
312-
| `touch-hub` | 给被大量 import 的接口改 mtime | **0.40s** | 83.4s | 82.1s |
313-
| `edit-body` | 函数体内部一处真实修改 | 76.2s | 85.6s | 84.6s |
310+
| `cold` | 什么都还没构建 | **79.54s** · 1.2x | 92.33s · 1.0x | 90.30s · 1.0x |
311+
| `noop` | 什么都没改 | **0.16s** · 1.8x | 0.28s · 1.0x | 0.38s · 0.7x |
312+
| `touch-hub` | 给被大量 import 的接口改 mtime | **0.40s** · 208x | 83.39s · 1.0x | 82.07s · 1.0x |
313+
| `edit-body` | 函数体内部一处真实修改 | **76.24s** · 1.1x | 85.64s · 1.0x | 84.61s · 1.0x |
314+
| `edit-comment` | 给被大量 import 的接口加一条注释 | **0.38s** · 218x | 82.96s · 1.0x | 82.73s · 1.0x |
314315

315316
* **冷构建三家都在 15% 以内** —— 依赖图是一条 26 层深的模块接口链,没有可供调度
316-
的余地。打开 `[build] bmi_schedule = "on"` 后 mcpp 的冷构建降到 **35.4s**
317+
的余地,任何引擎都缩短不了一条链
317318
* **`touch-hub` 才是一天里真正花掉的时间。** cmake 和 xmake 按时间戳判断,下游
318319
全量重建;mcpp 把编译器刚产出的 BMI 和上一份比对,接口没变就不级联 ——
319320
**0.40s 对 83s**
@@ -322,7 +323,10 @@ import mcpplibs.cmdline;
322323
📊 **[测量方法、钉住的版本与完整数据 → `bench/README.zh-CN.md`](bench/README.zh-CN.md)**
323324
· [English](bench/README.md)
324325

325-
<sub>Linux x86_64 · i9-13900K · gcc 16.1.0 · n=1 · 钉住的工作负载 `a749e9f`
326+
<sub>mcpp 为**默认配置**。Linux x86_64 · i9-13900K · gcc 16.1.0 · n=1 ·
327+
钉住的工作负载 `a749e9f`。opt-in 的 `[build] bmi_schedule = "on"` 能把 `cold`
328+
降到 35.4s,但它在增量重建上有一个尚未修好的正确性缺陷,因此这里不引用 ——
329+
`bench/README.md`
326330
套件还测量了第二个独立工程(xlings)的两种代码风格;那份对比、已声明的不对称、
327331
以及「什么时候一个格子**不能**拿来比较」的规则,都在 `bench/README.md`。</sub>
328332

src/build/schedule/detach_codegen.cppm

Lines changed: 72 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -227,6 +227,53 @@ void copy_first_rule(const std::filesystem::path& from, const std::filesystem::p
227227
// The BMI equivalence check, which used to be a POSIX shell one-liner inside the
228228
// generated ninja command — and was therefore skipped entirely on Windows.
229229
// Having it here is what brings cascade suppression to every platform.
230+
// A BMI's IDENTITY, so publication can be detected without the file ever having
231+
// to be absent.
232+
//
233+
// ⚠️ THE OLD DESIGN CREATED THE HOLE IT WAS TRYING TO AVOID. Phase 1 used to
234+
// `rename(bmi, bmi.bak)` before spawning the compiler — the comment said it was
235+
// so "its mere presence can never be mistaken for the new one landing". That is
236+
// a real hazard, but the cure left the module with NO BMI ON DISK from that
237+
// rename until the compiler republished: measured at ~208 ms of a single
238+
// incremental rebuild. Any importer scheduled inside that window dies with
239+
//
240+
// error: failed to read compiled module: No such file or directory
241+
// note: imports must be built before being imported
242+
//
243+
// reproducible at `-j1`, on four of six scenarios of the generated fixture.
244+
//
245+
// The previous BMI is now COPIED aside instead, so the file is continuously
246+
// readable and GCC's own atomic rename is what replaces it. Publication is
247+
// detected by the identity below changing, which is exactly the question the
248+
// existence check was a poor proxy for.
249+
struct BmiIdentity {
250+
bool present{};
251+
std::uintmax_t size{};
252+
std::filesystem::file_time_type mtime{};
253+
bool operator==(const BmiIdentity&) const = default;
254+
};
255+
256+
BmiIdentity bmi_identity(const std::filesystem::path& p) {
257+
BmiIdentity id;
258+
if (p.empty()) return id;
259+
std::error_code ec;
260+
// ⚠️ ASSIGN NOTHING BEFORE CHECKING `ec`. `file_size` returns
261+
// `static_cast<uintmax_t>(-1)` when it fails, so writing it into the struct
262+
// first makes "this file is missing" compare UNEQUAL to a default-built
263+
// identity — which is exactly the sentinel used for "there was no previous
264+
// BMI". Phase 1 then saw a difference on its very first poll and returned
265+
// before the compiler had produced anything, and every object edge failed
266+
// with `no compiler was started … phase 1 did not run`.
267+
const auto size = std::filesystem::file_size(p, ec);
268+
if (ec) return id;
269+
const auto mtime = std::filesystem::last_write_time(p, ec);
270+
if (ec) return id;
271+
id.present = true;
272+
id.size = size;
273+
id.mtime = mtime;
274+
return id;
275+
}
276+
230277
// Put the previous BMI back. Used when the compile failed: the unit still has
231278
// the BMI it had before, and leaving it parked in `.bak` would strand every
232279
// importer on a file that does not exist.
@@ -243,8 +290,11 @@ void settle_bmi(const std::filesystem::path& bmi) {
243290
const auto backup = suffixed(bmi, ".bak");
244291
if (!file_exists(backup)) return;
245292
std::error_code ec;
293+
// Equivalent → put the PREVIOUS file back, so its mtime does not advance and
294+
// ninja's restat stops the cascade. The rename is atomic, so the BMI is
295+
// readable throughout: there is no moment at which importers see nothing.
246296
if (stage::bmi_equivalent(bmi, backup))
247-
std::filesystem::rename(backup, bmi, ec); // keep the old mtime: no cascade
297+
std::filesystem::rename(backup, bmi, ec);
248298
else
249299
std::filesystem::remove(backup, ec);
250300
}
@@ -369,12 +419,27 @@ int compile_release_at_bmi(const CompileRequest& req) {
369419
std::filesystem::remove(suffixed(req.slot, ".rc"), ec);
370420
std::filesystem::remove(suffixed(req.slot, ".rc.tmp"), ec);
371421

372-
// Keep the previous BMI for the equivalence check AND get it out of the
373-
// way, so its mere presence can never be mistaken for the new one landing.
422+
// Keep the previous BMI for the equivalence check, and LEAVE THE ORIGINAL
423+
// IN PLACE — see BmiIdentity for why moving it away is what broke importers.
424+
// Snapshot through the SAME function in both cases, so "no previous BMI"
425+
// and "the BMI as it is now" are directly comparable.
426+
const BmiIdentity before = bmi_identity(req.bmi);
374427
if (!req.bmi.empty()) {
375428
const auto backup = suffixed(req.bmi, ".bak");
376429
std::filesystem::remove(backup, ec);
377-
if (file_exists(req.bmi)) std::filesystem::rename(req.bmi, backup, ec);
430+
if (before.present) {
431+
std::filesystem::copy_file(
432+
req.bmi, backup, std::filesystem::copy_options::overwrite_existing, ec);
433+
// ⚠️ AND CARRY THE MTIME ACROSS. `copy_file` stamps the copy with
434+
// the time of the copy, and `settle_bmi` restores this file when the
435+
// new BMI turns out equivalent — precisely so the mtime does NOT
436+
// advance and ninja's restat stops the cascade. Without this line
437+
// the restore moves the mtime forward instead, every importer is
438+
// rebuilt, and the optimisation is silently off: `touch-hub` came
439+
// back at 12.61s against a 12.37s cold build, with every cell
440+
// reporting `ok`. A status column cannot catch that; the number can.
441+
std::filesystem::last_write_time(backup, before.mtime, ec);
442+
}
378443
}
379444

380445
const auto token = acquire_token(req.semaphore, req.maxCompilers);
@@ -389,7 +454,9 @@ int compile_release_at_bmi(const CompileRequest& req) {
389454
if (!spawn_detached(sup)) return 2;
390455

391456
for (;;) {
392-
if (!req.bmi.empty() && file_exists(req.bmi)) {
457+
// Published = the file's identity is no longer the one we snapshotted.
458+
// For a unit with no previous BMI that reduces to "it now exists".
459+
if (!req.bmi.empty() && bmi_identity(req.bmi) != before) {
393460
settle_bmi(req.bmi);
394461
copy_first_rule(req.depFrom, req.depTo, req.bmi.string());
395462
return 0; // importers may proceed

0 commit comments

Comments
 (0)