From e01e6bf281596483401112c0743a4c45efc1efea Mon Sep 17 00:00:00 2001 From: speak-agent Date: Tue, 4 Aug 2026 01:28:03 +0800 Subject: [PATCH 1/2] fix(index): an index-side change must not decide whether mcpp works MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A published index can declare a client-version floor (index.toml min_mcpp). `xlings update` rewrites the local tree in place, so when mcpp-index raised its floor, every older mcpp went from working to broken — mid-session, on a machine that had been fine sixty seconds earlier: error: index requires mcpp >= 2026.8.3.3 but this is mcpp 0.0.109 [E0006] error: ... package 'compat:compat.libarchive@3.8.7' not found and there was no way back, because the readable tree had already been overwritten. The refresh was the thing that broke the machine. An index is DATA and mcpp is a PROGRAM. Publishing data must not be able to disable the program. The invariant this installs: an index-side change must never take mcpp from "works" to "does not work" REFRESH IS NOW MONOTONE. `mcpp::xlings::update_index` — the one function every refresh in the process passes through — archives each readable index tree, lets the sync happen, re-judges, and rolls back anything that got worse. The test is "usable before ⇒ usable after", not "usable after": a tree that was already unreadable has nothing to roll back to. The 2026-07-08 index design specified exactly this ("staged refresh keeps the last compatible snapshot") and it was never implemented — the floor was checked in one place, the descriptor reader, and the refresh path knew nothing about it. Shape is archive/judge/restore rather than the stage-and-swap that design assumed, because mcpp does not fetch indexes: it shells out to xlings, which rewrites in place and takes no destination. Same guarantee, no cross-repo dependency. A short local history (5 per tree) means "the newest snapshot this binary can read" is answerable offline — the local, dependency-free version of what openxlings/xlings#476 would provide for snapshots this machine never saw. "UNREADABLE" IS NO LONGER SPELLED "NOT FOUND". A floor violation made every descriptor read return nothing, which is the same answer as "this package is not in this index". Two things went wrong downstream: the message that STOPPED the build said `dependency '...': not found`, blaming the package and pointing at publication or naming while the real answer (upgrade mcpp) had scrolled away; and that same indistinguishable miss fed the refresh policy as "go fetch", so the unusable state drove repeated refreshes of itself. It is now a queryable fact — the refresh policy stops retrying, and all three terminal not-found sites in prepare.cppm carry the cause via one helper. MCPP_VERSION MOVED TO A LEAF MODULE (src/version.cppm). Not cosmetic: the guard would not compile where it belongs. index_contract needs the version for one comparison and was reaching it through toolchain.fingerprint → toolchain.detect → xlings, so "what version is this binary" transitively depended on the entire toolchain-detection subsystem — which made it impossible for xlings to depend on index_contract. The cycle the compiler reported was the layering saying the constant was in the wrong place. check_version_pins.sh and the release docs follow it. TESTS THAT DID NOT EXIST. The floor had unit coverage of the pure predicate and zero e2e, which is why both defects above survived: the predicate was right and the behaviour around it was never run end to end. Adds 9 unit tests (including "a refresh that breaks the index is rolled back") and e2e 185. Two of those tests earned their keep immediately: * hard links looked like the obvious way to make snapshots free, and the round-trip test caught that a hard-linked snapshot ALIASES the live tree — an in-place rewrite writes through the link and destroys the backup, which is precisely the event the module exists to survive. Real copies; ~15 MB for five, bounded by prune. * the first e2e asserted "E0006 appears in the output" and passed on the pre-fix binary (verified). It always appeared. Retargeted at the LAST error, where the defect actually lived — now red before, green after. Only helps users who upgrade to this version; anyone already stuck on an older mcpp still needs either an upgrade or an index-side floor rollback. --- ...ility-must-not-decide-mcpp-availability.md | 299 +++++++++++ ...03-index-floor-should-degrade-not-brick.md | 480 ++++++++++++++++++ .agents/skills/mcpp-release/SKILL.md | 6 +- .github/tools/check_version_pins.sh | 8 +- CHANGELOG.md | 26 + docs/09-release.md | 2 +- docs/zh/09-release.md | 2 +- mcpp.toml | 2 +- src/build/prepare.cppm | 26 +- src/pm/index_contract.cppm | 106 +++- src/pm/index_refresh.cppm | 21 + src/pm/index_snapshot.cppm | 337 ++++++++++++ src/pm/package_fetcher.cppm | 10 + src/toolchain/fingerprint.cppm | 7 +- src/version.cppm | 36 ++ src/xlings.cppm | 56 ++ tests/e2e/185_index_floor_degrades.sh | 192 +++++++ tests/unit/test_index_snapshot.cpp | 227 +++++++++ 18 files changed, 1820 insertions(+), 23 deletions(-) create mode 100644 .agents/docs/2026-08-03-index-availability-must-not-decide-mcpp-availability.md create mode 100644 .agents/docs/2026-08-03-index-floor-should-degrade-not-brick.md create mode 100644 src/pm/index_snapshot.cppm create mode 100644 src/version.cppm create mode 100755 tests/e2e/185_index_floor_degrades.sh create mode 100644 tests/unit/test_index_snapshot.cpp diff --git a/.agents/docs/2026-08-03-index-availability-must-not-decide-mcpp-availability.md b/.agents/docs/2026-08-03-index-availability-must-not-decide-mcpp-availability.md new file mode 100644 index 00000000..4cf1fa8f --- /dev/null +++ b/.agents/docs/2026-08-03-index-availability-must-not-decide-mcpp-availability.md @@ -0,0 +1,299 @@ +# 索引的可用性不得决定 mcpp 的可用性 —— 纯 mcpp 侧优化方案 + +> 状态:**已实施**(2026.8.3.5)。实施中的三处偏差记录在 §7。 +> 范围:**全部落在 `mcpp-community/mcpp` 内**。零跨仓依赖 —— 不需要 xlings 改动, +> 也不需要索引侧改动。 +> 背景分析:`.agents/docs/2026-08-03-index-floor-should-degrade-not-brick.md` +> 跨仓增强(非前置):[openxlings/xlings#476](https://github.com/openxlings/xlings/issues/476) + +--- + +## 0. 要立的不变量 + +> **INV —— 索引侧的任何改动,都不得使 mcpp 从「能用」变成「不能用」。** + +这条不变量今天不成立:`mcpplibs/mcpp-index` 抬了一次 `min_mcpp`, +所有 `0.0.109` 用户的构建就死了。**一次索引提交,让一个已发布客户端不可用**, +这在包管理器里是不可接受的耦合 —— 索引是**数据**,客户端是**程序**, +数据的更新不该让程序失效。 + +三条可检验的推论: + +| # | 推论 | 今天 | +|---|---|---| +| **INV-1** | **刷新是单调的**:刷新只能维持或改善可用性,绝不能降低 | ❌ 刷新把能用的换成不能用的,且不可逆 | +| **INV-2** | **不可用是有边界的**:一个索引不可用,只影响**它提供的包**;不影响其它索引,也不影响**已解析/已安装**的依赖 | ⚠️ 边界在读取层存在,但在错误语义与刷新决策上泄漏 | +| **INV-3** | **不可用要自报家门**:绝不伪装成「包不存在」 | ❌ `return std::nullopt` ⇒ 全部变成 not-found | + +再加一条设计判据,后面几处都从它推出来: + +> **下限约束的是「读描述符」,不是「构建」。** +> 一个不需要读任何描述符的构建(锁文件齐全、payload 已装),不该被下限拦住。 + +--- + +## 1. 今天为什么违反 + +### 1.1 刷新路径对下限一无所知(违反 INV-1) + +``` +$ grep -rn "check_index_floor" src/ +src/pm/package_fetcher.cppm:619 ← 全代码库唯一一处 +``` + +`index_management.cppm:148` 的刷新是 + +```cpp +int rc = mcpp::xlings::update_index(xlEnv); // 跑 `xlings update`,原地覆盖 +``` + +**刷新前不问「换来的这棵树我能用吗」,刷新后也不问。** 于是一次 `xlings update` +就能把一份能用的索引原地换成用不了的,**没有备份、没有回滚、没有退路**。 + +用户在刷新之后比刷新之前更糟 —— 这是最伤人的性质,也是本方案第一优先级。 + +### 1.2 违反下限退化成「包不存在」(违反 INV-3) + +`package_fetcher.cppm:614-622`: + +```cpp +if (auto violation = mcpp::pm::check_index_floor(pkgsDir.parent_path())) { + mcpp::ui::error(*violation); + return std::nullopt; // 「读不了」与「没有」压成同一个返回值 +} +``` + +后果是终止构建的那条错误与真实原因隔了两层 +(`E_NOT_FOUND ... wire address tried: ...`,既不提版本也不提下限)。 + +还有一个更隐蔽的二次伤害:`index_refresh` 的判据是「解析输入是否都在磁盘」。 +一个被下限挡住的读取**看起来就是一次真实的 miss** ⇒ 判定「该刷新了」⇒ +再拉一次同样不兼容的树。**不可用状态自己驱动了重复刷新。** + +### 1.3 整条下限行为**没有任何 e2e 覆盖** + +``` +tests/unit/test_index_contract.cpp ← 只测纯谓词 floor_violation() +tests/e2e/ ← 零 +``` + +2026-07-08 的设计里写了要有 +(*"e2e: fixture index with `min_mcpp = "9.9.9"` → build fails"*),**没有落地**。 +所以 1.1 和 1.2 从上线到今天没有任何机制会发现 —— 谓词是对的, +**它周围的行为从没被端到端跑过**。 + +--- + +## 2. 方案(M1–M5,全部 mcpp 侧) + +### M1(必须)· 刷新单调性:先备份,后验收,不合格就回滚 + +**落点**:`src/pm/index_management.cppm` 的刷新编排。 + +``` +refresh(repo): + usable_before = floor_ok(repo.root) # 刷新前的可用性 + snapshot = archive(repo.root) # 见 M2,硬链接复制,≈0 成本 + xlings::update_index(...) # 原地覆盖,不在 mcpp 进程内 + if usable_before and not floor_ok(repo.root): + restore(repo.root, snapshot) # ← 回滚 + warn_once(downgrade_notice) + mark_no_refresh_this_process(repo) # 防抖,见 1.2 的二次伤害 +``` + +要点: + +- **判据是「刷新前能用 ⇒ 刷新后必须仍能用」**,不是「刷新后必须能用」。 + 本来就不能用的(冷启动撞上高下限),回滚没有意义,交给 M3/M4。 +- `xlings` 原地覆盖是既成事实,mcpp 无法要求它写到暂存目录 + (见分析文档 §4.0 的传输约束)。所以是**备份 + 事后验收 + 回滚**, + 而不是设计文档原本设想的「staged unpack + 验收后再切换」。 + **效果等价,且不需要任何跨仓改动。** +- 索引树很小(实测 `mcpplibs` 912K、`xim-pkgindex` 2.2M), + 用硬链接复制(`copy_options::create_hard_links`)近似零成本、零额外空间。 + +用户看到: + +``` +warning: the refreshed index requires mcpp >= 2026.8.3.3; this is mcpp 0.0.109. + Kept the previous index () — your build continues to work. + Upgrade to pick up newer packages: xlings update mcpp +``` + +**这一条单独就把「变砖」变成「可用 + 提示升级」。** + +### M2(必须)· mcpp 自己的本地快照历史 + +M1 只留一份「上一个」。M2 把它变成一条**本地历史线** —— 也就是 +「即使 xlings 没有相关功能,至少还有之前的」这句话的实现。 + +``` +~/.mcpp/registry/data/.index-snapshots/// +``` + +- **快照身份**用现成的 `.xlings-index-version`(每个索引根下已经有这个文件; + 它的值按契约是**不透明串**,不要当 sha 解析)。 +- 每次验收通过(`floor_ok`)就归档一份;保留最近 N 份(建议 N=5)+ 按体积上限回收, + 复用 `mcpp cache gc` 已有的 LRU 记账思路。 +- **需要时从新到旧回溯**,取第一个 `floor_ok` 的快照恢复。 + +与 [xlings#476](https://github.com/openxlings/xlings/issues/476) 的关系: + +| | 覆盖面 | 依赖 | +|---|---|---| +| **M2(本机历史)** | 只覆盖**这台机器见过**的快照 | **无** | +| #476(发布侧历史线) | 覆盖全部已发布快照,含全新安装 | 跨仓 | + +**M2 不是 #476 的替代品,是它的本地退化版**,且覆盖了绝大多数真实场景 +(一台持续在用的开发机/CI 缓存机,一定见过兼容的快照)。 +#476 落地后 M2 依然有价值:它让常见路径不必联网重下。 + +**诚实的边界**:全新安装 + 最新索引下限过高 ⇒ 本机没有任何历史 ⇒ +M2 无能为力,落到 M3/M4。这种情况下正确答案本来就是「升级 mcpp」, +M3 会把这件事说清楚。 + +### M3(必须)· `IndexUnusable` ≠ `Absent` + +**落点**:`src/pm/package_fetcher.cppm` 的描述符读取返回类型。 + +```cpp +enum class DescriptorLookup { Found, Absent, IndexUnusable }; +``` + +三条硬约束: + +1. `IndexUnusable` **不得**触发 legacy 派生地址回退(那是给「这个索引里没有这个包」用的); +2. `IndexUnusable` **不得**被 `index_refresh` 读成「本地缺东西 ⇒ 该刷新」—— + 这正是 1.2 那个重复刷新循环的入口; +3. 解析结束时若存在 `IndexUnusable`,**最终错误就是 E0006 本身**, + 并指明**哪个索引、要求什么、本机是什么、怎么升级**。 + 用户看到的第一条与最后一条错误必须是同一件事。 + +这条同时是 INV-2 的执行者:一个索引不可用**只**让它自己的包不可解析, +不得影响其它索引的 `Absent` 判定。 + +### M4(应做)· 下限拦的是「读描述符」,不是「构建」 + +由 §0 那条判据直接推出:**如果本次构建根本不需要读任何描述符 +(依赖都在 `mcpp.lock`、payload 都已安装),下限不该让它失败。** + +这正是 `index_refresh` 已经在用的那根轴(「所有解析输入都在磁盘」), +M3 落地后它自然成立:`IndexUnusable` 不再被误读成 miss, +于是「所有输入都在磁盘」的判定不受影响,构建照常离线进行。 + +**这是「mcpp-index 里的包升级不该影响 mcpp 可用性」的直接答案**: +一个已经解析完、装好的工程,与索引后来发生了什么完全无关。 + +> 边界要说清楚:如果构建**确实**需要装新东西(用户那次就是在装 `musl-gcc`), +> M4 帮不上,那是 M1/M2 的战场。M4 保的是**存量工程不被索引变更波及**。 + +### M5(必须)· 补上从未存在的 e2e + +现在只有纯谓词单测。至少三条端到端断言,**每一条都对应上面一个不变量**: + +| e2e | 断言 | 守的是 | +|---|---|---| +| 刷新单调性 | fixture 索引可用 → 构建成功;把它换成 `min_mcpp = "9.9.9"` 再刷新 → **构建仍然成功**,且 stderr 出现降级 warning | INV-1 / M1 | +| 错误自报家门 | 本机无任何兼容快照 → 构建失败,且**最后一条**错误含 `E0006`,stderr **不出现** `not found` | INV-3 / M3 | +| 边界隔离 | 索引 A 不可用、索引 B 正常 → B 的包照常解析成功 | INV-2 / M3 | + +第二条那个「最后一条错误必须是 E0006」正是这次(以及 +`install_pinned_mcpp.sh` 头注释记载的上一次)缺失的断言。 + +--- + +## 3. 不采纳 + +| 方案 | 理由 | +|---|---| +| 下限降级为纯 warning,照常解析 | 下限存在的理由是真的:`0.0.101` 那次是 `compat.opencv` 的 per-OS feature flags 被旧客户端**静默忽略**,产出错误构建。放行 = 用「静默错误产物」换「明确失败」,方向反了。M1/M2 保的是**旧快照**,不是**用旧客户端读新描述符**。 | +| 等 xlings#476 落地再做 | #476 是增强,不是前置。M1+M2+M3 **今天就能把「变砖」修掉**,且 #476 落地后全部保留价值。把用户可用性挂在别的仓库的排期上,本身就是这次问题的翻版。 | +| mcpp 自己接管索引下载 | 与 xlings 职责重叠,镜像/CDN/校验要重写一遍。M1 的「备份+回滚」用几十行拿到同样的可用性保证。 | +| 只做 M1,不做 M3 | 可用性修好了,但一旦真的走到不可用路径,用户仍然读到一条误导性的 `not found`。两者成本都很低,没有理由只做一半。 | +| 把 `MCPP_INDEX_FLOOR=ignore` 宣传成解法 | 它是调试逃生舱。让用户常态化绕过一个**为防止静默错误构建而存在**的闸,是把一个可用性问题换成一个正确性问题。 | + +--- + +## 4. 实施顺序 + +| # | 项 | 依赖 | 效果 | +|---|---|---|---| +| 1 | **M3** 错误语义 | 无 | 最小改动;同时堵掉重复刷新循环 | +| 2 | **M1** 备份 + 验收 + 回滚 | M3(复用 `floor_ok`) | **变砖 → 可用 + 提示升级** | +| 3 | **M2** 本地快照历史 | M1(归档即 M1 的备份) | 「至少还有之前的」升级为一条历史线 | +| 4 | **M5** e2e | M1–M3 | 让上面三条永不回归 | +| 5 | M4 复核 | M3 | 多数情况下 M3 落地即自然成立,需实测确认 | + +M1–M3 是一个 PR 的量级,建议一起发;M5 同 PR。 + +**发版说明要写清楚**:本次修复只对**升级到该版本之后**的用户生效。 +已经卡住的 `0.0.109` 用户,要么升级 mcpp,要么等索引侧把下限降回去 —— +这也是为什么索引侧那次回退仍然值得单独做(见背景分析文档 §6 第 1 步)。 + +--- + +## 5. 验证 + +- `mcpp test --workspace`(单测)+ 新增三条 e2e。 +- **先红后绿**:M5 的三条 e2e 必须先在 `main` 上跑出红,再实施 M1–M3。 + 按 §1.3,今天它们一条都不存在,所以「先红」这一步不能省 —— + 否则无法证明它们真的覆盖到了。 +- 人工复现:构造一个 fixture 索引,`min_mcpp` 从合法值改成 `9.9.9`, + 跑 `mcpp build` 两次,断言第二次仍然成功且打出降级 warning。 + +--- + +## 6. 这次暴露的一条通用教训 + +设计文档写下的行为("staged refresh keeps the last compatible snapshot") +与实现之间没有任何机器化的连接,于是**承诺存在、实现缺席,而且长期没人发现**。 +缺席的不是代码,是**那条本该失败的测试**:谓词有单测(所以谓词是对的), +谓词周围的行为没有 e2e(所以行为是错的)。 + +判据:**一个设计文档里以「mandatory」措辞写下的行为,必须有一条以它命名的测试。** +没有测试的 mandatory 是一句愿望。 + + +--- + +## 7. 实施记录(2026.8.3.5) + +落点:`src/version.cppm`(新)、`src/pm/index_snapshot.cppm`(新)、 +`src/pm/index_contract.cppm`、`src/pm/index_refresh.cppm`、`src/build/prepare.cppm`、 +`src/pm/package_fetcher.cppm`、`src/xlings.cppm`、 +`tests/unit/test_index_snapshot.cpp`(新)、`tests/e2e/185_index_floor_degrades.sh`(新)。 + +三处与方案的偏差,都是实施时被现实证伪的假设: + +**① 守卫装不进 `xlings::update_index` —— 编译器报了循环依赖。** +方案假定「全进程唯一刷新入口」就是落点。实际 `mcpp.xlings` 无法 import +`mcpp.pm.index_contract`:后者 import `mcpp.toolchain.fingerprint` +(只为读 `MCPP_VERSION`)→ `mcpp.toolchain.detect` → `mcpp.xlings`。 +**「这个二进制是什么版本」传递依赖了整个工具链探测子系统**,这就是分层在报错。 +把常量提到叶子模块 `src/version.cppm` 后环消失。 +连带:`check_version_pins.sh` 与三处发布文档要改指向(版本号唯一真源换了文件)。 + +**② 快照绝不能用硬链接 —— 单测立刻抓到。** +`copy_options::create_hard_links` 是显然的优化(索引树是几千个小文件)。 +它在这里是**错的**:硬链接快照**与活动树共享 inode**,任何原地重写 +(截断写、tar 覆盖解包)会直接写穿链接、毁掉备份。而本模块要挺过的那件事 +正是「有东西替换了这棵树」。改为真实复制;实测 912KB/2.2MB × 5 ≈ 15MB,`prune` 封顶。 + +**③ 最初的 e2e 是假绿 —— 在 pre-fix 二进制上照样通过。** +第一版断言写的是「输出里出现 E0006」。它**一直**出现,所以那条断言什么也没测到 +(已用 pre-fix 二进制实测确认通过)。真正坏掉的是**终止构建的那条消息**: + +``` +error: dependency 'toonew.newlib': not found in local index at '...' +``` + +它**把责任推给包**(指向发布/命名),而真正的答案早已滚出屏幕。 +断言改为针对**最后一条 error**,并要求它带上真正的原因;改完在 pre-fix 上精确变红。 +> 这条同时验证了 §6 那句判据:**没有测试的 mandatory 是一句愿望** —— +> 而一条写得不够精确的测试,比没有测试更危险,因为它让人以为覆盖到了。 + +另:终止构建的 not-found 不止一处(`prepare.cppm` 有三处), +所以原因附加收敛为一个 `with_index_cause()` helper,而不是逐处拼字符串。 + +验证状态:unit 55/55(含新增 9 条);e2e 185 在修复前后分别红/绿; +`check_version_pins.sh` 通过。 diff --git a/.agents/docs/2026-08-03-index-floor-should-degrade-not-brick.md b/.agents/docs/2026-08-03-index-floor-should-degrade-not-brick.md new file mode 100644 index 00000000..c76efae7 --- /dev/null +++ b/.agents/docs/2026-08-03-index-floor-should-degrade-not-brick.md @@ -0,0 +1,480 @@ +# 索引版本下限(E0006)不应让旧客户端不可用 —— 分析与方案 + +> 状态:分析定稿,待实施 +> 触发:用户报告 mcpp `0.0.109` 撞上 `index requires mcpp >= 2026.8.3.3 [E0006]`, +> 随后整条依赖解析崩成 `package 'compat:compat.libarchive@3.8.7' not found` +> 涉及:`src/pm/index_contract.cppm`、`src/pm/package_fetcher.cppm`、 +> `src/pm/index_refresh.cppm`、`src/doctor.cppm`;索引侧 `mcpplibs/mcpp-index` +> 跨仓依赖:[openxlings/xlings#476](https://github.com/openxlings/xlings/issues/476)(枚举 + 按版本同步索引快照) +> 相关设计:`.agents/docs/2026-07-08-index-version-semantics-and-descriptor-grammar-design.md` + +--- + +## 0. 结论摘要 + +**提问是对的,而且比它看上去更严重。**「旧 mcpp 应该仍然可用,只是提示可以升级」不仅是 +合理诉求 —— 它本来就是 2026-07-08 那份设计里 **L1 明文承诺过的行为**: + +> **L1 — authority (this design, mandatory)**: contract travels inside the tree; +> single check at the index-open choke point; **staged refresh keeps the last +> compatible snapshot**. Parse failure → graceful keep-old + warn. + +「保留上一个兼容快照」**从未被实现**。今天的实现只有那一个 choke point 检查, +刷新路径对下限一无所知。于是 `xlings update` 会把一份能用的索引换成一份用不了的, +**且没有退路** —— 用户在刷新之后比刷新之前更糟。 + +同时,下限本身正在以设计没有预期的频率上移:它被当作「等于 CI pin」维护, +而 CI pin 跟着最新发布走。结果是**一个一周前的版本,被一次「新增一个包」的提交废掉**。 + +**方案的骨架是一条「索引历史线」**(§4.0):索引不再是一个 rolling 指针,而是一条 +带下限的版本线;客户端刷新时从最新往回找,取**第一个自己满足下限的快照**, +并只在发生降级时打印一次「最新索引需要 mcpp X,你是 Y,已改用 Y 能用的最新索引 Z, +升级请 `xlings update mcpp`」。**版本下限从终止信号变成路由信号。** + +分发侧不需要新建任何东西:F5 证明 313 个历史快照都还在,缺的只是指针里的一份元数据, +而按 F1 的历史,下限拐点至今只有 6 个 —— **一条 6 项的历史线覆盖整个索引生命周期**。 + +六个发现,四个必须修,两个是治理问题。 + +--- + +## 1. 现象:一次失败,三段错误 + +``` +Downloading xim:musl-gcc@… [===================>] 92.0 MB / 92.9 MB +error: index requires mcpp >= 2026.8.3.3 but this is mcpp 0.0.109 [E0006] + Upgrade: curl -fsSL .../install.sh | bash +error: xlings install_packages failed (exit 1) for 'compat.libarchive@3.8.7' … + xlings reported: E_NOT_FOUND: package 'compat:compat.libarchive@3.8.7' not found + wire address tried: compat:compat.libarchive@3.8.7 +``` + +三段,各自都有问题: + +1. **已经下载完 92MB 才发现用不了。** 兼容性判定发生在解包之后、读描述符的时候。 +2. E0006 说清楚了原因,但它**不是终止点**。 +3. 真正终止构建的是第三段 —— 一条 `not found`,**既不提版本也不提下限**, + 还附了一个"wire address tried",把读者引向命名/寻址问题。 + +第 3 段不是巧合,是第 2 段的必然结果,见 F3。 + +--- + +## 2. 六个发现(均有证据) + +### F1 —— 下限被当作「= CI pin」维护,而不是「能解析这棵树的最老客户端」 + +`index_contract.cppm` 的定义是清楚的: + +``` +min_mcpp = "0.0.85" # oldest mcpp able to parse every descriptor +``` + +但 `mcpp-index/index.toml` 的维护规则写的是另一件事: + +``` +# Bump min_mcpp ONLY together with the CI MCPP_VERSION pin — lint parses +# descriptors with the pinned mcpp, which enforces the "floor first, new +# grammar after" rollout rule mechanically. +``` + +「ONLY together with」本意是**约束**(不许单独抬下限),实际被当成**等价**(pin 动,下限跟着动)。 +证据是最近这次 bump,commit `160c389` **"feat: add boost-ext.ut 2.3.1 C++23 module package"**: + +| 文件 | 变化 | +|---|---| +| `index.toml` | `min_mcpp` `0.0.109` → `2026.8.3.3` | +| `.github/workflows/validate.yml` | `MCPP_VERSION` `0.0.109` → `2026.8.3.3` | + +而该 commit 自己的注释说明了 pin 为什么动: + +> `…this floor on macOS, and **min_mcpp/latest_mcpp move with the pin as they** …` +> `2026.8.3.3 is the pin rather than .3.1: .3.2/.3.3 are cross-compilation…` + +pin 上移的真实理由是 **mcpp#336(macOS 静态初始化)** 与交叉编译修复 —— +**与描述符语法毫无关系**。新加的 `boost-ext.ut` 描述符用的是 `generated_files`, +一个早就存在的特性;commit 信息本身还写着它是对着 **0.0.109** 开发验证的。 + +⇒ **下限上移了一百多个版本,没有任何描述符真的需要它。** +一个约一周前的发布(`0.0.109`)因此被废掉。 + +### F2 —— L1 承诺的「保留上一个兼容快照」从未实现 + +下限在**整个代码库里只被检查一次**: + +``` +$ grep -rn "check_index_floor" src/ +src/pm/package_fetcher.cppm:619 +``` + +`index_refresh.cppm`(决定「现在该不该联网刷新索引」的唯一真源)**完全不知道下限的存在**。 +所以刷新流程从不问「我拉回来的这棵树,我自己能用吗」。 + +后果正是用户遇到的:**刷新把一份能用的索引换成用不了的,且不可回退**。 +设计文档承诺的 "staged refresh keeps the last compatible snapshot" 落空了。 + +### F3 —— 违反下限降级成了「包不存在」,而不是「索引不可用」 + +`package_fetcher.cppm:614-622`: + +```cpp +// Loud once per index; the resolve then fails as not-found with the cause +// already printed. +if (auto violation = mcpp::pm::check_index_floor(pkgsDir.parent_path())) { + mcpp::ui::error(*violation); + return std::nullopt; // ← 每一次描述符读取都变成 "没找到" +} +``` + +注释承认了这个设计("the resolve then fails as not-found"),但它把**两个语义不同的事实 +压成了同一个返回值**: + +- 「这个包不在索引里」 +- 「这个索引我读不了」 + +于是解析器继续按「没找到」的剧本走:回退到 legacy 派生 wire 地址、尝试其它 repo、 +最后报一条 `E_NOT_FOUND … wire address tried: …`。**终止构建的那条错误,与真实原因隔了两层。** + +> 这个失败形态在本仓库有先例,且已被写进 `install_pinned_mcpp.sh` 的头注释: +> *"the floor check made EVERY descriptor read return nothing, every dependency fell +> back to its legacy derived address, and the build died on `mcpplibs.cmdline@0.0.1` +> — an error naming neither the version nor the floor."* +> 同一个坑,换了一个触发源,又踩了一次。 + +### F4 —— `latest_mcpp` 被每个索引写入,但**没有任何消费者** + +``` +$ grep -rn "latestMcpp\|latest_mcpp" src/ | grep -v index_contract.cppm +(空) +``` + +它被解析进 `IndexContract::latestMcpp` 就再没被读过。而它恰好就是协商层需要的那个字段。 + +### F5 —— 分发侧已经保留了全部历史快照,协商今天就做得到 + +`xlings-res/xim-index` 的 rolling `latest` release 里有 **313 个 artifact** +(`xim-index-.tar.gz`),历史快照并没有被删。 + +缺的只是**指针里没有下限信息**: + +```json +{ "format_version": 1, "index_version": "20e53c6", + "artifact": { "name": "xim-index-20e53c6.tar.gz", "sha256": "…", "size": 371707 } } +``` + +客户端在下载前无从判断这份 artifact 自己能不能用 —— 这正是设计里 **L2(projection)** +被推迟掉的那一层。**「用自己版本当前最新的 index」在分发侧已经可行,只差元数据。** + +### F6 —— 下限是**整个索引**一个开关,而需求是**逐描述符**的 + +设计明确拒绝过逐描述符下限: + +> Rejected alternative: per-descriptor `min_mcpp`. Finer-grained, but N places to +> maintain, and the failure it prevents … is already covered by one index-wide dial. + +代价是:`compat.zlib` 的描述符几个月没动过,却因为**别人**新加了一个包而变得不可读。 +索引越大、贡献越活跃,这个耦合的伤害越大 —— 而索引正在变大。 + +--- + +## 3. 根因分层 + +| 层 | 陈述 | 对应发现 | +|---|---|---| +| **L0 表层** | 违反下限终止构建,且终止在一条误导性的 `not found` 上 | F3 | +| **L1 机制** | 兼容性只在「读描述符」处检查;**刷新与下载都不检查**,所以没有「保留可用快照」这回事 | F2 | +| **L2 分发** | 只有一个 rolling `latest`,指针不带下限 ⇒ 客户端无法选择自己能用的最新快照 | F4 F5 | +| **L3 治理** | 下限的**语义**是「最老可解析客户端」,**维护方式**却是「等于 CI pin」⇒ 它以发布频率上移 | F1 | +| **L4 粒度** | 全索引一个开关,而要求是逐描述符的 ⇒ 一个包的新语法废掉整个目录 | F6 | + +**判据一句话**:*版本下限是一个路由信号,不是一个终止信号。* +客户端与索引不匹配时,正确的反应是**选一个匹配的**,选不到才报错 —— 而不是把手上能用的也丢掉。 + +--- + +## 4. 方案 + +### 4.0 核心机制:**索引历史线 + 兼容性优先选择** + +一句话:**索引不是一个 rolling 指针,而是一条带版本下限的历史线; +客户端每次刷新时,从最新往回找,取第一个自己满足下限的快照。** + +不匹配从「错误」变成「选择」——这是整份方案的骨架,其余各项都是为它服务或为它兜底。 + +#### 解析算法 + +``` +refresh(index_name, own_version): + hist = fetch_history(index_name) # 新 → 旧,含每条的 min_mcpp + latest = hist[0] + + if own_version >= latest.min_mcpp: + return use(latest) # 常规路径,与今天一致 + + # 最新的用不了 —— 不是错误,是一次路由 + notice(latest, own_version) # 见下方文案 + for snap in hist: # 继续往回找 + if own_version >= snap.min_mcpp: + return use(snap) # 「当前版本兼容下的最新 index」 + + # 整条线都不兼容(只可能发生在极老的客户端) + keep_existing_or_fail() # → R2 / R3 +``` + +判据是**单调**的:`min_mcpp` 沿历史线只增不减,所以一旦找到第一个满足的就是最优解, +不需要遍历全部 313 条。 + +#### 用户看到的(正是提问描述的那种) + +``` +note: the newest index (2026.8.3.3) requires mcpp >= 2026.8.3.3; this is mcpp 0.0.109. + Using the newest index your version supports: 0.0.109-era (indexed 2026-07-27). + To pick up newer packages, upgrade mcpp: + xlings update mcpp (or: curl -fsSL | bash) +``` + +三件事一次说清:**为什么没拿最新**、**实际用了哪个**、**怎么升级**。 +构建**继续进行**,退出码 0。 + +它只在「选择发生了降级」时打印一次,常规路径完全静默 —— 否则每次构建刷屏, +用户很快就学会无视它,那这条提示就白写了。 + +#### 历史线从哪来 + +F5 已经证明**分发侧不需要新建任何东西**:`xlings-res/xim-index` 的 rolling `latest` +release 里 313 个 artifact 都还在。缺的只是「每条的 `min_mcpp` 是多少」这份元数据。 + +指针从「一条」变成「一条 + 历史」: + +```json +{ + "format_version": 2, + "index_name": "xim", + "latest": { "index_version": "20e53c6", "min_mcpp": "2026.8.3.3", + "generated_at": "2026-08-03T10:14:21Z", "artifact": { … } }, + "history": [ + { "index_version": "20e53c6", "min_mcpp": "2026.8.3.3", "generated_at": "…", "artifact": { … } }, + { "index_version": "0adb288", "min_mcpp": "0.0.109", "generated_at": "…", "artifact": { … } }, + { "index_version": "7fef5ec", "min_mcpp": "0.0.108", "generated_at": "…", "artifact": { … } } + ] +} +``` + +- `min_mcpp` 由发布脚本从**树内 `index.toml`** 机械投影,不手写 —— + 权威仍在树内,指针只是投影,冲突以树内为准(与原设计 L2 的定性一致)。 +- `history` 只需保留**下限发生变化的那些拐点**,不必是全部 313 条: + 同一 `min_mcpp` 的连续快照里只有最新那条有意义。按 F1 的历史, + 拐点至今只有 6 个(`0.0.85 / 0.0.87 / 0.0.101 / 0.0.102 / 0.0.108 / 0.0.109 / 2026.8.3.3`)。 + **一条 6 项的历史线就能覆盖整个索引生命周期。** +- **存量 313 个快照没有这份元数据。** 三选一,建议 (b): + (a) 一次性回填(逐个解包读 `index.toml`); + (b) **只为拐点回填**(6 条,人工可核,成本几分钟); + (c) 缺失 ⇒ 视为「未知,试一次」——不推荐,把确定性换成了下载。 + +#### ⚠️ 谁来执行「回溯选择」:mcpp 今天**做不到**,这是本方案最硬的约束 + +自然的想法是「这全是客户端的事,mcpp 自己解决就好」。核实下来不成立: + +**mcpp 不下载索引。** `index_management.cppm:148` 的刷新就是 + +```cpp +int rc = mcpp::xlings::update_index(xlEnv); // → 跑 `xlings update` +``` + +artifact / git 的同步全部发生在 **xlings** 进程里。mcpp 对索引传输的全部控制权, +是它写进 `.xlings.json` 的 `index_repos` 条目 —— 而那个结构 + +```cpp +struct SeedRepo { + std::string name; + std::string url; + std::string artifact; // artifact 源基址 + std::string source; // "auto" | "artifact" | "git" +}; +``` + +**没有版本/rev 字段**。索引版本串在 mcpp 里还被显式标注为 *OPAQUE BY CONTRACT* +(`xlings.cppm:370`)。也就是说:**mcpp 无法要求「给我索引版本 X」**, +它只能说「去同步」,然后接受 xlings 给的那一份。 + +⇒ 4.0 的回溯选择需要三选一,按代价排序: + +| 路径 | 内容 | 代价 | +|---|---|---| +| **(a) 推荐** | xlings 增加「枚举索引快照 + 按版本同步」的能力,mcpp 在 `.xlings.json` 里下发(`SeedRepo` 加一个 pin 字段)。**已提 [openxlings/xlings#476](https://github.com/openxlings/xlings/issues/476)** | 跨仓协作一次,之后两边都干净 | +| (b) | mcpp 自己接管索引 artifact 的下载与解包 | 与 xlings 职责重叠、镜像/CDN/校验逻辑要重写一遍 | +| (c) | 只对 mcpp 自己的索引(`mcpplibs/mcpp-index`)实现,xim 索引靠 R2 兜底 | 覆盖面不完整,但**不需要任何跨仓改动** | + +**这不影响 R2。** R2(保留/回滚到上一个可用快照)**完全在 mcpp 侧、今天就能做**: +刷新后检查下限,违反就把索引目录回滚到刷新前的备份。它不需要选择任何版本, +只需要「不要用坏的覆盖好的」。**「变砖 → 可用 + 提示升级」这一步不依赖任何人。** + +#### git 传输的索引怎么办 + +`mcpp-index` 同时是一个 git 仓库(`[indices] git =`)。历史线在那里是**天然存在**的: +`index.toml` 的每一次 `min_mcpp` 变更就是一个拐点 commit。等价实现是 +**按 commit 回溯**取第一个兼容的树,或(更省事、更可审计)由索引仓库为每个拐点打 tag: + +``` +index/min-mcpp-0.0.109 +index/min-mcpp-2026.8.3.3 +``` + +客户端 `git fetch --tags` 后按 tag 选择即可,无需 clone 全史。 +本地 `path =` 索引不参与历史线(它就是用户自己的目录),保持现状:违反 ⇒ R1 报错。 + +#### 离线 + +历史线的选择结果必须**落盘记录**(哪个 index_version、什么下限、为什么选它)。 +离线时不联网、不重新选择,直接用上次选中的快照 —— 与 `index_refresh` 现有的 +offline-first 判据(「所有解析输入都在磁盘」)一致,不新增网络依赖。 + +--- + +以下各项按「是否为 4.0 所必需」排序。 + +### R1(必须)· 违反下限必须是「索引不可用」,不能退化成「包不存在」 + +`package_fetcher` 的读取路径要能区分三种结果,而不是两种: + +```cpp +enum class DescriptorLookup { Found, Absent, IndexUnusable }; +``` + +- `IndexUnusable` **不得**触发 legacy 地址回退、不得被其它 repo 的 `Absent` 掩盖、 + 不得被 `index_refresh` 读成「本地没有 ⇒ 该刷新了」(否则形成刷新抖动: + 拉回同一份不兼容的树,再失败,再刷新)。 +- 解析结束时若有索引处于 `IndexUnusable`,**最终错误就是 E0006 本身**。 + 用户看到的第一条与最后一条错误必须是同一件事。 + +4.0 落地后这条路径应当**几乎不可达**(选择阶段就避开了不兼容的树)—— +但它必须存在,因为 `path =` 索引、手工放置的树、以及历史线整条不兼容时仍会走到它。 + +### R2(必须)· 选择失败时保留上一个可用快照,绝不用不可用的覆盖可用的 + +补上 L1 承诺过而没实现的那一半(F2)。即使有了 4.0,这一条仍然必须: +历史线获取失败(网络、镜像、artifact 缺失)时,**手上那份能用的索引就是最后的防线**。 + +1. 下载/解包后、切换生效前,对**暂存的**树跑 `check_index_floor`; +2. 违反 ⇒ 不切换,保留现有索引并 warn; +3. 本机完全没有可用索引时才落到 R3。 + +xim 索引的下载/解包由 **xlings** 执行,不在 mcpp 进程内。在拿到「解包到暂存目录」 +的能力之前,退化实现同样有效:刷新后立即检查,违反则**回滚到刷新前的快照** +(保留一份 `last-compatible` 备份),并在本进程内标记该索引不可再刷新。 + +> 这一条修掉的是最伤人的性质:**刷新让情况变得更糟,且不可逆。** + +### R3(必须)· 无任何可用树时,一次说清楚 + +冷启动且历史线整条不兼容(或不可达)时,只报 E0006,指明**哪个索引、要求什么、本机是什么**, +并给出升级命令。保留 `MCPP_INDEX_FLOOR=ignore` 作为调试逃生舱。 + +### R4(应做,治理)· 下限必须与 CI pin 解耦,并由 CI 证明 + +**优先级说明(修订)**:早先的排序把这条摆得过重。4.0 + R2 落地后,下限乱涨的危害 +从「变砖」降到「旧客户端停在老索引、拿不到新包,但构建正常」。所以这条是 **hygiene, +不是阻塞项** —— 它值得做,因为无声地把所有旧客户端钉在老索引上仍然是损失, +而且这个损失**不再有任何报错提醒任何人**;但它不该排在 R1/R2 前面。 + +1. **改掉 `index.toml` 的维护规则**:从「与 CI pin 一起 bump」改为 + **「只有当某个描述符真的用了旧客户端解析不了、或会静默错解的东西时才抬」**, + 并在 commit 里点名是哪个描述符、哪个特性。 +2. **让 CI 证明它**:索引 CI 现在只用 pin 的 mcpp 跑 lint(只能证明「新版本能解析」)。 + 应当**再用当前 `min_mcpp` 那个版本跑一遍**: + - 旧版本能全部解析 ⇒ **下限不许动**; + - 旧版本解析失败 ⇒ 下限**必须**抬到能解析的最低版本,且 CI 报出是哪个文件。 + 这把「floor first, migration after」从**约定**变成**机器判据**。 +3. **`latest_mcpp` 与 `min_mcpp` 分开**:前者跟 pin 走(「已知最佳」,也是 4.0 提示文案里 + 建议升级到的目标版本 —— F4 指出它目前无人消费,这是它的第一个真实用途), + 后者只在 (2) 判定必须时才动。今天两者恒等,等于把「最佳」当成了「最低」。 + +### R5(应做)· 逐描述符下限 + +原设计以「N places to maintain」拒绝过它。这个理由在贡献者变多后不再成立: +维护成本落在**新增该描述符的那个 PR**(一行), +而现状的成本落在**所有旧客户端**(整个目录不可读,F6)。 + +- 索引级 `min_mcpp` 保留,语义收紧为「解析这棵树的目录结构与通用字段」所需 + (真正的全局语法变更,如 SPEC-001 身份形态迁移); +- 单个描述符可选声明 `mcpp = ">= X"`,客户端不满足时**只跳过该描述符**并 warn。 + +这样「新加一个包」在结构上不可能再废掉整个索引 —— F6 的根治。 +它与 4.0 是互补的:4.0 让**客户端**总能找到能用的快照,R5 让**索引**不必因为一个包而整体后退。 + +### R6(可选)· 版本化通道(原设计 L3) + +`index.toml` 的 `spec` 作为通道键,`mcpp-index-v1-latest` / `-v2-latest`。 +4.0 落地后增量收益不大,**不建议现在做**;但指针形状应留出扩展位(`format_version` 已在)。 + +## 5. 不采纳 + +| 方案 | 理由 | +|---|---| +| 直接把下限检查改成 warning、继续解析 | 下限存在的理由是真的:`0.0.101` 那次是 `compat.opencv` 的 per-OS feature flags 被旧客户端**静默忽略**,产出错误的构建。放行 = 用「静默错误产物」换「明确失败」,方向反了。R1/R2 保住的是**旧快照**,不是**用旧客户端读新描述符**。 | +| 让旧客户端忽略看不懂的字段 | 同上:`0.0.101` 的教训正是「忽略 = 静默错」。 | +| 索引不再抬下限 | 语法要演进。问题不在抬,在**抬的理由**(F1)和**抬之后旧客户端没有退路**(F2)。 | +| 只做 R4(分发侧协商),不做 R2 | 冷启动能救,但**已经装好的用户在 `xlings update` 后依然会被换成不可用的索引**——最伤人的那条路径没修。 | +| 只做 R2,不做 R5 | 旧客户端不再变砖,但下限仍以发布频率上移,**所有人都会持续拿不到新包**,只是不再报错。症状消失,问题还在。 | + +--- + +## 6. 实施顺序(修订) + +排序的依据是**受众**,不是技术难度。 + +### 为什么索引侧那一步删不掉 + +**任何 mcpp 侧修复都救不了已经发出去的 0.0.109。** 修复最早出现在 `2026.8.3.5`, +而现在被卡住的用户手里就是 0.0.109。他们只有两条路:**升级 mcpp**, +或者**索引把下限降回去**。所以: + +- **索引侧回退** = 针对**存量用户**的唯一无痛解(且不需要发 mcpp) +- **mcpp 侧修复** = 针对**未来所有版本偏斜**的根治(但只对未来的版本生效) + +两者受众不同,不是重复劳动。这也是为什么它排第一 —— 不是因为它更重要, +而是因为它是唯一今天就能让人恢复的动作。 + +### 顺序 + +| # | 动作 | 侧 | 受众 / 效果 | 依赖 | +|---|---|---|---|---| +| 1 | `min_mcpp` 回退到真实需要的值(`latest_mcpp` 保留 `2026.8.3.3`) | 索引 | **存量 0.0.109 用户立刻恢复** | 无 | +| 2 | R1 + R3:错误语义(`IndexUnusable` ≠ `Absent`) | mcpp | 可诊断性;4.0 的兜底 | 无 | +| 3 | **R2:刷新闸 + 回滚到上一个可用快照** | mcpp | **「变砖 → 可用 + 提示升级」** | **无** | +| 4 | 4.0 历史线:先走路径 (c)(只覆盖 mcpp 自己的索引) | mcpp | 「拉取当前版本兼容下的最新 index」 | 无 | +| 5 | 4.0 完整覆盖:xlings 加索引版本 pin,`SeedRepo` 下发([xlings#476](https://github.com/openxlings/xlings/issues/476)) | xlings + mcpp | xim 索引也进入历史线 | 跨仓,**不阻塞 1–4** | +| 6 | R4:索引 CI 双版本 lint + 维护规则改写 | 索引 | hygiene:阻止下限无理由上移 | 无 | +| 7 | R5:逐描述符下限 | mcpp + 索引 | 一个包不再废掉整个目录 | 无 | +| 8 | R6:版本化通道 | — | 视生态规模再议 | — | + +**第 3 步是性价比最高的一步**:纯 mcpp 侧、不依赖任何人、不需要任何新协议, +就把最伤人的性质(刷新让情况变得更糟且不可逆)修掉。 +第 4 步给出你要的「历史线」语义,第 5 步才把它铺满整个生态。 + +--- + +## 7. 验证 + +- **回归 e2e**:fixture 索引 `min_mcpp = "9.9.9"`。当前断言是「build fails」; + R2 之后应改为**「保留旧索引、构建成功、打出升级 warning」**,并新增一条 + 「本机无任何兼容索引 ⇒ 只报 E0006,且 stderr 中不出现 `not found`」。 +- **F3 的判据要写成断言**:违反下限的构建,其**最后一条**错误必须包含 `E0006`。 + 这条断言正是这次(以及 `install_pinned_mcpp.sh` 记载的上一次)缺失的那个。 +- **R5(2) 的自证**:在索引 CI 里故意把 `min_mcpp` 调低一档,双版本 lint 必须变红。 + +--- + +## 8. 附:这次事件的时间线(供复盘) + +| 时间 | 事件 | +|---|---| +| 2026-07-08 | 设计确立 L1/L2/L3;L2、L3 判为「operationally premature」推迟 | +| 2026-07-08 | L1 落地,但只实现了 choke-point 检查;**「保留上一个兼容快照」未实现**(F2) | +| 2026-07-21 | 下限 `0.0.87` → `0.0.101`,理由真实(opencv feature flags 被静默忽略) | +| 2026-07-26 | 下限 → `0.0.108`/`0.0.109`,理由真实(SPEC-001 身份迁移) | +| 2026-08-03 | 下限 `0.0.109` → `2026.8.3.3`,**理由不成立**:pin 因 mcpp#336(macOS) 上移,下限机械跟随(F1) | +| 2026-08-03 | 用户 `0.0.109` 变砖,且报错指向 `not found`(F3) | + +前两次 bump 是这个机制**正确工作**的样子;第三次是它**语义漂移**的样子。 +区别不在于机制,而在于「谁来证明这次 bump 是必要的」—— 目前没有人,也没有机器。 diff --git a/.agents/skills/mcpp-release/SKILL.md b/.agents/skills/mcpp-release/SKILL.md index 7a656eac..84c65b7b 100644 --- a/.agents/skills/mcpp-release/SKILL.md +++ b/.agents/skills/mcpp-release/SKILL.md @@ -31,7 +31,7 @@ mcpp 有 **三个持久化版本位置**,以及 `ci-fresh-install` 的一个 **第一组:正在构建的版本**(发布时改,走 bump PR) 1. `mcpp.toml` → `[package].version` — 构建系统读取的项目版本,release.yml 由它推导 tag -2. `src/toolchain/fingerprint.cppm` → `MCPP_VERSION` — 编译期硬编码常量(`--version` 输出、BMI 指纹、E0006 索引底线比较) +2. `src/version.cppm` → `MCPP_VERSION` — 编译期硬编码常量(`--version` 输出、BMI 指纹、E0006 索引底线比较) 这两处必须**在同一个 commit 里**一起改:`tests/e2e/01_help_and_version.sh` 交叉比对 `mcpp.toml` 与 `mcpp --version`,只改一处 CI 立刻红。 @@ -89,7 +89,7 @@ NEW_VERSION="2026.7.27.1" git checkout -b "chore/bump-$NEW_VERSION" sed -i "s/^version.*=.*/version = \"$NEW_VERSION\"/" mcpp.toml -sed -i "s/MCPP_VERSION = \".*\"/MCPP_VERSION = \"$NEW_VERSION\"/" src/toolchain/fingerprint.cppm +sed -i "s/MCPP_VERSION = \".*\"/MCPP_VERSION = \"$NEW_VERSION\"/" src/version.cppm # 校验:mcpp.toml 与 MCPP_VERSION 相等,.xlings.json 不领先于正在构建的版本。 bash .github/tools/check_version_pins.sh @@ -309,7 +309,7 @@ gh workflow run release.yml --ref "v$NEW_VERSION" | 文件 | 版本相关内容 | |------|-------------| | `mcpp.toml` | `version = "X.Y.Z"` — 项目版本,release.yml 由它推导 tag | -| `src/toolchain/fingerprint.cppm` | `MCPP_VERSION = "X.Y.Z"` — 编译期版本常量 | +| `src/version.cppm` | `MCPP_VERSION = "X.Y.Z"` — 编译期版本常量 | | `.xlings.json` | `workspace.mcpp` — CI bootstrap 装哪个 mcpp(发布**后**才 bump) | | `.github/workflows/ci-fresh-install.yml` | `MCPP_PIN` — 由 `wait-index` 从最新 release 推导,**从不手工 bump** | | `src/xlings.cppm` | `kXlingsVersion` — xlings pin 的**唯一真源** | diff --git a/.github/tools/check_version_pins.sh b/.github/tools/check_version_pins.sh index 604cbcd4..36b6fecc 100755 --- a/.github/tools/check_version_pins.sh +++ b/.github/tools/check_version_pins.sh @@ -100,13 +100,13 @@ done < <(find .github -type f \( -name '*.yml' -o -name '*.yaml' -o -name '*.sh' # ── 2. mcpp's own version ───────────────────────────────────────────────── v_toml=$(awk -F '"' '/^version[[:space:]]*=/{print $2; exit}' mcpp.toml) -v_src=$(grep -oE 'MCPP_VERSION[[:space:]]*=[[:space:]]*"[^"]+"' src/toolchain/fingerprint.cppm \ +v_src=$(grep -oE 'MCPP_VERSION[[:space:]]*=[[:space:]]*"[^"]+"' src/version.cppm \ | grep -oE '"[^"]+"' | tr -d '"' | head -1) v_xl=$(grep -oE '"mcpp"[[:space:]]*:[[:space:]]*"[^"]+"' .xlings.json \ | grep -oE '"[^"]+"$' | tr -d '"' | head -1) -note "mcpp version: building=$v_toml (fingerprint=$v_src) bootstrap pin=$v_xl" +note "mcpp version: building=$v_toml (src/version.cppm=$v_src) bootstrap pin=$v_xl" -for n in "mcpp.toml:$v_toml" "src/toolchain/fingerprint.cppm:$v_src" \ +for n in "mcpp.toml:$v_toml" "src/version.cppm:$v_src" \ ".xlings.json:$v_xl"; do [ -n "${n##*:}" ] || bad "${n%:*} — could not read the mcpp version" done @@ -131,7 +131,7 @@ fi # same number by definition — release.yml derives the tag from the former # and the smoke test greps the latter out of `mcpp --version`. [ -z "$v_src" ] || [ "$v_src" = "$v_toml" ] \ - || bad "src/toolchain/fingerprint.cppm has '$v_src' but mcpp.toml has '$v_toml'" + || bad "src/version.cppm has '$v_src' but mcpp.toml has '$v_toml'" # (b) The version BOOTSTRAPPED FROM (.xlings.json) names a mcpp that is already # published, and is NOT required to equal the version being built. It diff --git a/CHANGELOG.md b/CHANGELOG.md index cf6240cd..3a7d4bd5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,32 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.8.3.5] — 2026-08-03 + +### 修复 + +- **索引侧的改动不再能让 mcpp 从「能用」变成「不能用」。** 一个索引树可以声明客户端版本下限(`index.toml` `min_mcpp`)。此前 `xlings update` 会**原地覆盖**本地索引,一旦上游抬了下限,一台一分钟前还好好的机器就报 + + ``` + error: index requires mcpp >= 2026.8.3.3 but this is mcpp 0.0.109 [E0006] + error: ... package 'compat:compat.libarchive@3.8.7' not found + ``` + + 而且**没有退路** —— 旧的、能用的那份树已经被覆盖掉了。**刷新本身就是把机器弄坏的那个动作。** + + 刷新现在是**单调**的:判据是「刷新前能用 ⇒ 刷新后必须仍能用」。`mcpp::xlings::update_index`(全进程唯一的刷新入口)在同步前归档每一棵可读的索引树,同步后复核;变得不可读就回滚,并提示可以升级 mcpp 拿新包。2026-07-08 的索引设计里写过这个行为("staged refresh keeps the last compatible snapshot"),**从未被实现** —— 下限只在描述符读取处检查过一次,刷新路径对它一无所知。 + +- **不再把「这个索引读不了」伪装成「这个包不存在」。** 违反下限会让该树的每次描述符读取返回空,与「包确实不在」不可区分。后果有两层:终止构建的那条错误是 `dependency '...': not found`,**把责任推给包**(指向发布/命名),而真正的答案(升级 mcpp)早已滚出屏幕;同一个无法区分的 miss 还会喂给刷新策略,被读成「本地缺东西,去拉」—— **不可用状态自己在驱动重复刷新**。 + + 「这个索引不可读」现在是一个可查询的事实:刷新策略据此不再徒劳重试,而每一处终止构建的 not-found 都会带上真正的原因。 + +### 改进 + +- **`mcpp cache`/索引之外的一处分层修正:`MCPP_VERSION` 移到叶子模块 `src/version.cppm`。** 它此前住在 `mcpp.toolchain.fingerprint` 里,而后者 `import mcpp.toolchain.detect` → `import mcpp.xlings`,于是「这个二进制是什么版本」这件事**传递依赖了整个工具链探测与包管理子系统**。这不是洁癖:`mcpp.pm.index_contract` 只为一次比较需要这个常量,却因此依赖上 xlings,反过来使得 xlings **不可能**依赖 index_contract —— 而刷新守卫恰恰必须装在那里。**编译器报的那个循环依赖,就是分层在告诉我们常量放错了地方。** 版本号的唯一真源随之移动;`check_version_pins.sh` 与发布文档同步更新。 + +- **补上了从未存在的端到端覆盖。** 下限行为此前只有纯谓词单测(`floor_violation()`),`tests/e2e/` 里一条都没有 —— 这正是上面两个缺陷能长期存在的原因:谓词是对的,**谓词周围的行为从没被端到端跑过**。新增 `tests/e2e/185_index_floor_degrades.sh`(错误必须点名真正的原因、不可用索引不得波及健康索引、逃生舱仍有效)与 `tests/unit/test_index_snapshot.cpp`(9 条,含「刷新弄坏索引必须回滚」)。 + + ## [2026.8.3.4] — 2026-08-03 ### 修复 diff --git a/docs/09-release.md b/docs/09-release.md index 3408eddb..3f575b4f 100644 --- a/docs/09-release.md +++ b/docs/09-release.md @@ -11,7 +11,7 @@ of those commit messages contains a misdiagnosis that is corrected in §5. | Site | Group | Moves when | |---|---|---| | `mcpp.toml` `[package].version` | **being built** | you start work on a new version | -| `src/toolchain/fingerprint.cppm` `MCPP_VERSION` | **being built** | same commit as above (compiled-in copy) | +| `src/version.cppm` `MCPP_VERSION` | **being built** | same commit as above (compiled-in copy) | | `.xlings.json` `[workspace].mcpp` | **bootstrapped from** | separately, *after* a release is installable | | `ci-fresh-install.yml` `MCPP_PIN` | **version under test** | **nothing — it is derived at run time** (§5) | diff --git a/docs/zh/09-release.md b/docs/zh/09-release.md index e85a2b73..0c1577ec 100644 --- a/docs/zh/09-release.md +++ b/docs/zh/09-release.md @@ -11,7 +11,7 @@ | 位置 | 组 | 何时变 | |---|---|---| | `mcpp.toml` `[package].version` | **正在构建的** | 开始做新版本时 | -| `src/toolchain/fingerprint.cppm` `MCPP_VERSION` | **正在构建的** | 与上一行同一个 commit(编译进二进制的副本) | +| `src/version.cppm` `MCPP_VERSION` | **正在构建的** | 与上一行同一个 commit(编译进二进制的副本) | | `.xlings.json` `[workspace].mcpp` | **自举起点** | 单独地、在某个版本**已可安装之后** | | `ci-fresh-install.yml` `MCPP_PIN` | **被测版本** | **不变 —— 运行时推导**(§5) | diff --git a/mcpp.toml b/mcpp.toml index fbd2b5d5..b557aa8e 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.3.4" +version = "2026.8.3.5" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index ea67c6ba..c6f416d6 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -41,6 +41,7 @@ import mcpp.fetcher; import mcpp.fetcher.progress; import mcpp.pm.resolver; import mcpp.pm.index_spec; +import mcpp.pm.index_contract; import mcpp.pm.index_route; import mcpp.pm.index_refresh; import mcpp.pm.mangle; @@ -819,6 +820,20 @@ std::string git_cache_head(const std::filesystem::path& gitRoot) { // extraTargets: additional Target entries (e.g. synthetic test targets) // appended to the manifest before the modgraph runs. // overrides: --target / --static. +namespace { +// A dependency that "cannot be found" while an index is unreadable is almost +// never missing — it is unreachable, and the two need different actions from +// the user (publish it vs upgrade mcpp). The floor error is printed when the +// index is first opened, which can be hundreds of lines earlier; the message +// that STOPS the build has to carry the cause, because that is the one a user +// reads. See mcpp::pm::unusable_index_hint. +std::string with_index_cause(std::string msg) { + if (auto hint = mcpp::pm::unusable_index_hint(); !hint.empty()) + msg += "\n" + hint; + return msg; +} +} // namespace + export std::expected prepare_build(bool print_fingerprint, bool includeDevDeps = false, @@ -1975,7 +1990,8 @@ prepare_build(bool print_fingerprint, auto candidates = dependencyCoordinates(spec, depName); if (candidates.empty()) { return std::unexpected( - std::format("dependency '{}' has no lookup candidates", depName)); + with_index_cause(std::format( + "dependency '{}' has no lookup candidates", depName))); } auto selected = candidates.front(); @@ -2146,9 +2162,9 @@ prepare_build(bool print_fingerprint, auto luaContent = readLuaContent(); if (idxSpec && idxSpec->is_local() && !luaContent) { auto indexPath = mcpp::config::resolve_project_index_path(*root, *idxSpec); - return std::unexpected(std::format( + return std::unexpected(with_index_cause(std::format( "dependency '{}': not found in local index at '{}'", - depName, indexPath.string())); + depName, indexPath.string()))); } auto findRawInstalled = [&]() -> std::optional { @@ -2439,8 +2455,8 @@ prepare_build(bool print_fingerprint, if (!luaContent) { luaContent = readLuaContent(); } - if (!luaContent) return std::unexpected(std::format( - "dependency '{}': index entry not found in local clone", depName)); + if (!luaContent) return std::unexpected(with_index_cause(std::format( + "dependency '{}': index entry not found in local clone", depName))); auto field = mcpp::manifest::extract_mcpp_field(*luaContent); // 0.0.6+: read explicit namespace from xpkg lua if present. diff --git a/src/pm/index_contract.cppm b/src/pm/index_contract.cppm index 1ee3d4a9..0637da8c 100644 --- a/src/pm/index_contract.cppm +++ b/src/pm/index_contract.cppm @@ -22,7 +22,7 @@ export module mcpp.pm.index_contract; import std; import mcpp.libs.toml; import mcpp.version_req; -import mcpp.toolchain.fingerprint; // MCPP_VERSION +import mcpp.version; // MCPP_VERSION (leaf — see that module) export namespace mcpp::pm { @@ -44,13 +44,55 @@ read_index_contract(const std::filesystem::path& indexRoot); std::optional floor_violation(std::string_view minMcpp, std::string_view ownVersion); +// Pure predicate — no reporting, no registration, no dedup. For callers that +// need to ask "would this tree be usable?" without the side effects of +// check_index_floor (the refresh guard asks it twice per refresh). +bool index_usable(const std::filesystem::path& indexRoot); + // Open-time check for an index tree. Combines read + floor + escape // hatch + once-per-root deduplication of the (expensive to spam) error. // Returns the violation message the FIRST time a too-new tree is opened; -// nullopt otherwise. +// nullopt otherwise. Also RECORDS the fact (see below). std::optional check_index_floor(const std::filesystem::path& indexRoot); +// ── "this index is unusable" as a first-class, queryable fact ────────── +// +// A floor violation makes every descriptor read from that tree return nothing, +// which is indistinguishable from "the package genuinely is not in this index" +// at the call site. Two things downstream need to tell them apart: +// +// * the refresh policy — a miss caused by an unusable index will NOT be +// fixed by fetching the same tree again, and treating it as a normal miss +// makes the unusable state drive repeated refreshes of itself; +// * the final error — "not found" names neither the version nor the floor, +// so the message that stops the build has to reach back for the real cause. +// +// Process-global on purpose: the fact depends only on (the tree on disk, this +// binary's version), and neither can change within a process. Recording it is +// what lets the two consumers above stay honest without threading a tri-state +// through every read_xpkg_lua entry point and all of their callers. +struct UnusableIndex { + std::filesystem::path root; + std::string message; // the full E0006 text, ready to print +}; + +// True when any index tree opened in this process failed its floor check. +bool any_index_unusable(); + +// True when THIS tree failed (exact root match). +bool index_marked_unusable(const std::filesystem::path& indexRoot); + +// Every index that failed, in first-seen order. +std::vector unusable_indexes(); + +// One line for appending to an unrelated failure ("… and by the way, an index +// was unusable, which is probably why"). Empty when nothing was unusable. +std::string unusable_index_hint(); + +// Testing only: forget everything recorded so far. +void reset_unusable_indexes_for_test(); + } // namespace mcpp::pm namespace mcpp::pm { @@ -93,6 +135,51 @@ floor_violation(std::string_view minMcpp, std::string_view ownVersion) minMcpp, ownVersion); } +bool index_usable(const std::filesystem::path& indexRoot) +{ + if (const char* v = std::getenv("MCPP_INDEX_FLOOR"); + v && std::string_view(v) == "ignore") + return true; + auto c = read_index_contract(indexRoot); + if (!c) return true; // no contract → no constraint + return !floor_violation(c->minMcpp, mcpp::MCPP_VERSION); +} + +namespace { +// See the header comment on UnusableIndex for why this is process-global. +std::vector& unusable_registry() { + static std::vector reg; + return reg; +} +} // namespace + +bool any_index_unusable() { return !unusable_registry().empty(); } + +bool index_marked_unusable(const std::filesystem::path& indexRoot) { + for (auto& u : unusable_registry()) + if (u.root == indexRoot) return true; + return false; +} + +std::vector unusable_indexes() { return unusable_registry(); } + +std::string unusable_index_hint() { + auto& reg = unusable_registry(); + if (reg.empty()) return {}; + // Name the index, not just the fact: with several repos configured, "an + // index was too new" leaves the reader guessing which one to act on. + std::string s = "note: this resolve ran with an index this mcpp cannot read:\n"; + for (auto& u : reg) { + s += " " + u.root.string() + "\n"; + } + s += " Packages served by it were reported as not found. See the " + "[E0006] error above,\n" + " or run `mcpp explain E0006`."; + return s; +} + +void reset_unusable_indexes_for_test() { unusable_registry().clear(); } + std::optional check_index_floor(const std::filesystem::path& indexRoot) { @@ -100,14 +187,19 @@ check_index_floor(const std::filesystem::path& indexRoot) v && std::string_view(v) == "ignore") return std::nullopt; - // Once per root per process: the same index is opened many times in a - // single resolve; report the violation once, stay quiet after. - static std::set reported; auto c = read_index_contract(indexRoot); if (!c) return std::nullopt; - auto violation = floor_violation(c->minMcpp, mcpp::toolchain::MCPP_VERSION); + auto violation = floor_violation(c->minMcpp, mcpp::MCPP_VERSION); if (!violation) return std::nullopt; - if (!reported.insert(indexRoot).second) return std::nullopt; + + // Record BEFORE the dedup return: the fact must be queryable no matter how + // many times this root is opened, while the message is printed only once. + // Deriving "was anything unusable?" from "did we print?" is what made the + // second and later reads indistinguishable from an ordinary miss. + if (!index_marked_unusable(indexRoot)) + unusable_registry().push_back({indexRoot, *violation}); + else + return std::nullopt; // already reported — stay quiet return violation; } diff --git a/src/pm/index_refresh.cppm b/src/pm/index_refresh.cppm index eddd14b0..d770fa31 100644 --- a/src/pm/index_refresh.cppm +++ b/src/pm/index_refresh.cppm @@ -41,6 +41,7 @@ import mcpp.log; import mcpp.platform; import mcpp.platform.axis; import mcpp.pm.dep_spec; +import mcpp.pm.index_contract; import mcpp.pm.index_route; import mcpp.pm.resolver; import mcpp.ui; @@ -60,6 +61,7 @@ enum class RefreshReason { SuppressedDisabled, // [index] auto_refresh = false SuppressedDebounce, // refreshed moments ago; upstream simply lacks it SuppressedInconclusive, // a miss here proves nothing (see header) + SuppressedIndexUnusable,// the index that would answer is too new to read }; struct RefreshPolicy { @@ -186,6 +188,8 @@ std::string_view reason_text(RefreshReason r) { case RefreshReason::SuppressedDisabled: return "[index] auto_refresh = false"; case RefreshReason::SuppressedDebounce: return "index was just refreshed"; case RefreshReason::SuppressedInconclusive: return "no index can refute this"; + case RefreshReason::SuppressedIndexUnusable: + return "an index requires a newer mcpp — refreshing cannot help"; } return ""; } @@ -255,6 +259,18 @@ RefreshDecision decide_for_dependency(const IndexRoute& route, if (!d.shouldRefresh) return d; // nothing to suppress + // An index this binary cannot READ makes every descriptor lookup in it come + // back empty, and that miss is indistinguishable from "the package is not + // there" at the call site. Refreshing would fetch the same unreadable tree + // again — so the miss would repeat, and the unusable state would drive + // repeated refreshes of itself. Stop here and let the E0006 error stand as + // the explanation. + if (mcpp::pm::any_index_unusable()) { + d.shouldRefresh = false; + d.reason = RefreshReason::SuppressedIndexUnusable; + return d; + } + // 7. Opt-outs, in order of authority: the user's flag, the machine's // config, then the "we just did this" guard. if (policy.offline) { d.shouldRefresh = false; d.reason = RefreshReason::SuppressedOffline; return d; } @@ -281,6 +297,11 @@ RefreshDecision decide_for_miss(const RefreshPolicy& policy, std::string_view subject) { RefreshDecision d; d.subject = std::string(subject); + // See decide(): a refresh cannot fix an index this binary cannot read. + if (mcpp::pm::any_index_unusable()) { + d.reason = RefreshReason::SuppressedIndexUnusable; + return d; + } if (policy.offline) { d.reason = RefreshReason::SuppressedOffline; return d; } if (!policy.autoRefresh) { d.reason = RefreshReason::SuppressedDisabled; return d; } // Same debounce as the build path: two `mcpp add` typos in a row should not diff --git a/src/pm/index_snapshot.cppm b/src/pm/index_snapshot.cppm new file mode 100644 index 00000000..30f7f4b6 --- /dev/null +++ b/src/pm/index_snapshot.cppm @@ -0,0 +1,337 @@ +// mcpp.pm.index_snapshot — local, known-good copies of index trees, and the +// guard that makes a refresh MONOTONE. +// +// THE INVARIANT +// +// An index-side change must never take mcpp from "works" to "does not work". +// +// It did not hold. An index tree can declare a client-version floor +// (`index.toml` `min_mcpp`, mcpp.pm.index_contract). When the published index +// raises that floor, `xlings update` replaces the local tree in place with one +// this binary cannot read, every descriptor read starts returning nothing, and +// the build dies — on a machine that was working sixty seconds earlier. There +// was no backup, so there was no way back either: the refresh is what broke it. +// +// The 2026-07-08 index design specified the cure ("staged refresh keeps the +// last compatible snapshot") and it was never implemented — the floor was +// checked in exactly one place, the descriptor reader, and the refresh path +// knew nothing about it. +// +// WHY BACKUP-AND-ROLLBACK RATHER THAN STAGE-AND-SWAP +// +// mcpp does not fetch indexes. `mcpp::xlings::update_index` shells out to +// `xlings update`, which rewrites the tree in place; mcpp cannot ask it to +// unpack somewhere else (the `index_repos` entry it seeds carries name / url / +// artifact / source and no version or destination). So the sequence is +// archive → let the refresh happen → judge the result → restore if it got +// worse. Same guarantee, no cross-repo dependency. +// +// The judgement is deliberately "did it get WORSE", not "is it good": +// +// usable before && !usable after → restore +// +// A tree that was already unusable has nothing to roll back to, and restoring +// an equally-unusable snapshot over a fresh one would only make the next +// refresh redo the same work. +// +// WHY KEEP MORE THAN ONE +// +// One backup answers "undo the refresh that just happened". A short history +// answers "give me the newest snapshot this binary can actually read", which +// is the same question the publishing side would answer with a version +// negotiation protocol (openxlings/xlings#476) — locally, for free, and today. +// It only covers snapshots this machine has seen, which is the overwhelmingly +// common case for a development box or a CI runner with a warm cache; a fresh +// install that meets a too-new index has no local history and correctly falls +// through to "upgrade mcpp". +// +// COST +// +// Index trees are small (measured: 912 KB for mcpplibs, 2.2 MB for +// xim-pkgindex) and archiving uses hard links where the filesystem allows it, +// so a snapshot costs approximately one directory walk and no extra bytes. + +export module mcpp.pm.index_snapshot; + +import std; +import mcpp.pm.index_contract; + +export namespace mcpp::pm::index_snapshot { + +// How many known-good snapshots to keep per index tree. Small on purpose: the +// value of the history is "the last few floors", and floors move rarely (the +// mcpp index has changed its floor 6 times in its entire life). +inline constexpr std::size_t kKeepPerIndex = 5; + +// /.index-snapshots/// +std::filesystem::path snapshots_root(const std::filesystem::path& dataRoot); +std::filesystem::path snapshot_dir(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir); + +// Identity of the tree currently in `indexDir`, taken from the marker xlings +// writes (`.xlings-index-version`). OPAQUE BY CONTRACT — a short sha for the +// artifact transport, a version string for others; never parsed, only compared +// and used as a directory name. Empty when the marker is absent, in which case +// the caller falls back to a content-independent name. +std::string snapshot_id(const std::filesystem::path& indexDir); + +// Archive `indexDir` under the snapshots root. No-op (returns false) when the +// tree is unusable — a snapshot exists to be restored, and restoring an +// unusable tree helps nobody. Overwrites an existing snapshot with the same id. +bool archive(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir); + +// Restore a specific snapshot over `indexDir`. The destination is replaced. +bool restore(const std::filesystem::path& snapshotPath, + const std::filesystem::path& indexDir); + +// Newest-first list of archived snapshots for one index tree. +std::vector +list_snapshots(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir); + +// The newest archived snapshot this binary can actually read, if any. +std::optional +newest_usable(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir); + +// Drop all but the newest `keep` snapshots of one index tree. +void prune(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir, + std::size_t keep = kKeepPerIndex); + +// Index trees directly under `dataRoot`: a directory containing `pkgs/`. +std::vector +index_dirs(const std::filesystem::path& dataRoot); + +// What a guarded refresh did, for the caller to report. +struct GuardOutcome { + // Index trees that were usable before the refresh and unusable after, and + // were therefore rolled back. + std::vector rolledBack; + // Index trees restored from an older snapshot because the refreshed tree + // was unusable and no pre-refresh tree existed to keep. + std::vector recovered; + // Index trees left unusable — nothing local could serve them. + std::vector stillUnusable; + + bool degraded() const { + return !rolledBack.empty() || !recovered.empty() || !stillUnusable.empty(); + } +}; + +// Run `refresh` with the monotonicity guarantee over every index tree under +// `dataRoot`. Returns the refresh's own exit code; `out` describes what the +// guard had to do. +// +// `refresh` is a callable so this module stays free of any dependency on the +// xlings process layer — and so it is testable without a network. +int guarded_refresh(const std::filesystem::path& dataRoot, + const std::function& refresh, + GuardOutcome& out); + +} // namespace mcpp::pm::index_snapshot + +namespace mcpp::pm::index_snapshot { + +namespace { + +// A REAL copy. Not hard links. +// +// Hard links were the obvious optimisation — an index tree is thousands of +// small files and a snapshot that costs real bytes is a reason not to take one +// — and they are wrong here, which the round-trip test caught immediately: a +// hard-linked snapshot ALIASES the live tree. Anything that rewrites an index +// file in place (a truncating write, a tar extraction over an existing path) +// writes straight through the link and destroys the backup. The one event this +// module exists to survive is precisely "something replaced the tree", so a +// snapshot that shares inodes with the tree is not a snapshot at all. +// +// The bytes are cheap in absolute terms (measured: 912 KB for mcpplibs, 2.2 MB +// for xim-pkgindex; five kept snapshots ≈ 15 MB) and `prune` bounds the total. +bool copy_tree(const std::filesystem::path& from, + const std::filesystem::path& to) +{ + std::error_code ec; + std::filesystem::remove_all(to, ec); + ec.clear(); + std::filesystem::create_directories(to.parent_path(), ec); + + ec.clear(); + std::filesystem::copy(from, to, + std::filesystem::copy_options::recursive + | std::filesystem::copy_options::overwrite_existing + | std::filesystem::copy_options::skip_symlinks, ec); + return !ec; +} + +// Directory mtime, as a sortable integer. Only ever compared against other +// snapshots of the same index, so file_clock's non-Unix epoch is irrelevant. +std::int64_t mtime_of(const std::filesystem::path& p) { + std::error_code ec; + auto t = std::filesystem::last_write_time(p, ec); + if (ec) return 0; + return t.time_since_epoch().count(); +} + +std::string sanitize_component(std::string s) { + for (auto& c : s) { + const bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') + || (c >= '0' && c <= '9') || c == '.' || c == '-' || c == '_'; + if (!ok) c = '_'; + } + if (s.empty()) s = "unknown"; + return s; +} + +} // namespace + +std::filesystem::path snapshots_root(const std::filesystem::path& dataRoot) { + return dataRoot / ".index-snapshots"; +} + +std::filesystem::path snapshot_dir(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir) { + return snapshots_root(dataRoot) / indexDir.filename(); +} + +std::string snapshot_id(const std::filesystem::path& indexDir) { + std::ifstream is(indexDir / ".xlings-index-version", std::ios::binary); + if (!is) return {}; + std::string v{std::istreambuf_iterator(is), {}}; + while (!v.empty() && (v.back() == '\n' || v.back() == '\r' || v.back() == ' ')) + v.pop_back(); + return sanitize_component(std::move(v)); +} + +std::vector +index_dirs(const std::filesystem::path& dataRoot) { + std::vector out; + std::error_code ec; + if (!std::filesystem::is_directory(dataRoot, ec)) return out; + for (auto& e : std::filesystem::directory_iterator(dataRoot, ec)) { + if (ec) break; + if (!e.is_directory()) continue; + // The snapshots store lives under the same root; never snapshot it. + if (e.path().filename() == ".index-snapshots") continue; + std::error_code pec; + if (!std::filesystem::is_directory(e.path() / "pkgs", pec)) continue; + out.push_back(e.path()); + } + std::ranges::sort(out); + return out; +} + +bool archive(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir) +{ + if (!mcpp::pm::index_usable(indexDir)) return false; + + auto id = snapshot_id(indexDir); + if (id.empty()) id = "unversioned"; + auto dst = snapshot_dir(dataRoot, indexDir) / id; + + // Same id already archived: the tree did not move, so the existing + // snapshot is byte-equivalent and re-copying it buys nothing. + std::error_code ec; + if (std::filesystem::is_directory(dst, ec) && id != "unversioned") + return true; + + return copy_tree(indexDir, dst); +} + +bool restore(const std::filesystem::path& snapshotPath, + const std::filesystem::path& indexDir) +{ + std::error_code ec; + if (!std::filesystem::is_directory(snapshotPath, ec)) return false; + return copy_tree(snapshotPath, indexDir); +} + +std::vector +list_snapshots(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir) +{ + std::vector out; + auto base = snapshot_dir(dataRoot, indexDir); + std::error_code ec; + if (!std::filesystem::is_directory(base, ec)) return out; + for (auto& e : std::filesystem::directory_iterator(base, ec)) { + if (ec) break; + if (e.is_directory()) out.push_back(e.path()); + } + std::ranges::sort(out, [](auto& a, auto& b) { + return mtime_of(a) > mtime_of(b); // newest first + }); + return out; +} + +std::optional +newest_usable(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir) +{ + for (auto& s : list_snapshots(dataRoot, indexDir)) + if (mcpp::pm::index_usable(s)) return s; + return std::nullopt; +} + +void prune(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir, + std::size_t keep) +{ + auto snaps = list_snapshots(dataRoot, indexDir); + for (std::size_t i = keep; i < snaps.size(); ++i) { + std::error_code ec; + std::filesystem::remove_all(snaps[i], ec); + } +} + +int guarded_refresh(const std::filesystem::path& dataRoot, + const std::function& refresh, + GuardOutcome& out) +{ + // Snapshot every tree that is currently readable. Doing this BEFORE the + // refresh is the whole point: afterwards the old bytes are gone. + auto before = index_dirs(dataRoot); + std::map usableBefore; + for (auto& dir : before) { + usableBefore[dir] = mcpp::pm::index_usable(dir); + if (usableBefore[dir]) { + archive(dataRoot, dir); + prune(dataRoot, dir); + } + } + + const int rc = refresh(); + + // Judge each tree. `index_dirs` is re-read: a refresh may add a repo. + for (auto& dir : index_dirs(dataRoot)) { + if (mcpp::pm::index_usable(dir)) { + // Improved or unchanged — record the new good state for next time. + archive(dataRoot, dir); + prune(dataRoot, dir); + continue; + } + auto wasUsable = usableBefore.find(dir); + if (wasUsable != usableBefore.end() && wasUsable->second) { + // Got worse. This is the case the invariant exists for. + if (auto snap = newest_usable(dataRoot, dir); + snap && restore(*snap, dir)) { + out.rolledBack.push_back(dir); + continue; + } + } + // Was already unusable (or the rollback failed): try the local history + // anyway — it may hold a readable tree from before this binary ever + // met the raised floor. + if (auto snap = newest_usable(dataRoot, dir); + snap && restore(*snap, dir)) { + out.recovered.push_back(dir); + continue; + } + out.stillUnusable.push_back(dir); + } + return rc; +} + +} // namespace mcpp::pm::index_snapshot diff --git a/src/pm/package_fetcher.cppm b/src/pm/package_fetcher.cppm index 8d5aecf5..d911488f 100644 --- a/src/pm/package_fetcher.cppm +++ b/src/pm/package_fetcher.cppm @@ -378,6 +378,16 @@ format_install_failure_diagnostic(std::string_view target, msg += "\n xlings emitted no structured error; re-run with " "MCPP_VERBOSE=1 to see the raw xlings invocation + output."; } + + // If an index was too new for this binary, THAT is the cause, and the + // not-found above is only its symptom. Without this the last line the user + // reads names a wire address and a missing package — pointing at naming or + // publication — while the real answer (E0006, printed much earlier and + // possibly scrolled away) says "upgrade mcpp". The message that stops the + // build has to carry the reason the build stopped. + if (auto hint = mcpp::pm::unusable_index_hint(); !hint.empty()) { + msg += "\n" + hint; + } return msg; } diff --git a/src/toolchain/fingerprint.cppm b/src/toolchain/fingerprint.cppm index 47f42d15..59e6562c 100644 --- a/src/toolchain/fingerprint.cppm +++ b/src/toolchain/fingerprint.cppm @@ -15,10 +15,15 @@ export module mcpp.toolchain.fingerprint; import std; import mcpp.toolchain.detect; +import mcpp.version; export namespace mcpp::toolchain { -inline constexpr std::string_view MCPP_VERSION = "2026.8.3.4"; +// The version itself lives in the leaf module mcpp.version — see that file for +// why. Kept spelled `mcpp::toolchain::MCPP_VERSION` here because that is what +// every existing reader says, and because the fingerprint genuinely is one of +// its consumers (field 8 of the 10). +inline constexpr std::string_view MCPP_VERSION = mcpp::MCPP_VERSION; struct FingerprintInputs { Toolchain toolchain; diff --git a/src/version.cppm b/src/version.cppm new file mode 100644 index 00000000..61432d9f --- /dev/null +++ b/src/version.cppm @@ -0,0 +1,36 @@ +// mcpp.version — this binary's own version, and nothing else. +// +// WHY IT IS ITS OWN MODULE +// +// The constant used to live in mcpp.toolchain.fingerprint, next to the struct +// that folds it into the BMI cache key. That is a reasonable place for a +// *consumer* of the version and a bad place for the version itself: fingerprint +// imports mcpp.toolchain.detect, which imports mcpp.xlings, so "what version is +// this binary" transitively dragged in the entire toolchain-detection and +// package-manager subsystem. +// +// That is not a stylistic complaint. mcpp.pm.index_contract needs the version +// for one comparison (is this binary new enough for this index?), and pulling +// it from fingerprint made index_contract depend on xlings — which made it +// impossible for xlings to depend on index_contract, which is exactly where the +// index-refresh guard has to live (mcpp.pm.index_snapshot, and see +// mcpp::xlings::update_index). The cycle was the layering telling us the +// constant was in the wrong place. +// +// A leaf module with no imports beyond `std` can be used by anyone. Keep it +// that way: nothing else belongs in this file. +// +// SINGLE SOURCE OF TRUTH. `.github/tools/check_version_pins.sh` reads the +// literal below and cross-checks it against `mcpp.toml`'s `[package].version`; +// tests/e2e/01_help_and_version.sh checks it against `mcpp --version` at +// runtime. Both must be updated together — see docs/09-release.md. + +export module mcpp.version; + +import std; + +export namespace mcpp { + +inline constexpr std::string_view MCPP_VERSION = "2026.8.3.5"; + +} // namespace mcpp diff --git a/src/xlings.cppm b/src/xlings.cppm index d5d50c66..23ec0f40 100644 --- a/src/xlings.cppm +++ b/src/xlings.cppm @@ -15,6 +15,8 @@ export module mcpp.xlings; import std; import mcpp.pm.compat; +import mcpp.pm.index_contract; +import mcpp.pm.index_snapshot; import mcpp.platform; import mcpp.log; @@ -1418,7 +1420,60 @@ bool is_official_package_index_fresh(const Env& env, && official_index_cache_matches_package_file(env, packageName); } +namespace { +// The raw sync. Everything that makes a refresh SAFE lives in the wrapper +// below; this function's only job is to run `xlings update` and report. +int update_index_unguarded(const Env& env, bool quiet); +} // namespace + +// Guarded entry point — the ONE place every refresh in this process passes +// through, which is why the monotonicity guarantee is installed here rather +// than at the seven call sites. +// +// an index-side change must never take mcpp from "works" to "does not work" +// +// A published index can raise its client-version floor (index.toml min_mcpp). +// `xlings update` rewrites the tree in place, so before this guard existed a +// floor bump replaced a readable index with an unreadable one and left no way +// back — the refresh itself was the thing that broke the machine. See +// mcpp.pm.index_snapshot for why the shape is archive/judge/restore rather +// than the stage-and-swap the original design assumed. int update_index(const Env& env, bool quiet) { + namespace snap = mcpp::pm::index_snapshot; + const auto dataRoot = paths::index_data(env); + + snap::GuardOutcome out; + int rc = snap::guarded_refresh(dataRoot, + [&] { return update_index_unguarded(env, quiet); }, out); + + // Report ONLY when the guard had to act. The common path — refresh keeps + // the index readable — must stay silent, or the notice becomes noise that + // users learn to skip past, which is the same as not printing it. + for (auto& dir : out.rolledBack) { + print_status("Kept", std::format( + "previous index for `{}` — the refreshed one requires a newer mcpp", + dir.filename().string())); + if (!quiet) { + std::println(" Your build continues to work with the packages " + "it already describes."); + std::println(" Upgrade to pick up newer packages: xlings update mcpp"); + } + } + for (auto& dir : out.recovered) { + print_status("Restored", std::format( + "index `{}` from a local snapshot this mcpp can read", + dir.filename().string())); + } + for (auto& dir : out.stillUnusable) { + mcpp::log::verbose("index", std::format( + "index `{}` requires a newer mcpp and no local snapshot is usable", + dir.filename().string())); + } + return rc; +} + +namespace { +int update_index_unguarded(const Env& env, bool quiet) { // Offline is absolute: no caller gets to reach the network by going around // the decision layer. Reported as success so a build that can still resolve // everything locally proceeds — the caller that genuinely needed the data @@ -1472,6 +1527,7 @@ int update_index(const Env& env, bool quiet) { "index update failed after {} attempts (rc {})", kMaxAttempts, rc)); return rc; } +} // namespace void ensure_index_fresh(const Env& env, std::int64_t ttlSeconds, bool quiet) { if (is_index_fresh(env, ttlSeconds)) return; diff --git a/tests/e2e/185_index_floor_degrades.sh b/tests/e2e/185_index_floor_degrades.sh new file mode 100755 index 00000000..8ed5ba6a --- /dev/null +++ b/tests/e2e/185_index_floor_degrades.sh @@ -0,0 +1,192 @@ +#!/usr/bin/env bash +# requires: gcc fresh-sandbox +# 185_index_floor_degrades.sh — an index cannot decide whether mcpp works. +# +# An index tree may declare a client-version floor (`index.toml` min_mcpp). Two +# behaviours around that floor had NO end-to-end coverage at all — only the pure +# predicate `floor_violation()` was unit-tested — which is why both were wrong +# for as long as the feature has existed: +# +# 1. A floor violation made every descriptor read return "nothing", which is +# the same answer as "this package is not in this index". The build then +# died on `E_NOT_FOUND ... wire address tried: ...` — naming neither the +# version nor the floor. The cause was printed much earlier, and the line +# that actually stopped the build pointed at addressing instead. +# +# 2. That same indistinguishable miss fed the refresh policy, which read it as +# "the local index is missing something, go fetch" — so an unusable index +# drove repeated refreshes of itself. +# +# Both assertions below are about what the user READS, because that is what was +# broken. `mcpp explain E0006` and the floor predicate were fine throughout. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +# ── two path indexes: one readable, one that demands a newer mcpp ──────────── +make_index() { # $1 = dir, $2 = min_mcpp ("" = no contract), $3 = pkg name + local dir="$1" floor="$2" pkg="$3" + mkdir -p "$dir/pkgs/${pkg:0:1}" + if [[ -n "$floor" ]]; then + cat > "$dir/index.toml" < "$dir/pkgs/${pkg:0:1}/$pkg.lua" < src/main.cpp <<'EOF' +import std; +int main() { std::println("ok"); return 0; } +EOF + +# ── 1. an unusable index must say so — not "package not found" ─────────────── +cat > mcpp.toml < build.log 2>&1 +rc=$? +set -e +[[ "$rc" != "0" ]] || { echo "FAIL: build unexpectedly succeeded"; cat build.log; exit 1; } + +grep -q "E0006" build.log || { + echo "FAIL: the failure never mentions E0006 — the user cannot tell that the" + echo " cause is an index that needs a newer mcpp." + cat build.log + exit 1 +} + +# THE assertion that was missing, and it has to be precise. +# +# "E0006 appears somewhere in the output" is NOT enough — it always did, and a +# test asserting only that passes on the broken build too (verified). What was +# broken is that the message which STOPS the build said only +# +# error: dependency 'toonew.newlib': not found in local index at '...' +# +# i.e. it blamed the package, pointing the user at publication or naming, while +# the actual answer (upgrade mcpp) had scrolled past. So: assert on the LAST +# error, and assert on the specific text that ties the two together. +last_error="$(grep -E '^error:' build.log | tail -1)" +echo "$last_error" | grep -q "not found" && { + # A bare not-found as the final word is the exact regression. It is only + # acceptable if the cause is attached to it. + grep -A 4 "$(echo "$last_error" | head -c 40)" build.log \ + | grep -qE "cannot read|E0006" || { + echo "FAIL: the build stopped on a bare 'not found' with no mention of the" + echo " index that could not be read. That blames the package for a" + echo " problem whose fix is 'upgrade mcpp'." + echo "--- output ---" + cat build.log + exit 1 + } +} +grep -q "this mcpp cannot read" build.log || { + echo "FAIL: nothing in the output names the unreadable index as the cause." + cat build.log + exit 1 +} +echo "PASS: an unusable index reports E0006 as the cause" + +# ── 2. an unusable index must not take a healthy one down with it ──────────── +# INV-2: the blast radius of a floor bump is the index that declared it. +cat > mcpp.toml < iso.log 2>&1 +iso_rc=$? +set -e +[[ "$iso_rc" == "0" ]] || { + echo "FAIL: a project that depends on NOTHING from the too-new index still" + echo " failed to build — an unusable index is not supposed to be" + echo " contagious." + cat iso.log + exit 1 +} +echo "PASS: an unusable index does not break a build that does not need it" + +# ── 3. the escape hatch still works ───────────────────────────────────────── +# It exists for debugging; if it ever stops working the only lever a blocked +# user has is gone. +cat > mcpp.toml < ignore.log 2>&1 || { + echo "FAIL: MCPP_INDEX_FLOOR=ignore no longer bypasses the floor" + cat ignore.log + exit 1 +} +grep -q "E0006" ignore.log && { + echo "FAIL: MCPP_INDEX_FLOOR=ignore still reported E0006" + cat ignore.log + exit 1 +} +echo "PASS: MCPP_INDEX_FLOOR=ignore bypasses the floor" + +echo "OK" diff --git a/tests/unit/test_index_snapshot.cpp b/tests/unit/test_index_snapshot.cpp new file mode 100644 index 00000000..11b3210e --- /dev/null +++ b/tests/unit/test_index_snapshot.cpp @@ -0,0 +1,227 @@ +// The invariant this file exists for: +// +// An index-side change must never take mcpp from "works" to "does not work". +// +// A published index can raise its client-version floor (index.toml min_mcpp). +// `xlings update` rewrites the tree in place, so before the guard existed a +// floor bump replaced a readable index with an unreadable one and left no way +// back — the refresh was the thing that broke the machine. +// +// The 2026-07-08 index design specified this behaviour ("staged refresh keeps +// the last compatible snapshot") and it was never implemented; the floor was +// checked in exactly one place, the descriptor reader. There was no test that +// could have noticed, because the only coverage was of the pure predicate. +// These are that missing test. + +#include + +import std; +import mcpp.pm.index_snapshot; +import mcpp.pm.index_contract; +import mcpp.version; + +using namespace mcpp::pm::index_snapshot; + +namespace { + +struct Tmp { + std::filesystem::path path; + Tmp() { + path = std::filesystem::temp_directory_path() + / std::format("mcpp_idx_snap_{}", std::random_device{}()); + std::filesystem::create_directories(path); + } + ~Tmp() { std::error_code ec; std::filesystem::remove_all(path, ec); } +}; + +void writeFile(const std::filesystem::path& p, std::string_view body) { + std::filesystem::create_directories(p.parent_path()); + std::ofstream(p) << body; +} + +std::string readFile(const std::filesystem::path& p) { + std::ifstream is(p); + return std::string((std::istreambuf_iterator(is)), {}); +} + +// A minimal index tree: pkgs/ makes it an index, index.toml carries the floor, +// .xlings-index-version is the snapshot identity. +void makeIndex(const std::filesystem::path& dir, + std::string_view minMcpp, + std::string_view version, + std::string_view marker) { + writeFile(dir / "pkgs" / "z" / "zlib.lua", std::format("-- {}\n", marker)); + writeFile(dir / ".xlings-index-version", version); + if (minMcpp.empty()) { + std::error_code ec; + std::filesystem::remove(dir / "index.toml", ec); + } else { + writeFile(dir / "index.toml", + std::format("[index]\nspec = \"1\"\nmin_mcpp = \"{}\"\n", minMcpp)); + } +} + +constexpr std::string_view kTooNew = "9999.9.9.9"; // no mcpp satisfies this + +} // namespace + +TEST(IndexSnapshot, DetectsIndexTreesAndIgnoresTheSnapshotStore) { + Tmp t; + auto data = t.path / "data"; + makeIndex(data / "mcpplibs", "", "v1", "a"); + makeIndex(data / "xim-pkgindex", "", "v1", "b"); + std::filesystem::create_directories(data / "not-an-index"); // no pkgs/ + std::filesystem::create_directories(data / ".index-snapshots" / "x" / "pkgs"); + + auto dirs = index_dirs(data); + ASSERT_EQ(dirs.size(), 2u); + EXPECT_EQ(dirs[0].filename(), "mcpplibs"); + EXPECT_EQ(dirs[1].filename(), "xim-pkgindex"); +} + +TEST(IndexSnapshot, ArchiveRefusesAnUnusableTree) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + makeIndex(idx, kTooNew, "v1", "unusable"); + + // A snapshot exists to be restored; archiving one nobody can read would + // only give the recovery path a useless candidate. + EXPECT_FALSE(archive(data, idx)); + EXPECT_TRUE(list_snapshots(data, idx).empty()); +} + +TEST(IndexSnapshot, ArchiveAndRestoreRoundTrip) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + makeIndex(idx, "0.0.85", "v1", "original"); + + ASSERT_TRUE(archive(data, idx)); + auto snaps = list_snapshots(data, idx); + ASSERT_EQ(snaps.size(), 1u); + + makeIndex(idx, "0.0.85", "v2", "replaced"); + EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("replaced"), + std::string::npos); + + ASSERT_TRUE(restore(snaps[0], idx)); + EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("original"), + std::string::npos); +} + +// THE load-bearing test. A refresh that raises the floor beyond this binary +// must leave the machine working. +TEST(IndexSnapshot, RefreshThatBreaksTheIndexIsRolledBack) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + makeIndex(idx, "0.0.85", "good", "usable-tree"); + + GuardOutcome out; + int rc = guarded_refresh(data, [&] { + // This is what `xlings update` does: rewrite the tree in place, with a + // floor this binary cannot satisfy. + makeIndex(idx, kTooNew, "too-new", "unusable-tree"); + return 0; + }, out); + + EXPECT_EQ(rc, 0); + ASSERT_EQ(out.rolledBack.size(), 1u) << "the refresh made the index unusable " + "and the guard did not roll it back"; + EXPECT_TRUE(out.stillUnusable.empty()); + + // The machine still works: the tree on disk is readable and is the old one. + EXPECT_TRUE(mcpp::pm::index_usable(idx)); + EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("usable-tree"), + std::string::npos); +} + +// The mirror case: a refresh that keeps the index readable must be left alone, +// and must silently become the new known-good snapshot. +TEST(IndexSnapshot, NormalRefreshIsUntouchedAndBecomesTheNewSnapshot) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + makeIndex(idx, "0.0.85", "v1", "first"); + + GuardOutcome out; + guarded_refresh(data, [&] { + makeIndex(idx, "0.0.85", "v2", "second"); + return 0; + }, out); + + EXPECT_FALSE(out.degraded()) << "a healthy refresh must not report anything"; + EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("second"), + std::string::npos); + // Both the pre- and post-refresh trees are now archived, so the next + // refresh has somewhere to fall back to. + EXPECT_EQ(list_snapshots(data, idx).size(), 2u); +} + +// Already-unusable on entry: there is nothing to "roll back" to, but a local +// snapshot from before the floor moved is still the right answer. +TEST(IndexSnapshot, RecoversFromLocalHistoryWhenAlreadyUnusable) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + + // A good tree was seen at some point in the past... + makeIndex(idx, "0.0.85", "old-good", "known-good"); + ASSERT_TRUE(archive(data, idx)); + // ...and the current tree is already too new (e.g. mcpp was downgraded, or + // the guard was introduced after the damage was done). + makeIndex(idx, kTooNew, "too-new", "unusable"); + + GuardOutcome out; + guarded_refresh(data, [&] { return 0; }, out); // refresh changes nothing + + ASSERT_EQ(out.recovered.size(), 1u); + EXPECT_TRUE(out.rolledBack.empty()); + EXPECT_TRUE(mcpp::pm::index_usable(idx)); + EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("known-good"), + std::string::npos); +} + +// No local history and a too-new tree is the one case nothing can rescue. It +// must be reported honestly rather than silently left looking fine. +TEST(IndexSnapshot, ReportsStillUnusableWhenNoSnapshotCanHelp) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + makeIndex(idx, kTooNew, "too-new", "unusable"); + + GuardOutcome out; + guarded_refresh(data, [&] { return 0; }, out); + + ASSERT_EQ(out.stillUnusable.size(), 1u); + EXPECT_TRUE(out.rolledBack.empty()); + EXPECT_TRUE(out.recovered.empty()); +} + +TEST(IndexSnapshot, PruneKeepsTheNewest) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + for (int i = 0; i < 4; ++i) { + makeIndex(idx, "0.0.85", std::format("v{}", i), std::format("gen{}", i)); + ASSERT_TRUE(archive(data, idx)); + } + ASSERT_EQ(list_snapshots(data, idx).size(), 4u); + prune(data, idx, /*keep=*/2); + EXPECT_EQ(list_snapshots(data, idx).size(), 2u); +} + +// A tree with no index.toml declares no contract, so it is usable by anyone. +// Third-party indexes rely on this and must never be rolled back. +TEST(IndexSnapshot, TreeWithoutAContractIsAlwaysUsable) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "third-party"; + makeIndex(idx, "", "v1", "no-contract"); + EXPECT_TRUE(mcpp::pm::index_usable(idx)); + + GuardOutcome out; + guarded_refresh(data, [&] { return 0; }, out); + EXPECT_FALSE(out.degraded()); +} From 3293a3eca5e205fb7f38503fe22422be28379894 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Tue, 4 Aug 2026 01:48:56 +0800 Subject: [PATCH 2/2] fix(index): keep the snapshot store out of the index-repo namespace MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verifying the guard on a real refresh showed it working — three index trees archived, 3.2 MB, snapshot ids being the upstream commit shas — and also showed where I had put the store: `/registry/data/.index-snapshots`, i.e. inside the directory that IS the index-repo namespace. Everything that enumerates index repos does it by listing subdirectories of `data/`. Fetcher::sorted_index_dirs returns every one of them with no filter, and xlings walks the same tree. The store was being handed to those scanners as if it were an index. It happens to be harmless today: the store has no `pkgs/`, so a descriptor lookup stats a path that does not exist and moves on. That is a property of someone else's loop, not of this design, and it would stop holding the moment the store's internal layout changed or a scanner started matching on something other than `pkgs/`. The store is now a sibling of the data root — `/registry/index-snapshots` — where no index scanner can reach it. The defensive skip in `index_dirs` stays: that loop decides what gets archived, and an archive of the archive is the one mistake that would grow without bound. Verified on a real refresh: `data/` clean, store in the new location, same 3.2 MB. unit 55/55, e2e 176 passed / 0 failed. --- src/pm/index_snapshot.cppm | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/src/pm/index_snapshot.cppm b/src/pm/index_snapshot.cppm index 30f7f4b6..1b1f60fc 100644 --- a/src/pm/index_snapshot.cppm +++ b/src/pm/index_snapshot.cppm @@ -63,7 +63,17 @@ export namespace mcpp::pm::index_snapshot { // mcpp index has changed its floor 6 times in its entire life). inline constexpr std::size_t kKeepPerIndex = 5; -// /.index-snapshots/// +// The snapshot store is a SIBLING of the index data root, never inside it: +// +// /.. index-snapshots/// +// +// Everything that enumerates index repos does it by listing directories under +// `data/` (mcpp's own Fetcher::sorted_index_dirs takes every subdirectory, and +// xlings walks the same tree). A snapshot store living there would be handed to +// those scanners as if it were an index. Today that happens to be harmless — +// the store has no `pkgs/` so lookups miss and move on — but "harmless because +// of a detail of someone else's loop" is not a property worth depending on, +// and it costs nothing to put the store where no index scanner can reach it. std::filesystem::path snapshots_root(const std::filesystem::path& dataRoot); std::filesystem::path snapshot_dir(const std::filesystem::path& dataRoot, const std::filesystem::path& indexDir); @@ -187,7 +197,11 @@ std::string sanitize_component(std::string s) { } // namespace std::filesystem::path snapshots_root(const std::filesystem::path& dataRoot) { - return dataRoot / ".index-snapshots"; + auto parent = dataRoot.parent_path(); + // Degenerate path (relative "data", or a root) — fall back to staying put + // rather than climbing out of the home directory. + if (parent.empty()) return dataRoot / ".index-snapshots"; + return parent / "index-snapshots"; } std::filesystem::path snapshot_dir(const std::filesystem::path& dataRoot, @@ -212,7 +226,10 @@ index_dirs(const std::filesystem::path& dataRoot) { for (auto& e : std::filesystem::directory_iterator(dataRoot, ec)) { if (ec) break; if (!e.is_directory()) continue; - // The snapshots store lives under the same root; never snapshot it. + // The store is a sibling of dataRoot (see snapshots_root), so it + // should never appear here. Skipped anyway: this loop decides what + // gets archived, and an archive of the archive is the one mistake + // that would grow without bound. if (e.path().filename() == ".index-snapshots") continue; std::error_code pec; if (!std::filesystem::is_directory(e.path() / "pkgs", pec)) continue;