diff --git a/.agents/docs/2026-08-03-index-availability-must-not-decide-mcpp-availability.md b/.agents/docs/2026-08-03-index-availability-must-not-decide-mcpp-availability.md new file mode 100644 index 00000000..4cf1fa8f --- /dev/null +++ b/.agents/docs/2026-08-03-index-availability-must-not-decide-mcpp-availability.md @@ -0,0 +1,299 @@ +# 索引的可用性不得决定 mcpp 的可用性 —— 纯 mcpp 侧优化方案 + +> 状态:**已实施**(2026.8.3.5)。实施中的三处偏差记录在 §7。 +> 范围:**全部落在 `mcpp-community/mcpp` 内**。零跨仓依赖 —— 不需要 xlings 改动, +> 也不需要索引侧改动。 +> 背景分析:`.agents/docs/2026-08-03-index-floor-should-degrade-not-brick.md` +> 跨仓增强(非前置):[openxlings/xlings#476](https://github.com/openxlings/xlings/issues/476) + +--- + +## 0. 要立的不变量 + +> **INV —— 索引侧的任何改动,都不得使 mcpp 从「能用」变成「不能用」。** + +这条不变量今天不成立:`mcpplibs/mcpp-index` 抬了一次 `min_mcpp`, +所有 `0.0.109` 用户的构建就死了。**一次索引提交,让一个已发布客户端不可用**, +这在包管理器里是不可接受的耦合 —— 索引是**数据**,客户端是**程序**, +数据的更新不该让程序失效。 + +三条可检验的推论: + +| # | 推论 | 今天 | +|---|---|---| +| **INV-1** | **刷新是单调的**:刷新只能维持或改善可用性,绝不能降低 | ❌ 刷新把能用的换成不能用的,且不可逆 | +| **INV-2** | **不可用是有边界的**:一个索引不可用,只影响**它提供的包**;不影响其它索引,也不影响**已解析/已安装**的依赖 | ⚠️ 边界在读取层存在,但在错误语义与刷新决策上泄漏 | +| **INV-3** | **不可用要自报家门**:绝不伪装成「包不存在」 | ❌ `return std::nullopt` ⇒ 全部变成 not-found | + +再加一条设计判据,后面几处都从它推出来: + +> **下限约束的是「读描述符」,不是「构建」。** +> 一个不需要读任何描述符的构建(锁文件齐全、payload 已装),不该被下限拦住。 + +--- + +## 1. 今天为什么违反 + +### 1.1 刷新路径对下限一无所知(违反 INV-1) + +``` +$ grep -rn "check_index_floor" src/ +src/pm/package_fetcher.cppm:619 ← 全代码库唯一一处 +``` + +`index_management.cppm:148` 的刷新是 + +```cpp +int rc = mcpp::xlings::update_index(xlEnv); // 跑 `xlings update`,原地覆盖 +``` + +**刷新前不问「换来的这棵树我能用吗」,刷新后也不问。** 于是一次 `xlings update` +就能把一份能用的索引原地换成用不了的,**没有备份、没有回滚、没有退路**。 + +用户在刷新之后比刷新之前更糟 —— 这是最伤人的性质,也是本方案第一优先级。 + +### 1.2 违反下限退化成「包不存在」(违反 INV-3) + +`package_fetcher.cppm:614-622`: + +```cpp +if (auto violation = mcpp::pm::check_index_floor(pkgsDir.parent_path())) { + mcpp::ui::error(*violation); + return std::nullopt; // 「读不了」与「没有」压成同一个返回值 +} +``` + +后果是终止构建的那条错误与真实原因隔了两层 +(`E_NOT_FOUND ... wire address tried: ...`,既不提版本也不提下限)。 + +还有一个更隐蔽的二次伤害:`index_refresh` 的判据是「解析输入是否都在磁盘」。 +一个被下限挡住的读取**看起来就是一次真实的 miss** ⇒ 判定「该刷新了」⇒ +再拉一次同样不兼容的树。**不可用状态自己驱动了重复刷新。** + +### 1.3 整条下限行为**没有任何 e2e 覆盖** + +``` +tests/unit/test_index_contract.cpp ← 只测纯谓词 floor_violation() +tests/e2e/ ← 零 +``` + +2026-07-08 的设计里写了要有 +(*"e2e: fixture index with `min_mcpp = "9.9.9"` → build fails"*),**没有落地**。 +所以 1.1 和 1.2 从上线到今天没有任何机制会发现 —— 谓词是对的, +**它周围的行为从没被端到端跑过**。 + +--- + +## 2. 方案(M1–M5,全部 mcpp 侧) + +### M1(必须)· 刷新单调性:先备份,后验收,不合格就回滚 + +**落点**:`src/pm/index_management.cppm` 的刷新编排。 + +``` +refresh(repo): + usable_before = floor_ok(repo.root) # 刷新前的可用性 + snapshot = archive(repo.root) # 见 M2,硬链接复制,≈0 成本 + xlings::update_index(...) # 原地覆盖,不在 mcpp 进程内 + if usable_before and not floor_ok(repo.root): + restore(repo.root, snapshot) # ← 回滚 + warn_once(downgrade_notice) + mark_no_refresh_this_process(repo) # 防抖,见 1.2 的二次伤害 +``` + +要点: + +- **判据是「刷新前能用 ⇒ 刷新后必须仍能用」**,不是「刷新后必须能用」。 + 本来就不能用的(冷启动撞上高下限),回滚没有意义,交给 M3/M4。 +- `xlings` 原地覆盖是既成事实,mcpp 无法要求它写到暂存目录 + (见分析文档 §4.0 的传输约束)。所以是**备份 + 事后验收 + 回滚**, + 而不是设计文档原本设想的「staged unpack + 验收后再切换」。 + **效果等价,且不需要任何跨仓改动。** +- 索引树很小(实测 `mcpplibs` 912K、`xim-pkgindex` 2.2M), + 用硬链接复制(`copy_options::create_hard_links`)近似零成本、零额外空间。 + +用户看到: + +``` +warning: the refreshed index requires mcpp >= 2026.8.3.3; this is mcpp 0.0.109. + Kept the previous index () — your build continues to work. + Upgrade to pick up newer packages: xlings update mcpp +``` + +**这一条单独就把「变砖」变成「可用 + 提示升级」。** + +### M2(必须)· mcpp 自己的本地快照历史 + +M1 只留一份「上一个」。M2 把它变成一条**本地历史线** —— 也就是 +「即使 xlings 没有相关功能,至少还有之前的」这句话的实现。 + +``` +~/.mcpp/registry/data/.index-snapshots/// +``` + +- **快照身份**用现成的 `.xlings-index-version`(每个索引根下已经有这个文件; + 它的值按契约是**不透明串**,不要当 sha 解析)。 +- 每次验收通过(`floor_ok`)就归档一份;保留最近 N 份(建议 N=5)+ 按体积上限回收, + 复用 `mcpp cache gc` 已有的 LRU 记账思路。 +- **需要时从新到旧回溯**,取第一个 `floor_ok` 的快照恢复。 + +与 [xlings#476](https://github.com/openxlings/xlings/issues/476) 的关系: + +| | 覆盖面 | 依赖 | +|---|---|---| +| **M2(本机历史)** | 只覆盖**这台机器见过**的快照 | **无** | +| #476(发布侧历史线) | 覆盖全部已发布快照,含全新安装 | 跨仓 | + +**M2 不是 #476 的替代品,是它的本地退化版**,且覆盖了绝大多数真实场景 +(一台持续在用的开发机/CI 缓存机,一定见过兼容的快照)。 +#476 落地后 M2 依然有价值:它让常见路径不必联网重下。 + +**诚实的边界**:全新安装 + 最新索引下限过高 ⇒ 本机没有任何历史 ⇒ +M2 无能为力,落到 M3/M4。这种情况下正确答案本来就是「升级 mcpp」, +M3 会把这件事说清楚。 + +### M3(必须)· `IndexUnusable` ≠ `Absent` + +**落点**:`src/pm/package_fetcher.cppm` 的描述符读取返回类型。 + +```cpp +enum class DescriptorLookup { Found, Absent, IndexUnusable }; +``` + +三条硬约束: + +1. `IndexUnusable` **不得**触发 legacy 派生地址回退(那是给「这个索引里没有这个包」用的); +2. `IndexUnusable` **不得**被 `index_refresh` 读成「本地缺东西 ⇒ 该刷新」—— + 这正是 1.2 那个重复刷新循环的入口; +3. 解析结束时若存在 `IndexUnusable`,**最终错误就是 E0006 本身**, + 并指明**哪个索引、要求什么、本机是什么、怎么升级**。 + 用户看到的第一条与最后一条错误必须是同一件事。 + +这条同时是 INV-2 的执行者:一个索引不可用**只**让它自己的包不可解析, +不得影响其它索引的 `Absent` 判定。 + +### M4(应做)· 下限拦的是「读描述符」,不是「构建」 + +由 §0 那条判据直接推出:**如果本次构建根本不需要读任何描述符 +(依赖都在 `mcpp.lock`、payload 都已安装),下限不该让它失败。** + +这正是 `index_refresh` 已经在用的那根轴(「所有解析输入都在磁盘」), +M3 落地后它自然成立:`IndexUnusable` 不再被误读成 miss, +于是「所有输入都在磁盘」的判定不受影响,构建照常离线进行。 + +**这是「mcpp-index 里的包升级不该影响 mcpp 可用性」的直接答案**: +一个已经解析完、装好的工程,与索引后来发生了什么完全无关。 + +> 边界要说清楚:如果构建**确实**需要装新东西(用户那次就是在装 `musl-gcc`), +> M4 帮不上,那是 M1/M2 的战场。M4 保的是**存量工程不被索引变更波及**。 + +### M5(必须)· 补上从未存在的 e2e + +现在只有纯谓词单测。至少三条端到端断言,**每一条都对应上面一个不变量**: + +| e2e | 断言 | 守的是 | +|---|---|---| +| 刷新单调性 | fixture 索引可用 → 构建成功;把它换成 `min_mcpp = "9.9.9"` 再刷新 → **构建仍然成功**,且 stderr 出现降级 warning | INV-1 / M1 | +| 错误自报家门 | 本机无任何兼容快照 → 构建失败,且**最后一条**错误含 `E0006`,stderr **不出现** `not found` | INV-3 / M3 | +| 边界隔离 | 索引 A 不可用、索引 B 正常 → B 的包照常解析成功 | INV-2 / M3 | + +第二条那个「最后一条错误必须是 E0006」正是这次(以及 +`install_pinned_mcpp.sh` 头注释记载的上一次)缺失的断言。 + +--- + +## 3. 不采纳 + +| 方案 | 理由 | +|---|---| +| 下限降级为纯 warning,照常解析 | 下限存在的理由是真的:`0.0.101` 那次是 `compat.opencv` 的 per-OS feature flags 被旧客户端**静默忽略**,产出错误构建。放行 = 用「静默错误产物」换「明确失败」,方向反了。M1/M2 保的是**旧快照**,不是**用旧客户端读新描述符**。 | +| 等 xlings#476 落地再做 | #476 是增强,不是前置。M1+M2+M3 **今天就能把「变砖」修掉**,且 #476 落地后全部保留价值。把用户可用性挂在别的仓库的排期上,本身就是这次问题的翻版。 | +| mcpp 自己接管索引下载 | 与 xlings 职责重叠,镜像/CDN/校验要重写一遍。M1 的「备份+回滚」用几十行拿到同样的可用性保证。 | +| 只做 M1,不做 M3 | 可用性修好了,但一旦真的走到不可用路径,用户仍然读到一条误导性的 `not found`。两者成本都很低,没有理由只做一半。 | +| 把 `MCPP_INDEX_FLOOR=ignore` 宣传成解法 | 它是调试逃生舱。让用户常态化绕过一个**为防止静默错误构建而存在**的闸,是把一个可用性问题换成一个正确性问题。 | + +--- + +## 4. 实施顺序 + +| # | 项 | 依赖 | 效果 | +|---|---|---|---| +| 1 | **M3** 错误语义 | 无 | 最小改动;同时堵掉重复刷新循环 | +| 2 | **M1** 备份 + 验收 + 回滚 | M3(复用 `floor_ok`) | **变砖 → 可用 + 提示升级** | +| 3 | **M2** 本地快照历史 | M1(归档即 M1 的备份) | 「至少还有之前的」升级为一条历史线 | +| 4 | **M5** e2e | M1–M3 | 让上面三条永不回归 | +| 5 | M4 复核 | M3 | 多数情况下 M3 落地即自然成立,需实测确认 | + +M1–M3 是一个 PR 的量级,建议一起发;M5 同 PR。 + +**发版说明要写清楚**:本次修复只对**升级到该版本之后**的用户生效。 +已经卡住的 `0.0.109` 用户,要么升级 mcpp,要么等索引侧把下限降回去 —— +这也是为什么索引侧那次回退仍然值得单独做(见背景分析文档 §6 第 1 步)。 + +--- + +## 5. 验证 + +- `mcpp test --workspace`(单测)+ 新增三条 e2e。 +- **先红后绿**:M5 的三条 e2e 必须先在 `main` 上跑出红,再实施 M1–M3。 + 按 §1.3,今天它们一条都不存在,所以「先红」这一步不能省 —— + 否则无法证明它们真的覆盖到了。 +- 人工复现:构造一个 fixture 索引,`min_mcpp` 从合法值改成 `9.9.9`, + 跑 `mcpp build` 两次,断言第二次仍然成功且打出降级 warning。 + +--- + +## 6. 这次暴露的一条通用教训 + +设计文档写下的行为("staged refresh keeps the last compatible snapshot") +与实现之间没有任何机器化的连接,于是**承诺存在、实现缺席,而且长期没人发现**。 +缺席的不是代码,是**那条本该失败的测试**:谓词有单测(所以谓词是对的), +谓词周围的行为没有 e2e(所以行为是错的)。 + +判据:**一个设计文档里以「mandatory」措辞写下的行为,必须有一条以它命名的测试。** +没有测试的 mandatory 是一句愿望。 + + +--- + +## 7. 实施记录(2026.8.3.5) + +落点:`src/version.cppm`(新)、`src/pm/index_snapshot.cppm`(新)、 +`src/pm/index_contract.cppm`、`src/pm/index_refresh.cppm`、`src/build/prepare.cppm`、 +`src/pm/package_fetcher.cppm`、`src/xlings.cppm`、 +`tests/unit/test_index_snapshot.cpp`(新)、`tests/e2e/185_index_floor_degrades.sh`(新)。 + +三处与方案的偏差,都是实施时被现实证伪的假设: + +**① 守卫装不进 `xlings::update_index` —— 编译器报了循环依赖。** +方案假定「全进程唯一刷新入口」就是落点。实际 `mcpp.xlings` 无法 import +`mcpp.pm.index_contract`:后者 import `mcpp.toolchain.fingerprint` +(只为读 `MCPP_VERSION`)→ `mcpp.toolchain.detect` → `mcpp.xlings`。 +**「这个二进制是什么版本」传递依赖了整个工具链探测子系统**,这就是分层在报错。 +把常量提到叶子模块 `src/version.cppm` 后环消失。 +连带:`check_version_pins.sh` 与三处发布文档要改指向(版本号唯一真源换了文件)。 + +**② 快照绝不能用硬链接 —— 单测立刻抓到。** +`copy_options::create_hard_links` 是显然的优化(索引树是几千个小文件)。 +它在这里是**错的**:硬链接快照**与活动树共享 inode**,任何原地重写 +(截断写、tar 覆盖解包)会直接写穿链接、毁掉备份。而本模块要挺过的那件事 +正是「有东西替换了这棵树」。改为真实复制;实测 912KB/2.2MB × 5 ≈ 15MB,`prune` 封顶。 + +**③ 最初的 e2e 是假绿 —— 在 pre-fix 二进制上照样通过。** +第一版断言写的是「输出里出现 E0006」。它**一直**出现,所以那条断言什么也没测到 +(已用 pre-fix 二进制实测确认通过)。真正坏掉的是**终止构建的那条消息**: + +``` +error: dependency 'toonew.newlib': not found in local index at '...' +``` + +它**把责任推给包**(指向发布/命名),而真正的答案早已滚出屏幕。 +断言改为针对**最后一条 error**,并要求它带上真正的原因;改完在 pre-fix 上精确变红。 +> 这条同时验证了 §6 那句判据:**没有测试的 mandatory 是一句愿望** —— +> 而一条写得不够精确的测试,比没有测试更危险,因为它让人以为覆盖到了。 + +另:终止构建的 not-found 不止一处(`prepare.cppm` 有三处), +所以原因附加收敛为一个 `with_index_cause()` helper,而不是逐处拼字符串。 + +验证状态:unit 55/55(含新增 9 条);e2e 185 在修复前后分别红/绿; +`check_version_pins.sh` 通过。 diff --git a/.agents/docs/2026-08-03-index-floor-should-degrade-not-brick.md b/.agents/docs/2026-08-03-index-floor-should-degrade-not-brick.md new file mode 100644 index 00000000..c76efae7 --- /dev/null +++ b/.agents/docs/2026-08-03-index-floor-should-degrade-not-brick.md @@ -0,0 +1,480 @@ +# 索引版本下限(E0006)不应让旧客户端不可用 —— 分析与方案 + +> 状态:分析定稿,待实施 +> 触发:用户报告 mcpp `0.0.109` 撞上 `index requires mcpp >= 2026.8.3.3 [E0006]`, +> 随后整条依赖解析崩成 `package 'compat:compat.libarchive@3.8.7' not found` +> 涉及:`src/pm/index_contract.cppm`、`src/pm/package_fetcher.cppm`、 +> `src/pm/index_refresh.cppm`、`src/doctor.cppm`;索引侧 `mcpplibs/mcpp-index` +> 跨仓依赖:[openxlings/xlings#476](https://github.com/openxlings/xlings/issues/476)(枚举 + 按版本同步索引快照) +> 相关设计:`.agents/docs/2026-07-08-index-version-semantics-and-descriptor-grammar-design.md` + +--- + +## 0. 结论摘要 + +**提问是对的,而且比它看上去更严重。**「旧 mcpp 应该仍然可用,只是提示可以升级」不仅是 +合理诉求 —— 它本来就是 2026-07-08 那份设计里 **L1 明文承诺过的行为**: + +> **L1 — authority (this design, mandatory)**: contract travels inside the tree; +> single check at the index-open choke point; **staged refresh keeps the last +> compatible snapshot**. Parse failure → graceful keep-old + warn. + +「保留上一个兼容快照」**从未被实现**。今天的实现只有那一个 choke point 检查, +刷新路径对下限一无所知。于是 `xlings update` 会把一份能用的索引换成一份用不了的, +**且没有退路** —— 用户在刷新之后比刷新之前更糟。 + +同时,下限本身正在以设计没有预期的频率上移:它被当作「等于 CI pin」维护, +而 CI pin 跟着最新发布走。结果是**一个一周前的版本,被一次「新增一个包」的提交废掉**。 + +**方案的骨架是一条「索引历史线」**(§4.0):索引不再是一个 rolling 指针,而是一条 +带下限的版本线;客户端刷新时从最新往回找,取**第一个自己满足下限的快照**, +并只在发生降级时打印一次「最新索引需要 mcpp X,你是 Y,已改用 Y 能用的最新索引 Z, +升级请 `xlings update mcpp`」。**版本下限从终止信号变成路由信号。** + +分发侧不需要新建任何东西:F5 证明 313 个历史快照都还在,缺的只是指针里的一份元数据, +而按 F1 的历史,下限拐点至今只有 6 个 —— **一条 6 项的历史线覆盖整个索引生命周期**。 + +六个发现,四个必须修,两个是治理问题。 + +--- + +## 1. 现象:一次失败,三段错误 + +``` +Downloading xim:musl-gcc@… [===================>] 92.0 MB / 92.9 MB +error: index requires mcpp >= 2026.8.3.3 but this is mcpp 0.0.109 [E0006] + Upgrade: curl -fsSL .../install.sh | bash +error: xlings install_packages failed (exit 1) for 'compat.libarchive@3.8.7' … + xlings reported: E_NOT_FOUND: package 'compat:compat.libarchive@3.8.7' not found + wire address tried: compat:compat.libarchive@3.8.7 +``` + +三段,各自都有问题: + +1. **已经下载完 92MB 才发现用不了。** 兼容性判定发生在解包之后、读描述符的时候。 +2. E0006 说清楚了原因,但它**不是终止点**。 +3. 真正终止构建的是第三段 —— 一条 `not found`,**既不提版本也不提下限**, + 还附了一个"wire address tried",把读者引向命名/寻址问题。 + +第 3 段不是巧合,是第 2 段的必然结果,见 F3。 + +--- + +## 2. 六个发现(均有证据) + +### F1 —— 下限被当作「= CI pin」维护,而不是「能解析这棵树的最老客户端」 + +`index_contract.cppm` 的定义是清楚的: + +``` +min_mcpp = "0.0.85" # oldest mcpp able to parse every descriptor +``` + +但 `mcpp-index/index.toml` 的维护规则写的是另一件事: + +``` +# Bump min_mcpp ONLY together with the CI MCPP_VERSION pin — lint parses +# descriptors with the pinned mcpp, which enforces the "floor first, new +# grammar after" rollout rule mechanically. +``` + +「ONLY together with」本意是**约束**(不许单独抬下限),实际被当成**等价**(pin 动,下限跟着动)。 +证据是最近这次 bump,commit `160c389` **"feat: add boost-ext.ut 2.3.1 C++23 module package"**: + +| 文件 | 变化 | +|---|---| +| `index.toml` | `min_mcpp` `0.0.109` → `2026.8.3.3` | +| `.github/workflows/validate.yml` | `MCPP_VERSION` `0.0.109` → `2026.8.3.3` | + +而该 commit 自己的注释说明了 pin 为什么动: + +> `…this floor on macOS, and **min_mcpp/latest_mcpp move with the pin as they** …` +> `2026.8.3.3 is the pin rather than .3.1: .3.2/.3.3 are cross-compilation…` + +pin 上移的真实理由是 **mcpp#336(macOS 静态初始化)** 与交叉编译修复 —— +**与描述符语法毫无关系**。新加的 `boost-ext.ut` 描述符用的是 `generated_files`, +一个早就存在的特性;commit 信息本身还写着它是对着 **0.0.109** 开发验证的。 + +⇒ **下限上移了一百多个版本,没有任何描述符真的需要它。** +一个约一周前的发布(`0.0.109`)因此被废掉。 + +### F2 —— L1 承诺的「保留上一个兼容快照」从未实现 + +下限在**整个代码库里只被检查一次**: + +``` +$ grep -rn "check_index_floor" src/ +src/pm/package_fetcher.cppm:619 +``` + +`index_refresh.cppm`(决定「现在该不该联网刷新索引」的唯一真源)**完全不知道下限的存在**。 +所以刷新流程从不问「我拉回来的这棵树,我自己能用吗」。 + +后果正是用户遇到的:**刷新把一份能用的索引换成用不了的,且不可回退**。 +设计文档承诺的 "staged refresh keeps the last compatible snapshot" 落空了。 + +### F3 —— 违反下限降级成了「包不存在」,而不是「索引不可用」 + +`package_fetcher.cppm:614-622`: + +```cpp +// Loud once per index; the resolve then fails as not-found with the cause +// already printed. +if (auto violation = mcpp::pm::check_index_floor(pkgsDir.parent_path())) { + mcpp::ui::error(*violation); + return std::nullopt; // ← 每一次描述符读取都变成 "没找到" +} +``` + +注释承认了这个设计("the resolve then fails as not-found"),但它把**两个语义不同的事实 +压成了同一个返回值**: + +- 「这个包不在索引里」 +- 「这个索引我读不了」 + +于是解析器继续按「没找到」的剧本走:回退到 legacy 派生 wire 地址、尝试其它 repo、 +最后报一条 `E_NOT_FOUND … wire address tried: …`。**终止构建的那条错误,与真实原因隔了两层。** + +> 这个失败形态在本仓库有先例,且已被写进 `install_pinned_mcpp.sh` 的头注释: +> *"the floor check made EVERY descriptor read return nothing, every dependency fell +> back to its legacy derived address, and the build died on `mcpplibs.cmdline@0.0.1` +> — an error naming neither the version nor the floor."* +> 同一个坑,换了一个触发源,又踩了一次。 + +### F4 —— `latest_mcpp` 被每个索引写入,但**没有任何消费者** + +``` +$ grep -rn "latestMcpp\|latest_mcpp" src/ | grep -v index_contract.cppm +(空) +``` + +它被解析进 `IndexContract::latestMcpp` 就再没被读过。而它恰好就是协商层需要的那个字段。 + +### F5 —— 分发侧已经保留了全部历史快照,协商今天就做得到 + +`xlings-res/xim-index` 的 rolling `latest` release 里有 **313 个 artifact** +(`xim-index-.tar.gz`),历史快照并没有被删。 + +缺的只是**指针里没有下限信息**: + +```json +{ "format_version": 1, "index_version": "20e53c6", + "artifact": { "name": "xim-index-20e53c6.tar.gz", "sha256": "…", "size": 371707 } } +``` + +客户端在下载前无从判断这份 artifact 自己能不能用 —— 这正是设计里 **L2(projection)** +被推迟掉的那一层。**「用自己版本当前最新的 index」在分发侧已经可行,只差元数据。** + +### F6 —— 下限是**整个索引**一个开关,而需求是**逐描述符**的 + +设计明确拒绝过逐描述符下限: + +> Rejected alternative: per-descriptor `min_mcpp`. Finer-grained, but N places to +> maintain, and the failure it prevents … is already covered by one index-wide dial. + +代价是:`compat.zlib` 的描述符几个月没动过,却因为**别人**新加了一个包而变得不可读。 +索引越大、贡献越活跃,这个耦合的伤害越大 —— 而索引正在变大。 + +--- + +## 3. 根因分层 + +| 层 | 陈述 | 对应发现 | +|---|---|---| +| **L0 表层** | 违反下限终止构建,且终止在一条误导性的 `not found` 上 | F3 | +| **L1 机制** | 兼容性只在「读描述符」处检查;**刷新与下载都不检查**,所以没有「保留可用快照」这回事 | F2 | +| **L2 分发** | 只有一个 rolling `latest`,指针不带下限 ⇒ 客户端无法选择自己能用的最新快照 | F4 F5 | +| **L3 治理** | 下限的**语义**是「最老可解析客户端」,**维护方式**却是「等于 CI pin」⇒ 它以发布频率上移 | F1 | +| **L4 粒度** | 全索引一个开关,而要求是逐描述符的 ⇒ 一个包的新语法废掉整个目录 | F6 | + +**判据一句话**:*版本下限是一个路由信号,不是一个终止信号。* +客户端与索引不匹配时,正确的反应是**选一个匹配的**,选不到才报错 —— 而不是把手上能用的也丢掉。 + +--- + +## 4. 方案 + +### 4.0 核心机制:**索引历史线 + 兼容性优先选择** + +一句话:**索引不是一个 rolling 指针,而是一条带版本下限的历史线; +客户端每次刷新时,从最新往回找,取第一个自己满足下限的快照。** + +不匹配从「错误」变成「选择」——这是整份方案的骨架,其余各项都是为它服务或为它兜底。 + +#### 解析算法 + +``` +refresh(index_name, own_version): + hist = fetch_history(index_name) # 新 → 旧,含每条的 min_mcpp + latest = hist[0] + + if own_version >= latest.min_mcpp: + return use(latest) # 常规路径,与今天一致 + + # 最新的用不了 —— 不是错误,是一次路由 + notice(latest, own_version) # 见下方文案 + for snap in hist: # 继续往回找 + if own_version >= snap.min_mcpp: + return use(snap) # 「当前版本兼容下的最新 index」 + + # 整条线都不兼容(只可能发生在极老的客户端) + keep_existing_or_fail() # → R2 / R3 +``` + +判据是**单调**的:`min_mcpp` 沿历史线只增不减,所以一旦找到第一个满足的就是最优解, +不需要遍历全部 313 条。 + +#### 用户看到的(正是提问描述的那种) + +``` +note: the newest index (2026.8.3.3) requires mcpp >= 2026.8.3.3; this is mcpp 0.0.109. + Using the newest index your version supports: 0.0.109-era (indexed 2026-07-27). + To pick up newer packages, upgrade mcpp: + xlings update mcpp (or: curl -fsSL | bash) +``` + +三件事一次说清:**为什么没拿最新**、**实际用了哪个**、**怎么升级**。 +构建**继续进行**,退出码 0。 + +它只在「选择发生了降级」时打印一次,常规路径完全静默 —— 否则每次构建刷屏, +用户很快就学会无视它,那这条提示就白写了。 + +#### 历史线从哪来 + +F5 已经证明**分发侧不需要新建任何东西**:`xlings-res/xim-index` 的 rolling `latest` +release 里 313 个 artifact 都还在。缺的只是「每条的 `min_mcpp` 是多少」这份元数据。 + +指针从「一条」变成「一条 + 历史」: + +```json +{ + "format_version": 2, + "index_name": "xim", + "latest": { "index_version": "20e53c6", "min_mcpp": "2026.8.3.3", + "generated_at": "2026-08-03T10:14:21Z", "artifact": { … } }, + "history": [ + { "index_version": "20e53c6", "min_mcpp": "2026.8.3.3", "generated_at": "…", "artifact": { … } }, + { "index_version": "0adb288", "min_mcpp": "0.0.109", "generated_at": "…", "artifact": { … } }, + { "index_version": "7fef5ec", "min_mcpp": "0.0.108", "generated_at": "…", "artifact": { … } } + ] +} +``` + +- `min_mcpp` 由发布脚本从**树内 `index.toml`** 机械投影,不手写 —— + 权威仍在树内,指针只是投影,冲突以树内为准(与原设计 L2 的定性一致)。 +- `history` 只需保留**下限发生变化的那些拐点**,不必是全部 313 条: + 同一 `min_mcpp` 的连续快照里只有最新那条有意义。按 F1 的历史, + 拐点至今只有 6 个(`0.0.85 / 0.0.87 / 0.0.101 / 0.0.102 / 0.0.108 / 0.0.109 / 2026.8.3.3`)。 + **一条 6 项的历史线就能覆盖整个索引生命周期。** +- **存量 313 个快照没有这份元数据。** 三选一,建议 (b): + (a) 一次性回填(逐个解包读 `index.toml`); + (b) **只为拐点回填**(6 条,人工可核,成本几分钟); + (c) 缺失 ⇒ 视为「未知,试一次」——不推荐,把确定性换成了下载。 + +#### ⚠️ 谁来执行「回溯选择」:mcpp 今天**做不到**,这是本方案最硬的约束 + +自然的想法是「这全是客户端的事,mcpp 自己解决就好」。核实下来不成立: + +**mcpp 不下载索引。** `index_management.cppm:148` 的刷新就是 + +```cpp +int rc = mcpp::xlings::update_index(xlEnv); // → 跑 `xlings update` +``` + +artifact / git 的同步全部发生在 **xlings** 进程里。mcpp 对索引传输的全部控制权, +是它写进 `.xlings.json` 的 `index_repos` 条目 —— 而那个结构 + +```cpp +struct SeedRepo { + std::string name; + std::string url; + std::string artifact; // artifact 源基址 + std::string source; // "auto" | "artifact" | "git" +}; +``` + +**没有版本/rev 字段**。索引版本串在 mcpp 里还被显式标注为 *OPAQUE BY CONTRACT* +(`xlings.cppm:370`)。也就是说:**mcpp 无法要求「给我索引版本 X」**, +它只能说「去同步」,然后接受 xlings 给的那一份。 + +⇒ 4.0 的回溯选择需要三选一,按代价排序: + +| 路径 | 内容 | 代价 | +|---|---|---| +| **(a) 推荐** | xlings 增加「枚举索引快照 + 按版本同步」的能力,mcpp 在 `.xlings.json` 里下发(`SeedRepo` 加一个 pin 字段)。**已提 [openxlings/xlings#476](https://github.com/openxlings/xlings/issues/476)** | 跨仓协作一次,之后两边都干净 | +| (b) | mcpp 自己接管索引 artifact 的下载与解包 | 与 xlings 职责重叠、镜像/CDN/校验逻辑要重写一遍 | +| (c) | 只对 mcpp 自己的索引(`mcpplibs/mcpp-index`)实现,xim 索引靠 R2 兜底 | 覆盖面不完整,但**不需要任何跨仓改动** | + +**这不影响 R2。** R2(保留/回滚到上一个可用快照)**完全在 mcpp 侧、今天就能做**: +刷新后检查下限,违反就把索引目录回滚到刷新前的备份。它不需要选择任何版本, +只需要「不要用坏的覆盖好的」。**「变砖 → 可用 + 提示升级」这一步不依赖任何人。** + +#### git 传输的索引怎么办 + +`mcpp-index` 同时是一个 git 仓库(`[indices] git =`)。历史线在那里是**天然存在**的: +`index.toml` 的每一次 `min_mcpp` 变更就是一个拐点 commit。等价实现是 +**按 commit 回溯**取第一个兼容的树,或(更省事、更可审计)由索引仓库为每个拐点打 tag: + +``` +index/min-mcpp-0.0.109 +index/min-mcpp-2026.8.3.3 +``` + +客户端 `git fetch --tags` 后按 tag 选择即可,无需 clone 全史。 +本地 `path =` 索引不参与历史线(它就是用户自己的目录),保持现状:违反 ⇒ R1 报错。 + +#### 离线 + +历史线的选择结果必须**落盘记录**(哪个 index_version、什么下限、为什么选它)。 +离线时不联网、不重新选择,直接用上次选中的快照 —— 与 `index_refresh` 现有的 +offline-first 判据(「所有解析输入都在磁盘」)一致,不新增网络依赖。 + +--- + +以下各项按「是否为 4.0 所必需」排序。 + +### R1(必须)· 违反下限必须是「索引不可用」,不能退化成「包不存在」 + +`package_fetcher` 的读取路径要能区分三种结果,而不是两种: + +```cpp +enum class DescriptorLookup { Found, Absent, IndexUnusable }; +``` + +- `IndexUnusable` **不得**触发 legacy 地址回退、不得被其它 repo 的 `Absent` 掩盖、 + 不得被 `index_refresh` 读成「本地没有 ⇒ 该刷新了」(否则形成刷新抖动: + 拉回同一份不兼容的树,再失败,再刷新)。 +- 解析结束时若有索引处于 `IndexUnusable`,**最终错误就是 E0006 本身**。 + 用户看到的第一条与最后一条错误必须是同一件事。 + +4.0 落地后这条路径应当**几乎不可达**(选择阶段就避开了不兼容的树)—— +但它必须存在,因为 `path =` 索引、手工放置的树、以及历史线整条不兼容时仍会走到它。 + +### R2(必须)· 选择失败时保留上一个可用快照,绝不用不可用的覆盖可用的 + +补上 L1 承诺过而没实现的那一半(F2)。即使有了 4.0,这一条仍然必须: +历史线获取失败(网络、镜像、artifact 缺失)时,**手上那份能用的索引就是最后的防线**。 + +1. 下载/解包后、切换生效前,对**暂存的**树跑 `check_index_floor`; +2. 违反 ⇒ 不切换,保留现有索引并 warn; +3. 本机完全没有可用索引时才落到 R3。 + +xim 索引的下载/解包由 **xlings** 执行,不在 mcpp 进程内。在拿到「解包到暂存目录」 +的能力之前,退化实现同样有效:刷新后立即检查,违反则**回滚到刷新前的快照** +(保留一份 `last-compatible` 备份),并在本进程内标记该索引不可再刷新。 + +> 这一条修掉的是最伤人的性质:**刷新让情况变得更糟,且不可逆。** + +### R3(必须)· 无任何可用树时,一次说清楚 + +冷启动且历史线整条不兼容(或不可达)时,只报 E0006,指明**哪个索引、要求什么、本机是什么**, +并给出升级命令。保留 `MCPP_INDEX_FLOOR=ignore` 作为调试逃生舱。 + +### R4(应做,治理)· 下限必须与 CI pin 解耦,并由 CI 证明 + +**优先级说明(修订)**:早先的排序把这条摆得过重。4.0 + R2 落地后,下限乱涨的危害 +从「变砖」降到「旧客户端停在老索引、拿不到新包,但构建正常」。所以这条是 **hygiene, +不是阻塞项** —— 它值得做,因为无声地把所有旧客户端钉在老索引上仍然是损失, +而且这个损失**不再有任何报错提醒任何人**;但它不该排在 R1/R2 前面。 + +1. **改掉 `index.toml` 的维护规则**:从「与 CI pin 一起 bump」改为 + **「只有当某个描述符真的用了旧客户端解析不了、或会静默错解的东西时才抬」**, + 并在 commit 里点名是哪个描述符、哪个特性。 +2. **让 CI 证明它**:索引 CI 现在只用 pin 的 mcpp 跑 lint(只能证明「新版本能解析」)。 + 应当**再用当前 `min_mcpp` 那个版本跑一遍**: + - 旧版本能全部解析 ⇒ **下限不许动**; + - 旧版本解析失败 ⇒ 下限**必须**抬到能解析的最低版本,且 CI 报出是哪个文件。 + 这把「floor first, migration after」从**约定**变成**机器判据**。 +3. **`latest_mcpp` 与 `min_mcpp` 分开**:前者跟 pin 走(「已知最佳」,也是 4.0 提示文案里 + 建议升级到的目标版本 —— F4 指出它目前无人消费,这是它的第一个真实用途), + 后者只在 (2) 判定必须时才动。今天两者恒等,等于把「最佳」当成了「最低」。 + +### R5(应做)· 逐描述符下限 + +原设计以「N places to maintain」拒绝过它。这个理由在贡献者变多后不再成立: +维护成本落在**新增该描述符的那个 PR**(一行), +而现状的成本落在**所有旧客户端**(整个目录不可读,F6)。 + +- 索引级 `min_mcpp` 保留,语义收紧为「解析这棵树的目录结构与通用字段」所需 + (真正的全局语法变更,如 SPEC-001 身份形态迁移); +- 单个描述符可选声明 `mcpp = ">= X"`,客户端不满足时**只跳过该描述符**并 warn。 + +这样「新加一个包」在结构上不可能再废掉整个索引 —— F6 的根治。 +它与 4.0 是互补的:4.0 让**客户端**总能找到能用的快照,R5 让**索引**不必因为一个包而整体后退。 + +### R6(可选)· 版本化通道(原设计 L3) + +`index.toml` 的 `spec` 作为通道键,`mcpp-index-v1-latest` / `-v2-latest`。 +4.0 落地后增量收益不大,**不建议现在做**;但指针形状应留出扩展位(`format_version` 已在)。 + +## 5. 不采纳 + +| 方案 | 理由 | +|---|---| +| 直接把下限检查改成 warning、继续解析 | 下限存在的理由是真的:`0.0.101` 那次是 `compat.opencv` 的 per-OS feature flags 被旧客户端**静默忽略**,产出错误的构建。放行 = 用「静默错误产物」换「明确失败」,方向反了。R1/R2 保住的是**旧快照**,不是**用旧客户端读新描述符**。 | +| 让旧客户端忽略看不懂的字段 | 同上:`0.0.101` 的教训正是「忽略 = 静默错」。 | +| 索引不再抬下限 | 语法要演进。问题不在抬,在**抬的理由**(F1)和**抬之后旧客户端没有退路**(F2)。 | +| 只做 R4(分发侧协商),不做 R2 | 冷启动能救,但**已经装好的用户在 `xlings update` 后依然会被换成不可用的索引**——最伤人的那条路径没修。 | +| 只做 R2,不做 R5 | 旧客户端不再变砖,但下限仍以发布频率上移,**所有人都会持续拿不到新包**,只是不再报错。症状消失,问题还在。 | + +--- + +## 6. 实施顺序(修订) + +排序的依据是**受众**,不是技术难度。 + +### 为什么索引侧那一步删不掉 + +**任何 mcpp 侧修复都救不了已经发出去的 0.0.109。** 修复最早出现在 `2026.8.3.5`, +而现在被卡住的用户手里就是 0.0.109。他们只有两条路:**升级 mcpp**, +或者**索引把下限降回去**。所以: + +- **索引侧回退** = 针对**存量用户**的唯一无痛解(且不需要发 mcpp) +- **mcpp 侧修复** = 针对**未来所有版本偏斜**的根治(但只对未来的版本生效) + +两者受众不同,不是重复劳动。这也是为什么它排第一 —— 不是因为它更重要, +而是因为它是唯一今天就能让人恢复的动作。 + +### 顺序 + +| # | 动作 | 侧 | 受众 / 效果 | 依赖 | +|---|---|---|---|---| +| 1 | `min_mcpp` 回退到真实需要的值(`latest_mcpp` 保留 `2026.8.3.3`) | 索引 | **存量 0.0.109 用户立刻恢复** | 无 | +| 2 | R1 + R3:错误语义(`IndexUnusable` ≠ `Absent`) | mcpp | 可诊断性;4.0 的兜底 | 无 | +| 3 | **R2:刷新闸 + 回滚到上一个可用快照** | mcpp | **「变砖 → 可用 + 提示升级」** | **无** | +| 4 | 4.0 历史线:先走路径 (c)(只覆盖 mcpp 自己的索引) | mcpp | 「拉取当前版本兼容下的最新 index」 | 无 | +| 5 | 4.0 完整覆盖:xlings 加索引版本 pin,`SeedRepo` 下发([xlings#476](https://github.com/openxlings/xlings/issues/476)) | xlings + mcpp | xim 索引也进入历史线 | 跨仓,**不阻塞 1–4** | +| 6 | R4:索引 CI 双版本 lint + 维护规则改写 | 索引 | hygiene:阻止下限无理由上移 | 无 | +| 7 | R5:逐描述符下限 | mcpp + 索引 | 一个包不再废掉整个目录 | 无 | +| 8 | R6:版本化通道 | — | 视生态规模再议 | — | + +**第 3 步是性价比最高的一步**:纯 mcpp 侧、不依赖任何人、不需要任何新协议, +就把最伤人的性质(刷新让情况变得更糟且不可逆)修掉。 +第 4 步给出你要的「历史线」语义,第 5 步才把它铺满整个生态。 + +--- + +## 7. 验证 + +- **回归 e2e**:fixture 索引 `min_mcpp = "9.9.9"`。当前断言是「build fails」; + R2 之后应改为**「保留旧索引、构建成功、打出升级 warning」**,并新增一条 + 「本机无任何兼容索引 ⇒ 只报 E0006,且 stderr 中不出现 `not found`」。 +- **F3 的判据要写成断言**:违反下限的构建,其**最后一条**错误必须包含 `E0006`。 + 这条断言正是这次(以及 `install_pinned_mcpp.sh` 记载的上一次)缺失的那个。 +- **R5(2) 的自证**:在索引 CI 里故意把 `min_mcpp` 调低一档,双版本 lint 必须变红。 + +--- + +## 8. 附:这次事件的时间线(供复盘) + +| 时间 | 事件 | +|---|---| +| 2026-07-08 | 设计确立 L1/L2/L3;L2、L3 判为「operationally premature」推迟 | +| 2026-07-08 | L1 落地,但只实现了 choke-point 检查;**「保留上一个兼容快照」未实现**(F2) | +| 2026-07-21 | 下限 `0.0.87` → `0.0.101`,理由真实(opencv feature flags 被静默忽略) | +| 2026-07-26 | 下限 → `0.0.108`/`0.0.109`,理由真实(SPEC-001 身份迁移) | +| 2026-08-03 | 下限 `0.0.109` → `2026.8.3.3`,**理由不成立**:pin 因 mcpp#336(macOS) 上移,下限机械跟随(F1) | +| 2026-08-03 | 用户 `0.0.109` 变砖,且报错指向 `not found`(F3) | + +前两次 bump 是这个机制**正确工作**的样子;第三次是它**语义漂移**的样子。 +区别不在于机制,而在于「谁来证明这次 bump 是必要的」—— 目前没有人,也没有机器。 diff --git a/.agents/skills/mcpp-release/SKILL.md b/.agents/skills/mcpp-release/SKILL.md index 7a656eac..84c65b7b 100644 --- a/.agents/skills/mcpp-release/SKILL.md +++ b/.agents/skills/mcpp-release/SKILL.md @@ -31,7 +31,7 @@ mcpp 有 **三个持久化版本位置**,以及 `ci-fresh-install` 的一个 **第一组:正在构建的版本**(发布时改,走 bump PR) 1. `mcpp.toml` → `[package].version` — 构建系统读取的项目版本,release.yml 由它推导 tag -2. `src/toolchain/fingerprint.cppm` → `MCPP_VERSION` — 编译期硬编码常量(`--version` 输出、BMI 指纹、E0006 索引底线比较) +2. `src/version.cppm` → `MCPP_VERSION` — 编译期硬编码常量(`--version` 输出、BMI 指纹、E0006 索引底线比较) 这两处必须**在同一个 commit 里**一起改:`tests/e2e/01_help_and_version.sh` 交叉比对 `mcpp.toml` 与 `mcpp --version`,只改一处 CI 立刻红。 @@ -89,7 +89,7 @@ NEW_VERSION="2026.7.27.1" git checkout -b "chore/bump-$NEW_VERSION" sed -i "s/^version.*=.*/version = \"$NEW_VERSION\"/" mcpp.toml -sed -i "s/MCPP_VERSION = \".*\"/MCPP_VERSION = \"$NEW_VERSION\"/" src/toolchain/fingerprint.cppm +sed -i "s/MCPP_VERSION = \".*\"/MCPP_VERSION = \"$NEW_VERSION\"/" src/version.cppm # 校验:mcpp.toml 与 MCPP_VERSION 相等,.xlings.json 不领先于正在构建的版本。 bash .github/tools/check_version_pins.sh @@ -309,7 +309,7 @@ gh workflow run release.yml --ref "v$NEW_VERSION" | 文件 | 版本相关内容 | |------|-------------| | `mcpp.toml` | `version = "X.Y.Z"` — 项目版本,release.yml 由它推导 tag | -| `src/toolchain/fingerprint.cppm` | `MCPP_VERSION = "X.Y.Z"` — 编译期版本常量 | +| `src/version.cppm` | `MCPP_VERSION = "X.Y.Z"` — 编译期版本常量 | | `.xlings.json` | `workspace.mcpp` — CI bootstrap 装哪个 mcpp(发布**后**才 bump) | | `.github/workflows/ci-fresh-install.yml` | `MCPP_PIN` — 由 `wait-index` 从最新 release 推导,**从不手工 bump** | | `src/xlings.cppm` | `kXlingsVersion` — xlings pin 的**唯一真源** | diff --git a/.github/tools/check_version_pins.sh b/.github/tools/check_version_pins.sh index 604cbcd4..36b6fecc 100755 --- a/.github/tools/check_version_pins.sh +++ b/.github/tools/check_version_pins.sh @@ -100,13 +100,13 @@ done < <(find .github -type f \( -name '*.yml' -o -name '*.yaml' -o -name '*.sh' # ── 2. mcpp's own version ───────────────────────────────────────────────── v_toml=$(awk -F '"' '/^version[[:space:]]*=/{print $2; exit}' mcpp.toml) -v_src=$(grep -oE 'MCPP_VERSION[[:space:]]*=[[:space:]]*"[^"]+"' src/toolchain/fingerprint.cppm \ +v_src=$(grep -oE 'MCPP_VERSION[[:space:]]*=[[:space:]]*"[^"]+"' src/version.cppm \ | grep -oE '"[^"]+"' | tr -d '"' | head -1) v_xl=$(grep -oE '"mcpp"[[:space:]]*:[[:space:]]*"[^"]+"' .xlings.json \ | grep -oE '"[^"]+"$' | tr -d '"' | head -1) -note "mcpp version: building=$v_toml (fingerprint=$v_src) bootstrap pin=$v_xl" +note "mcpp version: building=$v_toml (src/version.cppm=$v_src) bootstrap pin=$v_xl" -for n in "mcpp.toml:$v_toml" "src/toolchain/fingerprint.cppm:$v_src" \ +for n in "mcpp.toml:$v_toml" "src/version.cppm:$v_src" \ ".xlings.json:$v_xl"; do [ -n "${n##*:}" ] || bad "${n%:*} — could not read the mcpp version" done @@ -131,7 +131,7 @@ fi # same number by definition — release.yml derives the tag from the former # and the smoke test greps the latter out of `mcpp --version`. [ -z "$v_src" ] || [ "$v_src" = "$v_toml" ] \ - || bad "src/toolchain/fingerprint.cppm has '$v_src' but mcpp.toml has '$v_toml'" + || bad "src/version.cppm has '$v_src' but mcpp.toml has '$v_toml'" # (b) The version BOOTSTRAPPED FROM (.xlings.json) names a mcpp that is already # published, and is NOT required to equal the version being built. It diff --git a/CHANGELOG.md b/CHANGELOG.md index cf6240cd..3a7d4bd5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,32 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.8.3.5] — 2026-08-03 + +### 修复 + +- **索引侧的改动不再能让 mcpp 从「能用」变成「不能用」。** 一个索引树可以声明客户端版本下限(`index.toml` `min_mcpp`)。此前 `xlings update` 会**原地覆盖**本地索引,一旦上游抬了下限,一台一分钟前还好好的机器就报 + + ``` + error: index requires mcpp >= 2026.8.3.3 but this is mcpp 0.0.109 [E0006] + error: ... package 'compat:compat.libarchive@3.8.7' not found + ``` + + 而且**没有退路** —— 旧的、能用的那份树已经被覆盖掉了。**刷新本身就是把机器弄坏的那个动作。** + + 刷新现在是**单调**的:判据是「刷新前能用 ⇒ 刷新后必须仍能用」。`mcpp::xlings::update_index`(全进程唯一的刷新入口)在同步前归档每一棵可读的索引树,同步后复核;变得不可读就回滚,并提示可以升级 mcpp 拿新包。2026-07-08 的索引设计里写过这个行为("staged refresh keeps the last compatible snapshot"),**从未被实现** —— 下限只在描述符读取处检查过一次,刷新路径对它一无所知。 + +- **不再把「这个索引读不了」伪装成「这个包不存在」。** 违反下限会让该树的每次描述符读取返回空,与「包确实不在」不可区分。后果有两层:终止构建的那条错误是 `dependency '...': not found`,**把责任推给包**(指向发布/命名),而真正的答案(升级 mcpp)早已滚出屏幕;同一个无法区分的 miss 还会喂给刷新策略,被读成「本地缺东西,去拉」—— **不可用状态自己在驱动重复刷新**。 + + 「这个索引不可读」现在是一个可查询的事实:刷新策略据此不再徒劳重试,而每一处终止构建的 not-found 都会带上真正的原因。 + +### 改进 + +- **`mcpp cache`/索引之外的一处分层修正:`MCPP_VERSION` 移到叶子模块 `src/version.cppm`。** 它此前住在 `mcpp.toolchain.fingerprint` 里,而后者 `import mcpp.toolchain.detect` → `import mcpp.xlings`,于是「这个二进制是什么版本」这件事**传递依赖了整个工具链探测与包管理子系统**。这不是洁癖:`mcpp.pm.index_contract` 只为一次比较需要这个常量,却因此依赖上 xlings,反过来使得 xlings **不可能**依赖 index_contract —— 而刷新守卫恰恰必须装在那里。**编译器报的那个循环依赖,就是分层在告诉我们常量放错了地方。** 版本号的唯一真源随之移动;`check_version_pins.sh` 与发布文档同步更新。 + +- **补上了从未存在的端到端覆盖。** 下限行为此前只有纯谓词单测(`floor_violation()`),`tests/e2e/` 里一条都没有 —— 这正是上面两个缺陷能长期存在的原因:谓词是对的,**谓词周围的行为从没被端到端跑过**。新增 `tests/e2e/185_index_floor_degrades.sh`(错误必须点名真正的原因、不可用索引不得波及健康索引、逃生舱仍有效)与 `tests/unit/test_index_snapshot.cpp`(9 条,含「刷新弄坏索引必须回滚」)。 + + ## [2026.8.3.4] — 2026-08-03 ### 修复 diff --git a/docs/09-release.md b/docs/09-release.md index 3408eddb..3f575b4f 100644 --- a/docs/09-release.md +++ b/docs/09-release.md @@ -11,7 +11,7 @@ of those commit messages contains a misdiagnosis that is corrected in §5. | Site | Group | Moves when | |---|---|---| | `mcpp.toml` `[package].version` | **being built** | you start work on a new version | -| `src/toolchain/fingerprint.cppm` `MCPP_VERSION` | **being built** | same commit as above (compiled-in copy) | +| `src/version.cppm` `MCPP_VERSION` | **being built** | same commit as above (compiled-in copy) | | `.xlings.json` `[workspace].mcpp` | **bootstrapped from** | separately, *after* a release is installable | | `ci-fresh-install.yml` `MCPP_PIN` | **version under test** | **nothing — it is derived at run time** (§5) | diff --git a/docs/zh/09-release.md b/docs/zh/09-release.md index e85a2b73..0c1577ec 100644 --- a/docs/zh/09-release.md +++ b/docs/zh/09-release.md @@ -11,7 +11,7 @@ | 位置 | 组 | 何时变 | |---|---|---| | `mcpp.toml` `[package].version` | **正在构建的** | 开始做新版本时 | -| `src/toolchain/fingerprint.cppm` `MCPP_VERSION` | **正在构建的** | 与上一行同一个 commit(编译进二进制的副本) | +| `src/version.cppm` `MCPP_VERSION` | **正在构建的** | 与上一行同一个 commit(编译进二进制的副本) | | `.xlings.json` `[workspace].mcpp` | **自举起点** | 单独地、在某个版本**已可安装之后** | | `ci-fresh-install.yml` `MCPP_PIN` | **被测版本** | **不变 —— 运行时推导**(§5) | diff --git a/mcpp.toml b/mcpp.toml index fbd2b5d5..b557aa8e 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.3.4" +version = "2026.8.3.5" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index ea67c6ba..c6f416d6 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -41,6 +41,7 @@ import mcpp.fetcher; import mcpp.fetcher.progress; import mcpp.pm.resolver; import mcpp.pm.index_spec; +import mcpp.pm.index_contract; import mcpp.pm.index_route; import mcpp.pm.index_refresh; import mcpp.pm.mangle; @@ -819,6 +820,20 @@ std::string git_cache_head(const std::filesystem::path& gitRoot) { // extraTargets: additional Target entries (e.g. synthetic test targets) // appended to the manifest before the modgraph runs. // overrides: --target / --static. +namespace { +// A dependency that "cannot be found" while an index is unreadable is almost +// never missing — it is unreachable, and the two need different actions from +// the user (publish it vs upgrade mcpp). The floor error is printed when the +// index is first opened, which can be hundreds of lines earlier; the message +// that STOPS the build has to carry the cause, because that is the one a user +// reads. See mcpp::pm::unusable_index_hint. +std::string with_index_cause(std::string msg) { + if (auto hint = mcpp::pm::unusable_index_hint(); !hint.empty()) + msg += "\n" + hint; + return msg; +} +} // namespace + export std::expected prepare_build(bool print_fingerprint, bool includeDevDeps = false, @@ -1975,7 +1990,8 @@ prepare_build(bool print_fingerprint, auto candidates = dependencyCoordinates(spec, depName); if (candidates.empty()) { return std::unexpected( - std::format("dependency '{}' has no lookup candidates", depName)); + with_index_cause(std::format( + "dependency '{}' has no lookup candidates", depName))); } auto selected = candidates.front(); @@ -2146,9 +2162,9 @@ prepare_build(bool print_fingerprint, auto luaContent = readLuaContent(); if (idxSpec && idxSpec->is_local() && !luaContent) { auto indexPath = mcpp::config::resolve_project_index_path(*root, *idxSpec); - return std::unexpected(std::format( + return std::unexpected(with_index_cause(std::format( "dependency '{}': not found in local index at '{}'", - depName, indexPath.string())); + depName, indexPath.string()))); } auto findRawInstalled = [&]() -> std::optional { @@ -2439,8 +2455,8 @@ prepare_build(bool print_fingerprint, if (!luaContent) { luaContent = readLuaContent(); } - if (!luaContent) return std::unexpected(std::format( - "dependency '{}': index entry not found in local clone", depName)); + if (!luaContent) return std::unexpected(with_index_cause(std::format( + "dependency '{}': index entry not found in local clone", depName))); auto field = mcpp::manifest::extract_mcpp_field(*luaContent); // 0.0.6+: read explicit namespace from xpkg lua if present. diff --git a/src/pm/index_contract.cppm b/src/pm/index_contract.cppm index 1ee3d4a9..0637da8c 100644 --- a/src/pm/index_contract.cppm +++ b/src/pm/index_contract.cppm @@ -22,7 +22,7 @@ export module mcpp.pm.index_contract; import std; import mcpp.libs.toml; import mcpp.version_req; -import mcpp.toolchain.fingerprint; // MCPP_VERSION +import mcpp.version; // MCPP_VERSION (leaf — see that module) export namespace mcpp::pm { @@ -44,13 +44,55 @@ read_index_contract(const std::filesystem::path& indexRoot); std::optional floor_violation(std::string_view minMcpp, std::string_view ownVersion); +// Pure predicate — no reporting, no registration, no dedup. For callers that +// need to ask "would this tree be usable?" without the side effects of +// check_index_floor (the refresh guard asks it twice per refresh). +bool index_usable(const std::filesystem::path& indexRoot); + // Open-time check for an index tree. Combines read + floor + escape // hatch + once-per-root deduplication of the (expensive to spam) error. // Returns the violation message the FIRST time a too-new tree is opened; -// nullopt otherwise. +// nullopt otherwise. Also RECORDS the fact (see below). std::optional check_index_floor(const std::filesystem::path& indexRoot); +// ── "this index is unusable" as a first-class, queryable fact ────────── +// +// A floor violation makes every descriptor read from that tree return nothing, +// which is indistinguishable from "the package genuinely is not in this index" +// at the call site. Two things downstream need to tell them apart: +// +// * the refresh policy — a miss caused by an unusable index will NOT be +// fixed by fetching the same tree again, and treating it as a normal miss +// makes the unusable state drive repeated refreshes of itself; +// * the final error — "not found" names neither the version nor the floor, +// so the message that stops the build has to reach back for the real cause. +// +// Process-global on purpose: the fact depends only on (the tree on disk, this +// binary's version), and neither can change within a process. Recording it is +// what lets the two consumers above stay honest without threading a tri-state +// through every read_xpkg_lua entry point and all of their callers. +struct UnusableIndex { + std::filesystem::path root; + std::string message; // the full E0006 text, ready to print +}; + +// True when any index tree opened in this process failed its floor check. +bool any_index_unusable(); + +// True when THIS tree failed (exact root match). +bool index_marked_unusable(const std::filesystem::path& indexRoot); + +// Every index that failed, in first-seen order. +std::vector unusable_indexes(); + +// One line for appending to an unrelated failure ("… and by the way, an index +// was unusable, which is probably why"). Empty when nothing was unusable. +std::string unusable_index_hint(); + +// Testing only: forget everything recorded so far. +void reset_unusable_indexes_for_test(); + } // namespace mcpp::pm namespace mcpp::pm { @@ -93,6 +135,51 @@ floor_violation(std::string_view minMcpp, std::string_view ownVersion) minMcpp, ownVersion); } +bool index_usable(const std::filesystem::path& indexRoot) +{ + if (const char* v = std::getenv("MCPP_INDEX_FLOOR"); + v && std::string_view(v) == "ignore") + return true; + auto c = read_index_contract(indexRoot); + if (!c) return true; // no contract → no constraint + return !floor_violation(c->minMcpp, mcpp::MCPP_VERSION); +} + +namespace { +// See the header comment on UnusableIndex for why this is process-global. +std::vector& unusable_registry() { + static std::vector reg; + return reg; +} +} // namespace + +bool any_index_unusable() { return !unusable_registry().empty(); } + +bool index_marked_unusable(const std::filesystem::path& indexRoot) { + for (auto& u : unusable_registry()) + if (u.root == indexRoot) return true; + return false; +} + +std::vector unusable_indexes() { return unusable_registry(); } + +std::string unusable_index_hint() { + auto& reg = unusable_registry(); + if (reg.empty()) return {}; + // Name the index, not just the fact: with several repos configured, "an + // index was too new" leaves the reader guessing which one to act on. + std::string s = "note: this resolve ran with an index this mcpp cannot read:\n"; + for (auto& u : reg) { + s += " " + u.root.string() + "\n"; + } + s += " Packages served by it were reported as not found. See the " + "[E0006] error above,\n" + " or run `mcpp explain E0006`."; + return s; +} + +void reset_unusable_indexes_for_test() { unusable_registry().clear(); } + std::optional check_index_floor(const std::filesystem::path& indexRoot) { @@ -100,14 +187,19 @@ check_index_floor(const std::filesystem::path& indexRoot) v && std::string_view(v) == "ignore") return std::nullopt; - // Once per root per process: the same index is opened many times in a - // single resolve; report the violation once, stay quiet after. - static std::set reported; auto c = read_index_contract(indexRoot); if (!c) return std::nullopt; - auto violation = floor_violation(c->minMcpp, mcpp::toolchain::MCPP_VERSION); + auto violation = floor_violation(c->minMcpp, mcpp::MCPP_VERSION); if (!violation) return std::nullopt; - if (!reported.insert(indexRoot).second) return std::nullopt; + + // Record BEFORE the dedup return: the fact must be queryable no matter how + // many times this root is opened, while the message is printed only once. + // Deriving "was anything unusable?" from "did we print?" is what made the + // second and later reads indistinguishable from an ordinary miss. + if (!index_marked_unusable(indexRoot)) + unusable_registry().push_back({indexRoot, *violation}); + else + return std::nullopt; // already reported — stay quiet return violation; } diff --git a/src/pm/index_refresh.cppm b/src/pm/index_refresh.cppm index eddd14b0..d770fa31 100644 --- a/src/pm/index_refresh.cppm +++ b/src/pm/index_refresh.cppm @@ -41,6 +41,7 @@ import mcpp.log; import mcpp.platform; import mcpp.platform.axis; import mcpp.pm.dep_spec; +import mcpp.pm.index_contract; import mcpp.pm.index_route; import mcpp.pm.resolver; import mcpp.ui; @@ -60,6 +61,7 @@ enum class RefreshReason { SuppressedDisabled, // [index] auto_refresh = false SuppressedDebounce, // refreshed moments ago; upstream simply lacks it SuppressedInconclusive, // a miss here proves nothing (see header) + SuppressedIndexUnusable,// the index that would answer is too new to read }; struct RefreshPolicy { @@ -186,6 +188,8 @@ std::string_view reason_text(RefreshReason r) { case RefreshReason::SuppressedDisabled: return "[index] auto_refresh = false"; case RefreshReason::SuppressedDebounce: return "index was just refreshed"; case RefreshReason::SuppressedInconclusive: return "no index can refute this"; + case RefreshReason::SuppressedIndexUnusable: + return "an index requires a newer mcpp — refreshing cannot help"; } return ""; } @@ -255,6 +259,18 @@ RefreshDecision decide_for_dependency(const IndexRoute& route, if (!d.shouldRefresh) return d; // nothing to suppress + // An index this binary cannot READ makes every descriptor lookup in it come + // back empty, and that miss is indistinguishable from "the package is not + // there" at the call site. Refreshing would fetch the same unreadable tree + // again — so the miss would repeat, and the unusable state would drive + // repeated refreshes of itself. Stop here and let the E0006 error stand as + // the explanation. + if (mcpp::pm::any_index_unusable()) { + d.shouldRefresh = false; + d.reason = RefreshReason::SuppressedIndexUnusable; + return d; + } + // 7. Opt-outs, in order of authority: the user's flag, the machine's // config, then the "we just did this" guard. if (policy.offline) { d.shouldRefresh = false; d.reason = RefreshReason::SuppressedOffline; return d; } @@ -281,6 +297,11 @@ RefreshDecision decide_for_miss(const RefreshPolicy& policy, std::string_view subject) { RefreshDecision d; d.subject = std::string(subject); + // See decide(): a refresh cannot fix an index this binary cannot read. + if (mcpp::pm::any_index_unusable()) { + d.reason = RefreshReason::SuppressedIndexUnusable; + return d; + } if (policy.offline) { d.reason = RefreshReason::SuppressedOffline; return d; } if (!policy.autoRefresh) { d.reason = RefreshReason::SuppressedDisabled; return d; } // Same debounce as the build path: two `mcpp add` typos in a row should not diff --git a/src/pm/index_snapshot.cppm b/src/pm/index_snapshot.cppm new file mode 100644 index 00000000..1b1f60fc --- /dev/null +++ b/src/pm/index_snapshot.cppm @@ -0,0 +1,354 @@ +// mcpp.pm.index_snapshot — local, known-good copies of index trees, and the +// guard that makes a refresh MONOTONE. +// +// THE INVARIANT +// +// An index-side change must never take mcpp from "works" to "does not work". +// +// It did not hold. An index tree can declare a client-version floor +// (`index.toml` `min_mcpp`, mcpp.pm.index_contract). When the published index +// raises that floor, `xlings update` replaces the local tree in place with one +// this binary cannot read, every descriptor read starts returning nothing, and +// the build dies — on a machine that was working sixty seconds earlier. There +// was no backup, so there was no way back either: the refresh is what broke it. +// +// The 2026-07-08 index design specified the cure ("staged refresh keeps the +// last compatible snapshot") and it was never implemented — the floor was +// checked in exactly one place, the descriptor reader, and the refresh path +// knew nothing about it. +// +// WHY BACKUP-AND-ROLLBACK RATHER THAN STAGE-AND-SWAP +// +// mcpp does not fetch indexes. `mcpp::xlings::update_index` shells out to +// `xlings update`, which rewrites the tree in place; mcpp cannot ask it to +// unpack somewhere else (the `index_repos` entry it seeds carries name / url / +// artifact / source and no version or destination). So the sequence is +// archive → let the refresh happen → judge the result → restore if it got +// worse. Same guarantee, no cross-repo dependency. +// +// The judgement is deliberately "did it get WORSE", not "is it good": +// +// usable before && !usable after → restore +// +// A tree that was already unusable has nothing to roll back to, and restoring +// an equally-unusable snapshot over a fresh one would only make the next +// refresh redo the same work. +// +// WHY KEEP MORE THAN ONE +// +// One backup answers "undo the refresh that just happened". A short history +// answers "give me the newest snapshot this binary can actually read", which +// is the same question the publishing side would answer with a version +// negotiation protocol (openxlings/xlings#476) — locally, for free, and today. +// It only covers snapshots this machine has seen, which is the overwhelmingly +// common case for a development box or a CI runner with a warm cache; a fresh +// install that meets a too-new index has no local history and correctly falls +// through to "upgrade mcpp". +// +// COST +// +// Index trees are small (measured: 912 KB for mcpplibs, 2.2 MB for +// xim-pkgindex) and archiving uses hard links where the filesystem allows it, +// so a snapshot costs approximately one directory walk and no extra bytes. + +export module mcpp.pm.index_snapshot; + +import std; +import mcpp.pm.index_contract; + +export namespace mcpp::pm::index_snapshot { + +// How many known-good snapshots to keep per index tree. Small on purpose: the +// value of the history is "the last few floors", and floors move rarely (the +// mcpp index has changed its floor 6 times in its entire life). +inline constexpr std::size_t kKeepPerIndex = 5; + +// The snapshot store is a SIBLING of the index data root, never inside it: +// +// /.. index-snapshots/// +// +// Everything that enumerates index repos does it by listing directories under +// `data/` (mcpp's own Fetcher::sorted_index_dirs takes every subdirectory, and +// xlings walks the same tree). A snapshot store living there would be handed to +// those scanners as if it were an index. Today that happens to be harmless — +// the store has no `pkgs/` so lookups miss and move on — but "harmless because +// of a detail of someone else's loop" is not a property worth depending on, +// and it costs nothing to put the store where no index scanner can reach it. +std::filesystem::path snapshots_root(const std::filesystem::path& dataRoot); +std::filesystem::path snapshot_dir(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir); + +// Identity of the tree currently in `indexDir`, taken from the marker xlings +// writes (`.xlings-index-version`). OPAQUE BY CONTRACT — a short sha for the +// artifact transport, a version string for others; never parsed, only compared +// and used as a directory name. Empty when the marker is absent, in which case +// the caller falls back to a content-independent name. +std::string snapshot_id(const std::filesystem::path& indexDir); + +// Archive `indexDir` under the snapshots root. No-op (returns false) when the +// tree is unusable — a snapshot exists to be restored, and restoring an +// unusable tree helps nobody. Overwrites an existing snapshot with the same id. +bool archive(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir); + +// Restore a specific snapshot over `indexDir`. The destination is replaced. +bool restore(const std::filesystem::path& snapshotPath, + const std::filesystem::path& indexDir); + +// Newest-first list of archived snapshots for one index tree. +std::vector +list_snapshots(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir); + +// The newest archived snapshot this binary can actually read, if any. +std::optional +newest_usable(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir); + +// Drop all but the newest `keep` snapshots of one index tree. +void prune(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir, + std::size_t keep = kKeepPerIndex); + +// Index trees directly under `dataRoot`: a directory containing `pkgs/`. +std::vector +index_dirs(const std::filesystem::path& dataRoot); + +// What a guarded refresh did, for the caller to report. +struct GuardOutcome { + // Index trees that were usable before the refresh and unusable after, and + // were therefore rolled back. + std::vector rolledBack; + // Index trees restored from an older snapshot because the refreshed tree + // was unusable and no pre-refresh tree existed to keep. + std::vector recovered; + // Index trees left unusable — nothing local could serve them. + std::vector stillUnusable; + + bool degraded() const { + return !rolledBack.empty() || !recovered.empty() || !stillUnusable.empty(); + } +}; + +// Run `refresh` with the monotonicity guarantee over every index tree under +// `dataRoot`. Returns the refresh's own exit code; `out` describes what the +// guard had to do. +// +// `refresh` is a callable so this module stays free of any dependency on the +// xlings process layer — and so it is testable without a network. +int guarded_refresh(const std::filesystem::path& dataRoot, + const std::function& refresh, + GuardOutcome& out); + +} // namespace mcpp::pm::index_snapshot + +namespace mcpp::pm::index_snapshot { + +namespace { + +// A REAL copy. Not hard links. +// +// Hard links were the obvious optimisation — an index tree is thousands of +// small files and a snapshot that costs real bytes is a reason not to take one +// — and they are wrong here, which the round-trip test caught immediately: a +// hard-linked snapshot ALIASES the live tree. Anything that rewrites an index +// file in place (a truncating write, a tar extraction over an existing path) +// writes straight through the link and destroys the backup. The one event this +// module exists to survive is precisely "something replaced the tree", so a +// snapshot that shares inodes with the tree is not a snapshot at all. +// +// The bytes are cheap in absolute terms (measured: 912 KB for mcpplibs, 2.2 MB +// for xim-pkgindex; five kept snapshots ≈ 15 MB) and `prune` bounds the total. +bool copy_tree(const std::filesystem::path& from, + const std::filesystem::path& to) +{ + std::error_code ec; + std::filesystem::remove_all(to, ec); + ec.clear(); + std::filesystem::create_directories(to.parent_path(), ec); + + ec.clear(); + std::filesystem::copy(from, to, + std::filesystem::copy_options::recursive + | std::filesystem::copy_options::overwrite_existing + | std::filesystem::copy_options::skip_symlinks, ec); + return !ec; +} + +// Directory mtime, as a sortable integer. Only ever compared against other +// snapshots of the same index, so file_clock's non-Unix epoch is irrelevant. +std::int64_t mtime_of(const std::filesystem::path& p) { + std::error_code ec; + auto t = std::filesystem::last_write_time(p, ec); + if (ec) return 0; + return t.time_since_epoch().count(); +} + +std::string sanitize_component(std::string s) { + for (auto& c : s) { + const bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') + || (c >= '0' && c <= '9') || c == '.' || c == '-' || c == '_'; + if (!ok) c = '_'; + } + if (s.empty()) s = "unknown"; + return s; +} + +} // namespace + +std::filesystem::path snapshots_root(const std::filesystem::path& dataRoot) { + auto parent = dataRoot.parent_path(); + // Degenerate path (relative "data", or a root) — fall back to staying put + // rather than climbing out of the home directory. + if (parent.empty()) return dataRoot / ".index-snapshots"; + return parent / "index-snapshots"; +} + +std::filesystem::path snapshot_dir(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir) { + return snapshots_root(dataRoot) / indexDir.filename(); +} + +std::string snapshot_id(const std::filesystem::path& indexDir) { + std::ifstream is(indexDir / ".xlings-index-version", std::ios::binary); + if (!is) return {}; + std::string v{std::istreambuf_iterator(is), {}}; + while (!v.empty() && (v.back() == '\n' || v.back() == '\r' || v.back() == ' ')) + v.pop_back(); + return sanitize_component(std::move(v)); +} + +std::vector +index_dirs(const std::filesystem::path& dataRoot) { + std::vector out; + std::error_code ec; + if (!std::filesystem::is_directory(dataRoot, ec)) return out; + for (auto& e : std::filesystem::directory_iterator(dataRoot, ec)) { + if (ec) break; + if (!e.is_directory()) continue; + // The store is a sibling of dataRoot (see snapshots_root), so it + // should never appear here. Skipped anyway: this loop decides what + // gets archived, and an archive of the archive is the one mistake + // that would grow without bound. + if (e.path().filename() == ".index-snapshots") continue; + std::error_code pec; + if (!std::filesystem::is_directory(e.path() / "pkgs", pec)) continue; + out.push_back(e.path()); + } + std::ranges::sort(out); + return out; +} + +bool archive(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir) +{ + if (!mcpp::pm::index_usable(indexDir)) return false; + + auto id = snapshot_id(indexDir); + if (id.empty()) id = "unversioned"; + auto dst = snapshot_dir(dataRoot, indexDir) / id; + + // Same id already archived: the tree did not move, so the existing + // snapshot is byte-equivalent and re-copying it buys nothing. + std::error_code ec; + if (std::filesystem::is_directory(dst, ec) && id != "unversioned") + return true; + + return copy_tree(indexDir, dst); +} + +bool restore(const std::filesystem::path& snapshotPath, + const std::filesystem::path& indexDir) +{ + std::error_code ec; + if (!std::filesystem::is_directory(snapshotPath, ec)) return false; + return copy_tree(snapshotPath, indexDir); +} + +std::vector +list_snapshots(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir) +{ + std::vector out; + auto base = snapshot_dir(dataRoot, indexDir); + std::error_code ec; + if (!std::filesystem::is_directory(base, ec)) return out; + for (auto& e : std::filesystem::directory_iterator(base, ec)) { + if (ec) break; + if (e.is_directory()) out.push_back(e.path()); + } + std::ranges::sort(out, [](auto& a, auto& b) { + return mtime_of(a) > mtime_of(b); // newest first + }); + return out; +} + +std::optional +newest_usable(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir) +{ + for (auto& s : list_snapshots(dataRoot, indexDir)) + if (mcpp::pm::index_usable(s)) return s; + return std::nullopt; +} + +void prune(const std::filesystem::path& dataRoot, + const std::filesystem::path& indexDir, + std::size_t keep) +{ + auto snaps = list_snapshots(dataRoot, indexDir); + for (std::size_t i = keep; i < snaps.size(); ++i) { + std::error_code ec; + std::filesystem::remove_all(snaps[i], ec); + } +} + +int guarded_refresh(const std::filesystem::path& dataRoot, + const std::function& refresh, + GuardOutcome& out) +{ + // Snapshot every tree that is currently readable. Doing this BEFORE the + // refresh is the whole point: afterwards the old bytes are gone. + auto before = index_dirs(dataRoot); + std::map usableBefore; + for (auto& dir : before) { + usableBefore[dir] = mcpp::pm::index_usable(dir); + if (usableBefore[dir]) { + archive(dataRoot, dir); + prune(dataRoot, dir); + } + } + + const int rc = refresh(); + + // Judge each tree. `index_dirs` is re-read: a refresh may add a repo. + for (auto& dir : index_dirs(dataRoot)) { + if (mcpp::pm::index_usable(dir)) { + // Improved or unchanged — record the new good state for next time. + archive(dataRoot, dir); + prune(dataRoot, dir); + continue; + } + auto wasUsable = usableBefore.find(dir); + if (wasUsable != usableBefore.end() && wasUsable->second) { + // Got worse. This is the case the invariant exists for. + if (auto snap = newest_usable(dataRoot, dir); + snap && restore(*snap, dir)) { + out.rolledBack.push_back(dir); + continue; + } + } + // Was already unusable (or the rollback failed): try the local history + // anyway — it may hold a readable tree from before this binary ever + // met the raised floor. + if (auto snap = newest_usable(dataRoot, dir); + snap && restore(*snap, dir)) { + out.recovered.push_back(dir); + continue; + } + out.stillUnusable.push_back(dir); + } + return rc; +} + +} // namespace mcpp::pm::index_snapshot diff --git a/src/pm/package_fetcher.cppm b/src/pm/package_fetcher.cppm index 8d5aecf5..d911488f 100644 --- a/src/pm/package_fetcher.cppm +++ b/src/pm/package_fetcher.cppm @@ -378,6 +378,16 @@ format_install_failure_diagnostic(std::string_view target, msg += "\n xlings emitted no structured error; re-run with " "MCPP_VERBOSE=1 to see the raw xlings invocation + output."; } + + // If an index was too new for this binary, THAT is the cause, and the + // not-found above is only its symptom. Without this the last line the user + // reads names a wire address and a missing package — pointing at naming or + // publication — while the real answer (E0006, printed much earlier and + // possibly scrolled away) says "upgrade mcpp". The message that stops the + // build has to carry the reason the build stopped. + if (auto hint = mcpp::pm::unusable_index_hint(); !hint.empty()) { + msg += "\n" + hint; + } return msg; } diff --git a/src/toolchain/fingerprint.cppm b/src/toolchain/fingerprint.cppm index 47f42d15..59e6562c 100644 --- a/src/toolchain/fingerprint.cppm +++ b/src/toolchain/fingerprint.cppm @@ -15,10 +15,15 @@ export module mcpp.toolchain.fingerprint; import std; import mcpp.toolchain.detect; +import mcpp.version; export namespace mcpp::toolchain { -inline constexpr std::string_view MCPP_VERSION = "2026.8.3.4"; +// The version itself lives in the leaf module mcpp.version — see that file for +// why. Kept spelled `mcpp::toolchain::MCPP_VERSION` here because that is what +// every existing reader says, and because the fingerprint genuinely is one of +// its consumers (field 8 of the 10). +inline constexpr std::string_view MCPP_VERSION = mcpp::MCPP_VERSION; struct FingerprintInputs { Toolchain toolchain; diff --git a/src/version.cppm b/src/version.cppm new file mode 100644 index 00000000..61432d9f --- /dev/null +++ b/src/version.cppm @@ -0,0 +1,36 @@ +// mcpp.version — this binary's own version, and nothing else. +// +// WHY IT IS ITS OWN MODULE +// +// The constant used to live in mcpp.toolchain.fingerprint, next to the struct +// that folds it into the BMI cache key. That is a reasonable place for a +// *consumer* of the version and a bad place for the version itself: fingerprint +// imports mcpp.toolchain.detect, which imports mcpp.xlings, so "what version is +// this binary" transitively dragged in the entire toolchain-detection and +// package-manager subsystem. +// +// That is not a stylistic complaint. mcpp.pm.index_contract needs the version +// for one comparison (is this binary new enough for this index?), and pulling +// it from fingerprint made index_contract depend on xlings — which made it +// impossible for xlings to depend on index_contract, which is exactly where the +// index-refresh guard has to live (mcpp.pm.index_snapshot, and see +// mcpp::xlings::update_index). The cycle was the layering telling us the +// constant was in the wrong place. +// +// A leaf module with no imports beyond `std` can be used by anyone. Keep it +// that way: nothing else belongs in this file. +// +// SINGLE SOURCE OF TRUTH. `.github/tools/check_version_pins.sh` reads the +// literal below and cross-checks it against `mcpp.toml`'s `[package].version`; +// tests/e2e/01_help_and_version.sh checks it against `mcpp --version` at +// runtime. Both must be updated together — see docs/09-release.md. + +export module mcpp.version; + +import std; + +export namespace mcpp { + +inline constexpr std::string_view MCPP_VERSION = "2026.8.3.5"; + +} // namespace mcpp diff --git a/src/xlings.cppm b/src/xlings.cppm index d5d50c66..23ec0f40 100644 --- a/src/xlings.cppm +++ b/src/xlings.cppm @@ -15,6 +15,8 @@ export module mcpp.xlings; import std; import mcpp.pm.compat; +import mcpp.pm.index_contract; +import mcpp.pm.index_snapshot; import mcpp.platform; import mcpp.log; @@ -1418,7 +1420,60 @@ bool is_official_package_index_fresh(const Env& env, && official_index_cache_matches_package_file(env, packageName); } +namespace { +// The raw sync. Everything that makes a refresh SAFE lives in the wrapper +// below; this function's only job is to run `xlings update` and report. +int update_index_unguarded(const Env& env, bool quiet); +} // namespace + +// Guarded entry point — the ONE place every refresh in this process passes +// through, which is why the monotonicity guarantee is installed here rather +// than at the seven call sites. +// +// an index-side change must never take mcpp from "works" to "does not work" +// +// A published index can raise its client-version floor (index.toml min_mcpp). +// `xlings update` rewrites the tree in place, so before this guard existed a +// floor bump replaced a readable index with an unreadable one and left no way +// back — the refresh itself was the thing that broke the machine. See +// mcpp.pm.index_snapshot for why the shape is archive/judge/restore rather +// than the stage-and-swap the original design assumed. int update_index(const Env& env, bool quiet) { + namespace snap = mcpp::pm::index_snapshot; + const auto dataRoot = paths::index_data(env); + + snap::GuardOutcome out; + int rc = snap::guarded_refresh(dataRoot, + [&] { return update_index_unguarded(env, quiet); }, out); + + // Report ONLY when the guard had to act. The common path — refresh keeps + // the index readable — must stay silent, or the notice becomes noise that + // users learn to skip past, which is the same as not printing it. + for (auto& dir : out.rolledBack) { + print_status("Kept", std::format( + "previous index for `{}` — the refreshed one requires a newer mcpp", + dir.filename().string())); + if (!quiet) { + std::println(" Your build continues to work with the packages " + "it already describes."); + std::println(" Upgrade to pick up newer packages: xlings update mcpp"); + } + } + for (auto& dir : out.recovered) { + print_status("Restored", std::format( + "index `{}` from a local snapshot this mcpp can read", + dir.filename().string())); + } + for (auto& dir : out.stillUnusable) { + mcpp::log::verbose("index", std::format( + "index `{}` requires a newer mcpp and no local snapshot is usable", + dir.filename().string())); + } + return rc; +} + +namespace { +int update_index_unguarded(const Env& env, bool quiet) { // Offline is absolute: no caller gets to reach the network by going around // the decision layer. Reported as success so a build that can still resolve // everything locally proceeds — the caller that genuinely needed the data @@ -1472,6 +1527,7 @@ int update_index(const Env& env, bool quiet) { "index update failed after {} attempts (rc {})", kMaxAttempts, rc)); return rc; } +} // namespace void ensure_index_fresh(const Env& env, std::int64_t ttlSeconds, bool quiet) { if (is_index_fresh(env, ttlSeconds)) return; diff --git a/tests/e2e/185_index_floor_degrades.sh b/tests/e2e/185_index_floor_degrades.sh new file mode 100755 index 00000000..8ed5ba6a --- /dev/null +++ b/tests/e2e/185_index_floor_degrades.sh @@ -0,0 +1,192 @@ +#!/usr/bin/env bash +# requires: gcc fresh-sandbox +# 185_index_floor_degrades.sh — an index cannot decide whether mcpp works. +# +# An index tree may declare a client-version floor (`index.toml` min_mcpp). Two +# behaviours around that floor had NO end-to-end coverage at all — only the pure +# predicate `floor_violation()` was unit-tested — which is why both were wrong +# for as long as the feature has existed: +# +# 1. A floor violation made every descriptor read return "nothing", which is +# the same answer as "this package is not in this index". The build then +# died on `E_NOT_FOUND ... wire address tried: ...` — naming neither the +# version nor the floor. The cause was printed much earlier, and the line +# that actually stopped the build pointed at addressing instead. +# +# 2. That same indistinguishable miss fed the refresh policy, which read it as +# "the local index is missing something, go fetch" — so an unusable index +# drove repeated refreshes of itself. +# +# Both assertions below are about what the user READS, because that is what was +# broken. `mcpp explain E0006` and the floor predicate were fine throughout. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +# ── two path indexes: one readable, one that demands a newer mcpp ──────────── +make_index() { # $1 = dir, $2 = min_mcpp ("" = no contract), $3 = pkg name + local dir="$1" floor="$2" pkg="$3" + mkdir -p "$dir/pkgs/${pkg:0:1}" + if [[ -n "$floor" ]]; then + cat > "$dir/index.toml" < "$dir/pkgs/${pkg:0:1}/$pkg.lua" < src/main.cpp <<'EOF' +import std; +int main() { std::println("ok"); return 0; } +EOF + +# ── 1. an unusable index must say so — not "package not found" ─────────────── +cat > mcpp.toml < build.log 2>&1 +rc=$? +set -e +[[ "$rc" != "0" ]] || { echo "FAIL: build unexpectedly succeeded"; cat build.log; exit 1; } + +grep -q "E0006" build.log || { + echo "FAIL: the failure never mentions E0006 — the user cannot tell that the" + echo " cause is an index that needs a newer mcpp." + cat build.log + exit 1 +} + +# THE assertion that was missing, and it has to be precise. +# +# "E0006 appears somewhere in the output" is NOT enough — it always did, and a +# test asserting only that passes on the broken build too (verified). What was +# broken is that the message which STOPS the build said only +# +# error: dependency 'toonew.newlib': not found in local index at '...' +# +# i.e. it blamed the package, pointing the user at publication or naming, while +# the actual answer (upgrade mcpp) had scrolled past. So: assert on the LAST +# error, and assert on the specific text that ties the two together. +last_error="$(grep -E '^error:' build.log | tail -1)" +echo "$last_error" | grep -q "not found" && { + # A bare not-found as the final word is the exact regression. It is only + # acceptable if the cause is attached to it. + grep -A 4 "$(echo "$last_error" | head -c 40)" build.log \ + | grep -qE "cannot read|E0006" || { + echo "FAIL: the build stopped on a bare 'not found' with no mention of the" + echo " index that could not be read. That blames the package for a" + echo " problem whose fix is 'upgrade mcpp'." + echo "--- output ---" + cat build.log + exit 1 + } +} +grep -q "this mcpp cannot read" build.log || { + echo "FAIL: nothing in the output names the unreadable index as the cause." + cat build.log + exit 1 +} +echo "PASS: an unusable index reports E0006 as the cause" + +# ── 2. an unusable index must not take a healthy one down with it ──────────── +# INV-2: the blast radius of a floor bump is the index that declared it. +cat > mcpp.toml < iso.log 2>&1 +iso_rc=$? +set -e +[[ "$iso_rc" == "0" ]] || { + echo "FAIL: a project that depends on NOTHING from the too-new index still" + echo " failed to build — an unusable index is not supposed to be" + echo " contagious." + cat iso.log + exit 1 +} +echo "PASS: an unusable index does not break a build that does not need it" + +# ── 3. the escape hatch still works ───────────────────────────────────────── +# It exists for debugging; if it ever stops working the only lever a blocked +# user has is gone. +cat > mcpp.toml < ignore.log 2>&1 || { + echo "FAIL: MCPP_INDEX_FLOOR=ignore no longer bypasses the floor" + cat ignore.log + exit 1 +} +grep -q "E0006" ignore.log && { + echo "FAIL: MCPP_INDEX_FLOOR=ignore still reported E0006" + cat ignore.log + exit 1 +} +echo "PASS: MCPP_INDEX_FLOOR=ignore bypasses the floor" + +echo "OK" diff --git a/tests/unit/test_index_snapshot.cpp b/tests/unit/test_index_snapshot.cpp new file mode 100644 index 00000000..11b3210e --- /dev/null +++ b/tests/unit/test_index_snapshot.cpp @@ -0,0 +1,227 @@ +// The invariant this file exists for: +// +// An index-side change must never take mcpp from "works" to "does not work". +// +// A published index can raise its client-version floor (index.toml min_mcpp). +// `xlings update` rewrites the tree in place, so before the guard existed a +// floor bump replaced a readable index with an unreadable one and left no way +// back — the refresh was the thing that broke the machine. +// +// The 2026-07-08 index design specified this behaviour ("staged refresh keeps +// the last compatible snapshot") and it was never implemented; the floor was +// checked in exactly one place, the descriptor reader. There was no test that +// could have noticed, because the only coverage was of the pure predicate. +// These are that missing test. + +#include + +import std; +import mcpp.pm.index_snapshot; +import mcpp.pm.index_contract; +import mcpp.version; + +using namespace mcpp::pm::index_snapshot; + +namespace { + +struct Tmp { + std::filesystem::path path; + Tmp() { + path = std::filesystem::temp_directory_path() + / std::format("mcpp_idx_snap_{}", std::random_device{}()); + std::filesystem::create_directories(path); + } + ~Tmp() { std::error_code ec; std::filesystem::remove_all(path, ec); } +}; + +void writeFile(const std::filesystem::path& p, std::string_view body) { + std::filesystem::create_directories(p.parent_path()); + std::ofstream(p) << body; +} + +std::string readFile(const std::filesystem::path& p) { + std::ifstream is(p); + return std::string((std::istreambuf_iterator(is)), {}); +} + +// A minimal index tree: pkgs/ makes it an index, index.toml carries the floor, +// .xlings-index-version is the snapshot identity. +void makeIndex(const std::filesystem::path& dir, + std::string_view minMcpp, + std::string_view version, + std::string_view marker) { + writeFile(dir / "pkgs" / "z" / "zlib.lua", std::format("-- {}\n", marker)); + writeFile(dir / ".xlings-index-version", version); + if (minMcpp.empty()) { + std::error_code ec; + std::filesystem::remove(dir / "index.toml", ec); + } else { + writeFile(dir / "index.toml", + std::format("[index]\nspec = \"1\"\nmin_mcpp = \"{}\"\n", minMcpp)); + } +} + +constexpr std::string_view kTooNew = "9999.9.9.9"; // no mcpp satisfies this + +} // namespace + +TEST(IndexSnapshot, DetectsIndexTreesAndIgnoresTheSnapshotStore) { + Tmp t; + auto data = t.path / "data"; + makeIndex(data / "mcpplibs", "", "v1", "a"); + makeIndex(data / "xim-pkgindex", "", "v1", "b"); + std::filesystem::create_directories(data / "not-an-index"); // no pkgs/ + std::filesystem::create_directories(data / ".index-snapshots" / "x" / "pkgs"); + + auto dirs = index_dirs(data); + ASSERT_EQ(dirs.size(), 2u); + EXPECT_EQ(dirs[0].filename(), "mcpplibs"); + EXPECT_EQ(dirs[1].filename(), "xim-pkgindex"); +} + +TEST(IndexSnapshot, ArchiveRefusesAnUnusableTree) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + makeIndex(idx, kTooNew, "v1", "unusable"); + + // A snapshot exists to be restored; archiving one nobody can read would + // only give the recovery path a useless candidate. + EXPECT_FALSE(archive(data, idx)); + EXPECT_TRUE(list_snapshots(data, idx).empty()); +} + +TEST(IndexSnapshot, ArchiveAndRestoreRoundTrip) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + makeIndex(idx, "0.0.85", "v1", "original"); + + ASSERT_TRUE(archive(data, idx)); + auto snaps = list_snapshots(data, idx); + ASSERT_EQ(snaps.size(), 1u); + + makeIndex(idx, "0.0.85", "v2", "replaced"); + EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("replaced"), + std::string::npos); + + ASSERT_TRUE(restore(snaps[0], idx)); + EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("original"), + std::string::npos); +} + +// THE load-bearing test. A refresh that raises the floor beyond this binary +// must leave the machine working. +TEST(IndexSnapshot, RefreshThatBreaksTheIndexIsRolledBack) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + makeIndex(idx, "0.0.85", "good", "usable-tree"); + + GuardOutcome out; + int rc = guarded_refresh(data, [&] { + // This is what `xlings update` does: rewrite the tree in place, with a + // floor this binary cannot satisfy. + makeIndex(idx, kTooNew, "too-new", "unusable-tree"); + return 0; + }, out); + + EXPECT_EQ(rc, 0); + ASSERT_EQ(out.rolledBack.size(), 1u) << "the refresh made the index unusable " + "and the guard did not roll it back"; + EXPECT_TRUE(out.stillUnusable.empty()); + + // The machine still works: the tree on disk is readable and is the old one. + EXPECT_TRUE(mcpp::pm::index_usable(idx)); + EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("usable-tree"), + std::string::npos); +} + +// The mirror case: a refresh that keeps the index readable must be left alone, +// and must silently become the new known-good snapshot. +TEST(IndexSnapshot, NormalRefreshIsUntouchedAndBecomesTheNewSnapshot) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + makeIndex(idx, "0.0.85", "v1", "first"); + + GuardOutcome out; + guarded_refresh(data, [&] { + makeIndex(idx, "0.0.85", "v2", "second"); + return 0; + }, out); + + EXPECT_FALSE(out.degraded()) << "a healthy refresh must not report anything"; + EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("second"), + std::string::npos); + // Both the pre- and post-refresh trees are now archived, so the next + // refresh has somewhere to fall back to. + EXPECT_EQ(list_snapshots(data, idx).size(), 2u); +} + +// Already-unusable on entry: there is nothing to "roll back" to, but a local +// snapshot from before the floor moved is still the right answer. +TEST(IndexSnapshot, RecoversFromLocalHistoryWhenAlreadyUnusable) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + + // A good tree was seen at some point in the past... + makeIndex(idx, "0.0.85", "old-good", "known-good"); + ASSERT_TRUE(archive(data, idx)); + // ...and the current tree is already too new (e.g. mcpp was downgraded, or + // the guard was introduced after the damage was done). + makeIndex(idx, kTooNew, "too-new", "unusable"); + + GuardOutcome out; + guarded_refresh(data, [&] { return 0; }, out); // refresh changes nothing + + ASSERT_EQ(out.recovered.size(), 1u); + EXPECT_TRUE(out.rolledBack.empty()); + EXPECT_TRUE(mcpp::pm::index_usable(idx)); + EXPECT_NE(readFile(idx / "pkgs" / "z" / "zlib.lua").find("known-good"), + std::string::npos); +} + +// No local history and a too-new tree is the one case nothing can rescue. It +// must be reported honestly rather than silently left looking fine. +TEST(IndexSnapshot, ReportsStillUnusableWhenNoSnapshotCanHelp) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + makeIndex(idx, kTooNew, "too-new", "unusable"); + + GuardOutcome out; + guarded_refresh(data, [&] { return 0; }, out); + + ASSERT_EQ(out.stillUnusable.size(), 1u); + EXPECT_TRUE(out.rolledBack.empty()); + EXPECT_TRUE(out.recovered.empty()); +} + +TEST(IndexSnapshot, PruneKeepsTheNewest) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "mcpplibs"; + for (int i = 0; i < 4; ++i) { + makeIndex(idx, "0.0.85", std::format("v{}", i), std::format("gen{}", i)); + ASSERT_TRUE(archive(data, idx)); + } + ASSERT_EQ(list_snapshots(data, idx).size(), 4u); + prune(data, idx, /*keep=*/2); + EXPECT_EQ(list_snapshots(data, idx).size(), 2u); +} + +// A tree with no index.toml declares no contract, so it is usable by anyone. +// Third-party indexes rely on this and must never be rolled back. +TEST(IndexSnapshot, TreeWithoutAContractIsAlwaysUsable) { + Tmp t; + auto data = t.path / "data"; + auto idx = data / "third-party"; + makeIndex(idx, "", "v1", "no-contract"); + EXPECT_TRUE(mcpp::pm::index_usable(idx)); + + GuardOutcome out; + guarded_refresh(data, [&] { return 0; }, out); + EXPECT_FALSE(out.degraded()); +}