diff --git a/.agents/docs/2026-09-26-issue-23-fix-plan.md b/.agents/docs/2026-09-26-issue-23-fix-plan.md new file mode 100644 index 0000000..9f5eee7 --- /dev/null +++ b/.agents/docs/2026-09-26-issue-23-fix-plan.md @@ -0,0 +1,485 @@ +# mcppls 修复与优化方案：issue #23（Windows/LTO）与 hello 项目实测（编辑中的卡死与重启） + +状态：方案第 3 版（D1–D5 已定；新增 F18 导出与脱敏；D4 的防频繁策略；自我 review）；§7 为 0.0.5 的实施计划（一个 PR 全部实现） · 2026-09-26 · 基于 `main` 04b186a（0.0.4） + +依据： + +- 分析报告 `.agents/docs/2026-09-26-issue-23-lto-module-scan.md`（含 GalTranslPP 的 Windows CI 实测）； +- hello 项目实测：用户 VS Code 会话日志 `~/.cache/mcppls/logs/server-20260925-215311.673-2d10.log`，以及在副本上逐字输入、 + 模拟自动保存、按线程采样 CPU 的复现； +- 上游问题统一登记在 issue #24（本文引用其中的 `UP-`）。**F 编号保持稳定**：#24 的评论引用了 F1、F3、F4、F5、F7、F11、 + F12、F16。 + +## 0. 摘要 + +### 0.1 两条问题链 + +**A. issue #23（Windows/MSVC + LTO）**：给 clangd 的命令没有 `-c` 却带 `-flto` → 模块扫描全部失败、不建 BMI（UP-11）→ Windows +clangd 在无 BMI 的模块单元上崩溃（UP-12）→ mcppls 隔离错了文件、临时模型阶段的崩溃先耗掉额度 → clangd 被判不可用。另有独立的 +上游崩溃（UP-13）和 BOM 识别缺陷（M2）。 + +**B. hello 项目（Linux，`files.autoSave: afterDelay`）**：自动保存把写了一半的 `import hello.` 写到磁盘 → WA-CLANGD-001 只改写 +LSP 文本、clangd 却从磁盘解析前置模块（UP-14）→ 该文件的 worker 进入 UP-01 死循环（实测单核满载，改回缓冲区也停不下）→ +该文件的全部 clangd 请求（含关键字补全、hover）排队超时；同时写到一半的模块名让计划来回变（stand-in 增删、数据库 13↔14 条）→ +每变一次重启一次（另一次来自切换工具链）→ 10 分钟 3 次后被上限锁死，之后空转的 clangd 无法再被恢复。另有诊断吵闹与位置偏移（UP-15）、未保存的新 import 不构建（UP-14）等体验问题。 + +### 0.2 条目总表（按主题） + +| 主题 | # | 修什么 | 来源 | 优先级 | 估算 | +|---|---|---|---|---|---| +| **正确性** | F1 | 给 clangd 的每条命令都带 `-c` | #23 根因（UP-11/12） | P0 | 1 天 | +| | F2 | 扫描器跳过 UTF-8 BOM | #23 附带（M2） | P0 | 0.5 天 | +| | F16 | WA-CLANGD-001 的磁盘漏洞：磁盘上的半截 import 让 clangd 死循环 | hello：提示消失、请求超时的根因（UP-01×UP-14） | P0 | 1–1.5 天 | +| | F5 | 兜底扫描排除包管理器目录 | #23 CI（M5） | P2 | 0.5 天 | +| **引擎恢复** | F3 | 按 clangd 崩溃上下文精确隔离，记录退出码 | #23（M3；UP-12/13 止损） | P1 | 1 天 | +| | F13 | 编辑中的计划变化防抖；不为仍在输入的 import 建 stand-in；重启合并 | hello（3 次重启） | P1 | 1.5–2 天 | +| | F14 | 重启分预算、达到上限后退避，不再锁死 10 分钟 | hello（被锁死） | P1 | 1 天 | +| | F4 | producer 回答前 clangd 等待；模型切换时清零崩溃记账 | #23（M4） | P1 | 2–3 天 | +| **编辑体验** | F11 | 未保存缓冲区里新加的 import：not found 改写为"保存后可用" | UP-14 | P1 | 1 天 | +| | F12 | 漏 `;` 的错误移回本行；可选的诊断防抖 | UP-15 | P2 | 0.5 天 | +| | F9 | `import ` 空格后自动弹出模块列表（仅 import 行） | hello | P2 | 0.5–1 天 | +| | F15 | 模块语法关键字补全由 mcppls 提供 | hello | P2 | 0.5–1 天 | +| | F8 | 声明前的 `export` 与 `export module` 高亮一致 | 用户提问 | P2 | 0.5 天 | +| **可观察性** | F17 | 常开日志环形缓冲、事故快照、计划差异、workaround 前提守卫、状态栏给原因 | 本次排查成本 | P1 | 2–3 天 | +| | F18 | 一键导出问题包（环境、日志、事故、崩溃、引擎数据库），统一脱敏（用户名、主目录、主机名、密钥），校验不泄漏才写出 | 用户要求 | P1 | 2.5–3 天 | +| | F6 | 扫描失败汇总成 issue；崩溃/扫描行不被限流 | #23 | P1 | 1–1.5 天 | +| **其他** | F7 | macOS 部署目标降到 11.0 | UP-M1（mcpp#685 已修） | P2 | 0.5 天 | +| | F10 | mcpp#699 落地后显示失败成员的诊断 | UP-M2/M3 | 后续 | 0.5 天 | +| **验证** | V | Windows（GalTranslPP CI）+ Linux（hello/自动保存夹具）验收 | 全部 | — | 0.5–1 天 | + +合计（不含 F10）约 **18–23 人日**。 + +### 0.3 版本划分建议 + +| 版本 | 目标 | 内容 | 估算 | +|---|---|---|---| +| **0.0.5 "稳"** | issue #23 关闭；hello 场景不再卡死、不再被锁死；出事能看清原因并一键导出 | F1、F2、F16、F3、F13、F14、F6、F7、F17 的 1–2 项（环形缓冲 + 事故快照）、F18 | 约 12–14 天（另加验证 V） | +| **0.0.6 "顺"** | 编辑体验 | F4、F11、F9 + F15、F12、F8、F5、F17 的 3–6 项 | 约 6–9 天 | +| 之后 | 跟随上游 | F10（mcpp#699）；UP-01 的上游处理与（必要时）自建 clangd（F16.3，D2：推迟） | — | + +### 0.4 依赖关系 + +- **F3 → F14 → F17 → F18**：F14 的分预算需要 F3 给出的崩溃归因；F17 的事故快照复用 F3 的崩溃上下文与 F14 的原因分类；F18 把 + F17 的事故目录、日志与报告打包并脱敏（没有 F17 时 F18 仍可导出日志、报告与环境，只是少了事故现场）。 +- **F13 ↔ F16**：两者都在"来自正在编辑的文档的变化"上做判断（F13 防抖计划变化，F16 按磁盘内容主动隔离），共用"文档最近编辑时间 + + 磁盘内容检查"的状态，一起实现。 +- **F4 ↔ F14**：都改崩溃/重启记账（F4 的"模型切换清零"与 F14 的"分预算"），一起设计。 +- **F9 ↔ F15**：都改补全路由（空格触发只交给 mcppls 引擎；关键字补全由 mcppls 提供并与 clangd 结果合并），一起实现。 +- **与 mcpp 无依赖**：F7 只需 mcpp ≥ 2026.9.24.1（已发布）；F10 等 mcpp#699。 + +## 1. 正确性 + +### F1. 给 clangd 的每条命令都带 `-c`（P0） + +**问题**：`src/normalize/gnu.cpp:20,49` 的 `ALWAYS_STRIP_EXACT` 删掉 `-c` 后无处补回；`translate_msvc`（`src/normalize/msvc.cpp`） +的输出同样没有；`semantic.cpp:86` 加的 `-c` 也会被 `translate_gnu` 删掉。最终阶段是 Link，windows-msvc + LTO 下驱动报错，clangd +的模块扫描失败（UP-11），不建 BMI，进而在 Windows 上触发 UP-12。 + +**改法**： + +1. `translate_gnu`、`translate_msvc` 仍删除构建命令自带的 `-c`、`--precompile`，但在输出末尾**补一个 `-c`**（已有则不重复）； + `kit_arguments` 同样补。 +2. 派生条目自动继承：项目单元（`plan.cpp:473`）、游离文件（`:296`）、prime（`:499-501`）、std（`:523-547`）、stand-in + （`:577-581`）。 +3. **不删 `-flto` 等链接参数**：`-c` 一次关掉所有只在链接阶段做的驱动检查，不需维护链接参数清单，也不改变项目的 LTO（#23 报告者 + 明确要求）。 + +**为什么安全**：clangd 建 AST/BMI 时 `createInvocation` 插入 `-fsyntax-only`，与 `-c` 并存时取更早的阶段，行为不变；CMake、mcpp +自己写的构建数据库本来就带 `-c`。 + +**测试**：`tests/test_normalize.cpp:70,114` 改为"恰好一个 `-c`"；新增 windows-msvc + `-flto` 的 GNU、clang-cl 与 kit 三种输入； +plan 级测试覆盖各派生条目；conformance：Windows CI 的 `mcpp-llvm-msvc-lto`，以及 Linux 可跑的 compdb 夹具（本机已证明该驱动检查 +在 Linux 上同样触发）。 + +### F2. 扫描器跳过 UTF-8 BOM（P0） + +**问题**：`src/project/scan.cpp` 从偏移 0 开始词法分析，BOM + `export module X;` 开头的文件认不出模块声明。所有文本扫描都走 +`project::scan_source`（计划 `plan.cpp:288`、自带模块引擎 `engine/native/index.cpp:71`、AI review、查询视图）， +`engine/native/exports.cpp` 另有一份镜像的声明解析。本机复现：可信模型下 mcppls 引擎误报 `module 'm' not found`；兜底模型下还会 +漏掉 `-x c++-module`。 + +**改法**：`Lexer` 构造时若以 `EF BB BF` 开头则从偏移 3 开始，偏移仍按原文计算（语义 token、诊断位置不变）；`exports.cpp` 同样处理。 + +**测试**：BOM + `export module`/`module;`/`module m;`/`import` 四种；语法 token 列号与无 BOM 时一致；conformance `mcpp-bom`。 + +### F16. WA-CLANGD-001 的磁盘漏洞（P0） + +**实测（hello 副本，0.0.4，`Worker:main.cpp` 在 20 秒内的 CPU tick）**：`import hello.` 只在缓冲区 → 2；自动保存写盘后 → 约 2000； +缓冲区改回 `import hello.greet;` 而磁盘仍是 `import hello.` → 约 2000。WA-CLANGD-001 只改写 LSP 文本，而 clangd 从磁盘解析前置模块 +（UP-14），于是进入 UP-01 的死循环，只有重启能解开。这是 hello 场景"只有 `import hello.` 有提示""停久了请求超时"的根因； +会话里计入的 3 次重启另有来源（两次计划抖动见 F13，一次切换工具链），达到上限后，因空转而请求的重启又被拦下（22:02:25）， +clangd 从此停在空转里（F14）。 + +**改法**： + +1. **磁盘侧检测 + 主动隔离**：`didSave` 与文件监视时，对数据库内的文件用 WA-CLANGD-001 的同一判定检查磁盘内容；命中时在 clangd + 重建该文件的 preamble 之前把它交给 mcppls 自己的引擎（隔离），磁盘内容变回合法后交还。不等空转、不走重启、不耗额度。 +2. **兜底**：若已在空转（`Worker:` 持续满载且磁盘内容命中），立即重启 clangd 并隔离该文件到磁盘恢复为止，原因写明，不计入 + 计划重启额度。 +3. **根治（D2 已定：推迟）**：本期只做 1、2 的规避；上游处理（bisect `6dcfc17b1b`、提交、申请 23.1.x backport）推迟，已在 #24 + 的 UP-01 标记为 deferred；若迟迟没有 backport，再自建 23.1.x + cherry-pick 作为后备。UP-14 若在上游修复（扫描改用含缓冲区 + 的 `TFS`），磁盘路径也随之消失。 + +**测试**：conformance——逐字输入、在 `import hello.` 处写盘 + `didSave`；断言无满载 worker、无重启、其他文件照常应答，改成完整名字 +并保存后该文件回到 clangd。 + +### F5. 兜底扫描排除包管理器目录（P2） + +`src/project/infer.cpp:21` 的 `SKIPPED_DIRECTORIES` 加入 `vcpkg_installed`、`vcpkg`、`.conan`/`.conan2`、`.xmake`、`.cache`、 +`.git`，并跳过含 `vcpkg.json` 的子项目根。#23 的 CI 中 producer 失败时 349 条里有 166 条来自 `vcpkg_installed/`，产生伪 +`ambiguous-module`。测试：`test_project.cpp` 增加对应用例。 + +## 2. 引擎恢复 + +### F3. 按 clangd 的崩溃上下文精确隔离，记录退出码（P1） + +**问题**：`handle_closed_`（`src/engine/clangd.cpp:1528-1575`）以"未答请求 + 退出前 10 秒碰过的文件"为嫌疑；Windows 实测中 clangd +报 `NormalJsonTranslator.Core.cpp`，mcppls 却隔离了 `GPPDefines.ixx`，前两次崩溃无嫌疑。 + +**改法**：在 stderr 转发处（`clangd.cpp:1066-1100`，与 `parse_module_failure` 并列）解析 +`Signalled during AST worker action: <动作>` / `Filename:` / `Exception Code:`（或 POSIX 信号）；本代进程有崩溃上下文时只隔离 +那个文件，没有时才回退启发式；`engine-exit` 事件与报告增加 `exitCode`（`Connection::exit_code()` 已有）、`crashFile`、 +`crashAction`；issue 文案写明"clangd crashed while building `<文件>`"。 + +**测试**：解析器单测（Windows/POSIX、多行、夹杂日志）；conformance 用替身 clangd 打印崩溃上下文后退出。 + +### F13. 编辑中的计划变化不让 clangd 抖（P1） + +**实测（用户会话日志）**：21:54 输入 `import hello…`，自动保存把 `import h` 写盘，clangd 报 `Failed to build module h`，`h` 是合法名 +字，生成 stand-in，4 秒后撤掉 → 重启 1；22:01 新建 `src/test/test.cppm`，先与 `src/test.cppm` 重复声明 `hello.test`、后为 +`hello.test.`，mcpp emit 两次失败，数据库 13↔14 条 → 重启 3。 + +**改法**： + +1. 来自正在编辑的文档的 import/模块声明引起的计划变化（stand-in 增删、补位条目、provider 变化），在该文档停止修改约 2 秒后才应用； + 自动保存不算"稳定"。 +2. 不为最近编辑过的文档里的 import 生成 stand-in（H3 从"非法名字"扩大到"仍在输入"）。 +3. 计划驱动的重启合并为最多每 N 秒一次、总用最新计划；clangd 自己重读数据库（约 5 秒）能解决的不重启。 +4. producer 在编辑中途失败时保持上一份计划，打开文件的补位条目也不来回增删。 + +**测试**：conformance——逐字输入 `import hello.test.a;`，中途模拟自动保存；断言无 stand-in、无 `engine-restart`。 + +### F14. 重启上限不再把 clangd 锁死（P1） + +**实测**：22:01:24 起 `not restarting clangd again: already 3 restarts in the last 10 minutes`，22:02–22:04 clangd 对全部请求 +不应答约 90 次，`main.cpp` 被隔离。 + +**改法**：重启按原因分预算（计划驱动 / 崩溃与卡住）；模型的工具链或 profile 变化（如用户切换工具链）引起的重启不计入任何预算；达到上限后按 1、2、4、8 分钟退避，到点即可恢复卡住的 +clangd；受限期间状态栏给出原因与"Restart clangd"动作。 + +**测试**：预算与退避单测；conformance：连续触发计划重启后 clangd 仍能在退避后恢复。 + +### F4. producer 回答前 clangd 等待；模型切换时清零崩溃记账（P1） + +**问题**：`start_inferred_load()`（`workspace.cpp` ~770）在 producer 回答前用语义套件的临时模型（Windows 上是 `x86_64-w64-mingw32`， +无项目 include 目录）喂给 clangd。#23 CI：E1 的 5 次退出中 3 次发生在前 18 秒；E4（关 LTO）可信阶段本身正常，仍因这一阶段的崩溃 +`engine-restart-capped`。 + +**改法**：识别出构建系统（level 3）且 producer 正在运行时，clangd 等待（打开的文件进"等待数据库"队列，由 mcppls 引擎回答），上限为 +producer 超时（默认 60 秒）；超时或失败后才用临时/兜底计划。模型来源变化（`inferred` → `producer`）时清零崩溃计数、释放因临时模型进入的 +隔离、重置重启门。没有构建系统的项目保持现状。 + +**取舍**：等待期间只有模块级功能（GalTranslPP 温启动 producer 约 24 秒）；换来可信模型到达时 clangd 不带"前科"。 + +**测试**：mockmcpp 延迟回答（沿用 `mcpp-emit-hang`），断言回答前无 `engine-start`；"模型切换清零"单测。 + +## 3. 编辑体验 + +### F11. 未保存缓冲区里新加的 import（UP-14，P1） + +**实测**：未保存时补全出 `import a.b;`，`Module 'a.b' not found` 45 秒以上不消失；保存后 1–8 秒消失（hello 与 cmp 两个项目）。 + +**改法**：clangd 对打开且有未保存修改的文档报 `Module 'X' not found`，而 X 在计划里有提供者、这行 import 只在缓冲区不在磁盘时，替换为 +信息级诊断"module 'X' is in the project; clangd loads it once the file is saved"；X 不在计划里时保留原错误。其间 X 相关的模块级请求 +由 mcppls 引擎回答。上游修复建议已写入 UP-14。 + +**测试**：增量输入不保存，断言无该错误、出现信息；保存后信息消失。 + +### F12. 输入 import 时的诊断位置与闪烁（UP-15，P2） + +**实测**：输入 `import hello` 时 `Import directive must end with a ';'` 标在两行之后的 `auto main()` 上；逐字输入时每约 150 ms 一条 +（`Unknown type name 'i'`、`Module 'h' not found`……）。 + +**改法**：该诊断落在第 N 行、且其上方最近的非空行是未以 `;` 结束的 `import`/`export import`/`module` 指令时，把范围移到那一行末尾 +（登记为新的 `WA-CLANGD-`）；可选：对正在编辑的文档把引擎诊断推迟到最后一次修改后约 400 ms，保存或停顿后照常发布。 + +### F9 + F15. 补全：空格触发与模块语法关键字（P2） + +**实测（LSP 层）**：模块名补全（`import hello.`、`import hello.test.`、`import a`、`import :`）瞬时可用；关键字补全来自 clangd +（`i` 起即含 `import name;`，`e` 起含 `export`，`isIncomplete` 正确透传）；clangd 该文件卡住时关键字补全随之消失；`triggerCharacters` +是 clangd 的 `. < > : " / *`，没有空格；clangd 不提供 `export module`、`export import` 组合。 + +**改法**： + +- F9：`triggerCharacters` 加空格；空格触发的补全只在行前缀为 `[export] import ` 时交给 mcppls 引擎，其余立即回空、**不转发 clangd**。 + **避免频繁处理（D4）**，由外到内四层： + 1. **编辑器侧拦截（VS Code）**：插件的 language client middleware 在发请求前检查 `triggerCharacter === ' '` 与当前行文本 + （`document.lineAt`），不匹配就直接返回空，**普通代码里的空格不产生任何 IPC**。 + 2. **服务端门槛**：路由收到空格触发的请求时，只取该文档当前行到光标的文本（文档已在内存）做一次前缀判断 + （`^\s*(export\s+)?import\s$`：关键字后恰好一个空格、之后没有别的字符），不匹配立即回 `{isIncomplete:false, items:[]}`； + 不转发 clangd、不查索引、不写 info 日志（只计数）。 + 3. **结果缓存**：模块名候选按计划代次缓存，匹配时直接复制，不随每次按键重建。 + 4. **按客户端启用**：只对已知能优雅处理空结果的客户端（`clientInfo.name` 为 VS Code / Cursor）声明空格触发；Neovim、Zed、 + CLion 默认不声明，可用初始化选项 `mcppls.completion.triggerOnSpace` 打开。VS Code 另有 F15 的补充：接受 `import` 关键字后 + 随即弹出模块列表（补全项带 `editor.action.triggerSuggest`），不依赖空格触发。 + 报告（F17）记录空格触发的请求数、通过门槛的次数与耗时，用来确认开销可以忽略。 +- F15：mcppls 引擎在行首上下文提供 `import`、`export import`、`module;`（仅文件开头）、`export module`、`module`（实现单元）、 + `module :private;`（仅接口单元），与 clangd 结果合并去重；接受 import 类关键字后接着弹出模块名列表。 + +**测试**：`completion-contains` 覆盖各前缀、空格触发的两种行、clangd 不可用时仍有关键字。 + +### F8. 声明前的 `export` 与 `export module` 高亮一致（P2） + +**实测**：`module;`、`export module`、`export import`、`import`、`module :private;` 的关键字都有 `keyword` 语义 token；声明前的 +`export`（`export namespace`、`export {`、`export int f()`）服务端不发，VS Code 内置 C++ 语法也不给作用域，显示为普通文本。 + +**改法**：`scan_syntax_tokens` 对模块单元里声明开头的 `export` 发 `keyword`；注入语法增加对应规则。 + +## 4. 可观察性 + +### F17. 一次复现就能看清原因（P1；0.0.5 先做 1–2） + +这次排查的代价：用户的 info 级日志（487 行）只有"重启了""没应答"，没有哪个文件、哪个线程、clangd 看到的是缓冲区还是磁盘、计划为什么 +变；每个关键事实都要另起环境复现才得到。 + +1. **常开的 clangd 日志环形缓冲**：clangd 以 `--log=info` 运行（D3 已定），最近 N 行（例如 4000 行 / 4 MB）只留在内存；出事时整段 + 落盘并记入报告；需要更细时由事故触发临时切到 verbose 再采一段。 +2. **事故快照**（崩溃/卡住/隔离/重启/达到上限各一份，`/incidents/<时间>-<类型>/`）：原因链；涉及文件的缓冲区版本与磁盘内容差异 + （只记涉及的行）；引擎数据库中的条目；`fileStatus` 时间线；按线程 CPU（Linux `/proc//task`，Windows `GetThreadTimes`）； + 退出码与崩溃上下文（F3）；最近的计划差异。 +3. **计划差异日志**：列出增删改的条目及原因；`units are compiled with other arguments` 写出哪个单元、哪个参数。 +4. **workaround 前提守卫**：每个 `WA-CLANGD-` 声明前提（WA-001：clangd 只看到 LSP 文本），运行时检测到前提被破坏即记事故（本次即 + "磁盘内容命中判定"）。 +5. **状态栏写原因、给动作**："clangd 卡在 `main.cpp`：磁盘上的 `import hello.` 未写完（UP-01）"，附"查看事故 / 导出问题包（F18）/ 重启 clangd"。 +6. **conformance 覆盖自动保存路径**（本次漏洞就是这样漏过的）。 +7. **事故目录保留**：最近 20 次或 7 天，超出即删，避免无限增长。 + +### F18. 一键导出问题包，统一脱敏（P1） + +**为什么**：现在出了问题，用户能给的只有 `Collect Report`（一份 JSON，最近 300 行日志，提示"含本机路径，请自行编辑"）。本次排查 +真正用到的信息——多次会话的日志、clangd 自己的输出、线程 CPU、缓冲区与磁盘差异、工具链与编辑器环境、崩溃退出码——散在各处， +要靠开发者另起环境复现。用户也不该手工删除用户名和路径。 + +**入口**（同一个实现，服务端 C++，所有编辑器共用）： + +- VS Code：命令 **"C++ Modules: Export Diagnostic Bundle"**；F17 的事故通知、状态栏受限提示、`Collect Report` 的结果页都提供它。 +- 其他编辑器：`workspace/executeCommand` `mcppls.exportBundle`。 +- 命令行：`mcppls report --bundle [--root ] [--settle N] [--hide-project-paths] [--no-source-excerpts] + [--include-dumps] [--no-redact]`；CI（如 GalTranslPP 的验证 workflow）直接把问题包作为 artifact 上传。 + +**内容**（zip，默认写到 `/bundles/mcppls-bundle-<时间>.zip`，完成后给出"在文件夹中显示 / 复制路径"；**从不自动上传**）： + +| 文件 | 内容 | +|---|---| +| `manifest.json` | 格式版本、生成时间、包含了什么、应用了哪些脱敏规则及各自命中次数（**不含原值**）、各文件大小与哈希 | +| `report.json` | 当前的 `cxxModules/report`（引擎状态、计划、事件、工具运行、workaround） | +| `environment.json` | 操作系统/内核/架构、CPU 与内存、区域与代码页（Windows ACP）、编辑器名与版本、插件版本、其他 C++ 插件的安装与启用状态、mcppls 设置、payload 版本（`payload.json`）、clangd / mcpp / xlings 版本、工具链探测结果、环境变量**白名单**（`MCPP_*`、`XLINGS_*`、`LANG`/`LC_*`、`PATH`） | +| `logs/` | 最近 3 次会话或 24 小时内的服务端日志（超出按"头 + 尾"截断）；插件自己的输出通道日志 | +| `incidents/` | F17 的事故快照：clangd 日志片段、线程 CPU、fileStatus 时间线、缓冲区/磁盘差异、崩溃上下文与退出码 | +| `engine/` | 引擎数据库（`compile_commands.json`）、计划摘要、模块图 | +| `dumps/` | 仅 `--include-dumps` 或界面勾选时：Windows minidump（体积大，且含内存内容） | + +**不包含**：源代码全文。事故里只带与问题直接相关的行（例如 `import hello.`），可用 `--no-source-excerpts` 关掉；项目路径默认按 +主目录规则替换为 `~/…`，`--hide-project-paths` 再替换为 `/…`。 + +**脱敏**（对包内每个文本文件执行，JSON 与纯文本都适用）： + +1. **主目录 → `~`**：覆盖 POSIX 路径、Windows 路径（`\` 与 `/`、JSON 转义的 `\\`、盘符大小写）、8.3 短名（`C:\Users\RUNNER~1`）、 + `file://` URI 及其百分号编码（`%3A`）、WSL 的 `/mnt/c/Users/<名>`。 +2. **用户名 → ``、主机名 → ``**：只在路径与已知字段（环境变量值、URI 的 authority）中替换，避免把普通单词误改。 +3. **密钥 → ``**：环境变量只收白名单；键名像 `api_key`/`token`/`secret`/`password`/`authorization` 的值；已知前缀 + （`ghp_`、`github_pat_`、`sk-`、`xoxb-`）；邮箱；编译参数里名字像密钥的 `-D=`；AI 网关相关设置的值。 +4. **一致映射**：同一原值总是换成同一占位符，路径仍可比较；映射表本身不进包。 +5. **写出前自检，失败即不写**：打包完成后在所有文件里搜索原始的主目录、用户名、主机名（上述各种写法）；只要还有残留，就不生成 + 问题包，并报告是哪个文件、哪条规则漏了。 +6. `--no-redact` 只给本地排查用，界面上不提供。 + +**同步改动**：`Collect Report` 默认也走同一套脱敏（它本来就是为贴到 issue 准备的），界面文案从"请自行编辑"改为"已替换用户名与路径， +可在导出前预览"。 + +**测试**：脱敏单测覆盖上面每种写法（Linux 与 Windows、JSON 转义、URI 编码、8.3 短名、WSL）；自检"残留即失败"；conformance： +在一个用户名较长且出现在各类路径里的临时 HOME 下导出，断言包里任何文件都不含原用户名与主目录、manifest 与实际内容一致、 +大小在上限内（例如 25 MB）。 + +### F6. 扫描失败自解释（P1） + +解析 `Scanning modules dependencies for <文件> failed: <原因>` 汇总为 issue（次数、前几个文件、第一条原因；驱动错误时措辞为 +"clangd rejected the compile command for module scanning: `<原因>`"——#23 的报告者因此能直接看到 `LTO requires -fuse-ld=lld`）； +崩溃上下文、扫描失败、`Failed to build module` 行不受 `LineLimiter` 限流；报告增加 `scanFailures`。 + +## 5. 其他 + +- **F7（UP-M1）**：`.github/versions.env` 的 `MCPP_VERSION` 升到 ≥ 2026.9.24.1，`mcpp.toml` 的 `[build]` 加 + `macos_deployment_target = "11.0"`，用 `llvm-objdump --macho --private-headers` 验证 `minos 11.0`。 +- **F10（mcpp#699 落地后）**：`parse_database_envelope`（`src/spec/discovery.cpp:62`）只要有 `data` 就接受、不看退出码，部分数据库 + 会被直接使用（前提：mcpp 保持 `kindVersion = 1`）；届时只需把失败成员的诊断显示为 issue。 +- **V（验证）**： + - Windows：Sunrisepeak/GalTranslPP PR #1 的 workflow 加 `workflow_dispatch` 输入，改用 mcppls 某次 CI 的 win32-x64 payload；验收： + E1 可信阶段 0 次 LTO 扫描失败、hover 有内容（含跨模块）、隔离的都是 clangd 报出的文件、不因临时阶段被判不可用、BOM 文件无误报。 + - Linux：把本次的逐字输入脚本（`import hello.` 写盘 A/B、`import hello.test.a;` 未保存、线程 CPU 采样）固化为 conformance 夹具； + 验收：无满载 worker、无计划驱动的重启风暴、关键字补全在任何时候都可用。 + - 问题包（F18）：两个验证环境都用 `mcppls report --bundle` 产出问题包并作为 artifact 上传；验收：包含事故快照与多次会话日志，且任何 + 文件都不含 runner 的用户名与主目录（Windows `C:\Users\runneradmin`、8.3 短名 `RUNNER~1`，Linux 的 `$HOME`）。 + - 验证结束后关闭 GalTranslPP PR #1。 + +## 6. 决策与自我 review + +### 6.1 已定的决策（2026-09-26） + +| # | 决策 | 结论 | +|---|---|---| +| D1 | F4：producer 回答前 clangd 等待（最长 60 秒，只剩模块级功能） | 做，只对识别出构建系统的项目 | +| D2 | F16.3：UP-01 的根治是否捆绑自建 clangd | 本期只做 F16.1/F16.2 的规避；上游处理推迟，已在 #24 的 UP-01 标记（待账号权限恢复后写入，见 §6.4）；自建作为后备 | +| D3 | F17.1：常开的 clangd 日志级别 | `--log=info` 进环形缓冲，出事后临时提到 verbose 再采一段 | +| D4 | F9：空格触发如何避免频繁处理 | 四层：编辑器侧拦截（零 IPC）→ 服务端前缀门槛（不转发 clangd）→ 结果按计划代次缓存 → 只对已知客户端声明；并计数验证开销 | +| D5 | F15：`export module ` 后是否建议名字 | 不建议名字，只补关键字 | +| — | 新增 F18：问题包导出与脱敏 | 做，P1，进 0.0.5 | + +### 6.2 这一版做了什么 + +- 写入 D1–D5；F9 增加 D4 的四层策略；F16.3、F17.1 按决策改写。 +- 新增 F18，并把 F17 的事故通知、状态栏提示、`Collect Report` 都接到它；事故目录加保留上限（F17.7）。 +- 重算：合计 18–23 人日；0.0.5 约 12–14 天（新增 F18）；依赖链改为 F3 → F14 → F17 → F18。 +- 更正：F16 是"只有 `import hello.` 有提示、停久了超时"的根因，但不是"重启 3 次"的根因（那是 F13 的计划抖动加一次切换工具链）； + §0.1、F16、§6.3 已按日志改写。F14 的"用户操作不计"改为以模型的工具链/profile 变化为判据。F18 的失败处理统一为"残留即失败"。 + +### 6.3 自我 review：检查过的点 + +- **编号与引用**：#24 引用的 F1、F3、F4、F5、F7、F11、F12、F16 都在，含义未变；新增的 F17、F18 未被外部引用。 +- **优先级一致**：P0 只有"会让功能完全失效"的三项（F1、F2、F16）；P1 是"会反复失效或无法定位"的（引擎恢复、F11、F17、F18、F6）； + 体验项为 P2。 +- **版本边界**：0.0.5 覆盖 issue #23 的关闭条件（F1、F2、F3、F6）与 hello 场景的根因与锁死（F16、F13、F14），并能导出现场（F17 部分、 + F18）；F4 放 0.0.6：hello 场景的 3 次重启里两次是计划抖动（F13 消除）、一次是切换工具链（F14 不再计入）；F4 针对的是 Windows 冷启动 + 临时阶段的崩溃，F14 的分预算已能防止它把 clangd 锁死，F3 能正确归因，因此可以晚一个版本。 +- **互相冲突的地方**：F13 的 2 秒防抖与 F11 的信息级诊断配合，不会出现"合法 import 被报错"；F9 的服务端门槛与 F15 的关键字补全 + 走同一条补全路由，一起实现避免两次改动；F18 的脱敏同时用于 `Collect Report`，避免两套规则。 +- **隐私**：问题包默认不含源代码全文、不含 minidump、不含非白名单环境变量；"残留即失败"的自检保证不会带着用户名写出。 + +### 6.4 剩余风险与待办 + +1. **F16 只覆盖了实测到的读盘路径**：后台索引同样读磁盘，可能在其他线程上遇到 UP-01。实现前先验证（写盘 `import hello.` 后观察 + `ground-worker-*` 线程）；若也会卡住，F16.1 需把该文件暂时移出后台索引，或在命中时推迟转发文件变化。 +2. **F1 改变所有命令**：clangd 模块缓存按命令哈希，升级后首次打开会重建全部 BMI（一次性），更新说明写明。 +3. **F13 的 2 秒防抖**让合法的新 import 晚约 2 秒生效；由 F11 的信息级诊断兜住体验。 +4. **F14 的退避可能掩盖崩溃循环**：用 F3 的归因区分，崩溃循环仍走现有上限。 +5. **Windows 的 UP-13** 在 0.0.5 只能止损，根因等取到调用栈（#24 待办）。 +6. **F18 的脱敏漏网**：用户名很短或是常见词时，只在路径与已知字段替换，其他位置可能残留；由"残留即失败"兜底：界面上只提供 + "隐藏项目路径后重试"，不提供"仍然导出"；确需原样内容时只能在命令行用 `--no-redact`，且只用于本地排查。 +7. **时序敏感的测试**（F13、F14、F16）：夹具用事件驱动而非固定 sleep，避免 CI 不稳定。 +8. **issue #24 的待写入更新**：当前 gh 账号 `speak-agent` 对仓库只有 pull 权限，UP-01 的"D2：上游处理推迟"说明与索引中 UP-01、UP-14 + 两行的更新已在本地准备好，需要切回有权限的账号（或给该账号授权）后写入。 + +## 7. 实施计划：0.0.5 一个 PR 做完（2026-09-26 定） + +范围调整：§0.3 的两个版本合为 **0.0.5 一次发布、一个 PR**（与 0.0.4 的 #22 同样的形态）；F1–F9、F11–F18 全做，F10 等 +mcpp#699（仍 open），F16.3 按 D2 推迟。 + +### 7.1 多角度的约束（每个工作包都按这 8 条自查） + +| 角度 | 约束 | +|---|---| +| 架构 | 新能力放进独立模块，引擎只通过 `Host` 与工作区交互：崩溃/扫描行解析在 `engine.clangd.process`；重启预算在 `engine.clangd.guard`（`RestartGate` 改为按原因分预算 + 退避）；日志环形缓冲与事故记录是引擎无关的新模块（`Host::record_incident`，默认空实现，与 `record_event` 同形）；问题包与脱敏是新目录 `src/bundle/`，只读缓存目录与报告，不反向依赖引擎 | +| 稳定性 | 任何新逻辑都不阻塞事件循环（问题包在后台线程生成、以事件回到循环）；预算/退避/防抖都有上限；脱敏失败即不写出；事故目录有保留上限；时序测试用事件驱动 | +| 优雅简洁 | F1 一条规则（补 `-c`）替代链接参数清单；F18 一套脱敏同时服务问题包与 Collect Report；F9/F15 走同一条补全路由；每个 workaround 仍在注册表里（F12 是 `WA-CLANGD-006`） | +| 用户体验 | 不卡死、不锁死；状态栏说出原因并给出动作（导出问题包 / 重启 clangd）；import 行空格即出模块列表、关键字补全任何时候可用；`export` 高亮一致；未保存的新 import 是信息而非错误 | +| 兼容性 | 非 VS Code 客户端：空格触发默认不声明、可用初始化选项打开；新命令都走标准 `workspace/executeCommand`；报告与状态只增字段（S3 附加）；clangd 日志 info 行仍只进 debug 级日志，不增加默认日志量 | +| 跨平台 | 崩溃上下文同时认 Windows（`Exception Code`）与 POSIX（信号）；线程 CPU 在 Linux 读 `/proc`，其他平台退化为进程 CPU；脱敏覆盖 Windows 路径、8.3 短名、`file://` 编码、WSL；zip 写出器纯 C++、无平台依赖；F7 macOS 11 | +| 一致性 | 事件名 `engine-*`、报告字段 camelCase、文档中英同步、S3 规范 + traceability + 夹具一起改、#24 与 workaround 注册表对应 | +| 无感升级 | 设置默认值不变；新设置可选；缓存目录只新增 `incidents/`、`bundles/`；F1 让 clangd 命令哈希变化 → 首次打开重建 BMI 一次（CHANGELOG 写明）；不需要迁移 | + +### 7.2 工作包与依赖 + +| 包 | 内容 | 主要文件 | 依赖 | 执行 | +|---|---|---|---|---| +| **A** | F1、F2、F5、F8 | `normalize/{gnu,msvc,semantic}.cpp`、`project/{scan,infer}.cpp`、`engine/native/exports.cpp`、VS Code 语法 | 无 | 并行（独立 worktree） | +| **C** | F9、F15 | `orchestrator/{routing,workspace(路由入口)}.cpp`、`engine/native{,/index}.cpp`、`server/session.cpp`（初始化选项）、VS Code middleware | 无 | 并行 | +| **D** | F18 + Collect Report 脱敏 + VS Code 两个命令（导出问题包、重启 clangd） | 新 `src/bundle/`、`cli/{options,commands}.cpp`、`server/session.cpp`（`mcppls.exportBundle`）、VS Code | 与 B 的契约（7.3） | 并行 | +| **B** | F3、F6、F14、F4、F13、F16、F11、F12、F17（1–7） | `engine/clangd{,/process,/guard,/workarounds}.cpp`、`orchestrator/workspace.cpp`、新事故模块 | 内部顺序：F3/F6 → F17.1–2 → F14 → F4 → F13+F16 → F11/F12 → F17.3–7 | 主线 | +| **E** | F7 | `.github/versions.env`、`mcpp.toml` | 无 | 主线 | +| **F** | 版本 0.0.5、CHANGELOG、docs（中英）、S3 规范/schema/traceability、design.md、本文实施记录 | — | A–E 合入后 | 主线 | +| **V** | Linux conformance 全量 + 新夹具；Windows：GalTranslPP 探针用本 PR 的 payload；问题包验收 | — | F 之后 | 主线 | + +合入顺序：A、C、D 各自在 worktree 分支上完成并通过单测，按 A → C → D 合进 `release/0.0.5`（冲突只可能在 +`session.cpp`、`workspace.cpp` 路由入口、`conformance.cpp`、`traceability.json`、VS Code `package.json`，均为追加），然后 B、E、F、V。 + +### 7.3 B ↔ D 的契约（问题包读什么） + +- 事故目录：`<工作区缓存>/incidents/-/`，内含 `incident.json`（kind、时间、根目录、原因链、 + 涉及文件与其缓冲区/磁盘差异的相关行、fileStatus 时间线、线程 CPU、退出码与崩溃上下文、最近计划差异、重启历史）与 + `clangd.log`（环形缓冲落盘）。保留最近 20 个或 7 天。 +- 服务端日志：`<缓存>/logs/server-*.log`（`open_log_file`）；工作区缓存目录在报告的 `roots[].cacheDirectory`。 +- 服务端命令：`mcppls.exportBundle`（D）、`mcppls.restartEngine`（B：绕过预算重启一次，记为用户操作、不计入预算）；VS Code 命令 + `mcppls.exportDiagnosticBundle`、`mcppls.restartClangd`（D）。状态 issue 的 `command` 可指向这两个（B）。 + +### 7.4 完成标准 + +单测（dev + release profile）、`devtools check all`、`validate.py`、全部 conformance（含新夹具）本地通过；PR 的 CI 三平台全绿； +自我 review（含一次独立 review agent）后无遗留问题；squash 合入；Release 工作流发布 0.0.5；下载发布产物、校验哈希、本地装 VSIX 与 +payload 验证；最后报告产物目录。 + +## 8. 实施记录（0.0.5，2026-09-26） + +一个 PR（#26，分支 `release/0.0.5`；并入 #25 的 S2 0.3.0，F10 因此不再推迟）完成 §7 的全部工作包：A、C、D 由三个并行分支各自实现、单测与夹具验证后合入，B、E、F 在主线实现。 +各条的落点与验证： + +| # | 实现 | 证据（在 0.0.4 上失败、在 0.0.5 上通过） | +|---|---|---| +| F1 | `translate_gnu`/`translate_msvc`/`kit_arguments` 末尾恰好一个 `-c`（在 `-x c++-module` 前），`-flto` 保留 | `test_normalize`；夹具 `compdb-lto-msvc`（Linux 上复现 `LTO requires -fuse-ld=lld`） | +| F2 | `base::byte_order_mark_size`；两个词法器从 BOM 之后开始，位置按原文；WA-CLANGD-001 的改写也跳过 BOM | `test_scan`、`test_exports`、`test_workarounds`；夹具 `inferred-bom` | +| F3 | `LogReader` 解析崩溃上下文（含 Windows 异常码）；退出在 500 ms 后结算，只隔离 clangd 点名的文件；`lastExit` 进报告与状态 | `test_server`（#23 的原始日志行）；夹具 `clangd-crash-context`（替身 clangd） | +| F4 | 识别出构建系统时，clangd 在 producer 回答或 60 s 前不接收计划、不被路由请求；临时模型被替换时清零其崩溃/重启/隔离记账 | 夹具 `mcpp-emit-wait` | +| F5 | `SKIPPED_DIRECTORIES` 与子目录 `vcpkg.json` | `test_project` | +| F6 | 扫描失败按驱动错误（environment）/缺头文件（project）/代码错误（仅计数）分类；关键行不受限流 | `test_server`；夹具 `compdb-rejected-command` | +| F7 | `MCPP_VERSION=2026.9.26.1`、`macos_deployment_target = "11.0"`；本机交叉编译读出 `minos 11.0` | 本机验证 | +| F8 | 模块单元中每个 `export` 都是 keyword；VS Code 注入语法新增规则 | `test_scan`/`test_tokens`、`grammar.test.ts`、`inferred` 夹具 | +| F9、F15 | 见 C 分支：四层防频繁、关键字合并、`mcppls.completion.triggerOnSpace`、S3 §6.2 | `test_completion`、夹具 `completion-keywords`、VS Code 单测；门槛开销均值 77 µs | +| F11、F12 | `WA-CLANGD-007`（未保存的 import 改为信息）、`WA-CLANGD-006`（缺 `;` 的诊断移回指令行），均在注册表内、可关闭 | `test_workarounds`；`typing-autosave` 的 D1、D2 | +| F13 | 编辑中的结构变化 2 s 静默后再计划；计划重启 2 s 合并；stand-in 进出不重启；clangd 未打开的文件改参数不重启；替身不被当作新提供者（hello 日志里第 1 次重启的真因） | `test_server`；`typing-autosave` 的 T6–T11 | +| F14 | `RestartGate` 按原因分预算，超限退避 1/2/4/8 分钟；用户重启与工具链切换不计 | `test_server` | +| F16 | 见下文"与方案的差异"1 | `typing-autosave`（0.0.4 在 T1 即全部请求无应答）；hello 副本线程 CPU 0 tick | +| F17 | clangd `--log=info` 进 4000 行内存环；事故目录（保留 20 个、7 天）含日志、编辑器/磁盘差异行、命令、fileStatus 时间线、线程 CPU；计划差异日志；workaround 前提守卫（WA-001/002）；状态按钮标题 | `test_server`、`test_process`；`typing-autosave` T3 | +| F10 | 并入 PR #25（S2 0.3.0，speak-agent）：`EnvelopeDiagnostic.path`；有 `data` 又有 `error` 的文档照常使用，每个未能描述的部分记为模型 issue `producer-partial`；mock mcpp 可给出这种文档；traceability S2-3.4-12/13 | `test_spec`；夹具 `mcpp-emit-partial`（0.0.4：用了数据但不报告） | +| F18 | 见 D 分支：`src/bundle/`（脱敏、zip、打包）、CLI/服务端/VS Code 入口、Collect Report 与 `cxxModules/report` 默认脱敏（S3-5.5-3） | `test_bundle`；夹具 `diagnostic-bundle`；CI 上传各平台问题包 | + +**与方案的差异**（实现中发现，已按根因处理）： + +1. **F16 扩为"磁盘文本对 clangd 是否安全"。** 按方案只查 `import hello.` 时，夹具暴露了第二种情况：自动保存把尚不存在的模块名 + （`import hello.e`）写到磁盘，clangd 从磁盘读到后同样卡死（UP-02 经 UP-14 的路径）。根因有三：H3 推迟 stand-in 的前提"正在输入的 + import 只在缓冲区里"被自动保存打破；计划对 inferred 模型的单元沿用加载时的 `requiredModules`，看不到正在编辑的 import；扫描器不认 + 缺 `;` 的 import，而 clang 认（P1857）。已分别修正：编辑中的文件以缓冲区与磁盘的 import 合并计划，磁盘上已有的 import 立即得到 + stand-in；扫描器记录 `unterminatedImports`；磁盘检查同时覆盖"数据库里还没有、或 clangd 还没重读到"的模块，文件在 stand-in 被 + 读到后（约 6 s）交还。自我 review 后收窄：这条只针对上一次计划之后新输入的 import（计划已见过的仍按原有机制处理），跳过 `std` + 与经模块清单解析的模块，且最多 30 s（计划始终不给单元时回到 0.0.4 的行为），避免打开文件时误隔离。 +2. **F16 的"保存前已开始的构建"。** 夹具的输入方式（改动与保存几乎同时）让 clangd 在保存前开始的构建读到新磁盘内容。按方案的 + "隔离即可"不够：已开始的构建停不下来。现为先让它结束（1.5 s），仍未结束则重启 clangd 且不带该文件；Windows 上这一情形是崩溃， + 崩溃文件若已知磁盘不安全，同样只隔离到磁盘修好。 +3. **F17.1 的"出事后临时切到 verbose"未做**：clangd 运行中不能改日志级别，需要重启；事故带 `info` 日志，需要 verbose 时按文档用 + `--log-level debug` 启动（design.md §7 已写明）。 +4. F13 的"计划变回原样则取消重启"未做：取消需要让旧 clangd 重新与计划同步（打开/关闭文件），复杂度与收益不相称；以 2 s 合并、 + stand-in 与未打开文件不触发重启代替。 + +**验证**：dev 与 release profile 单测 29/29；`devtools check all`、`validate.py` 通过；Windows/macOS 交叉编译通过；本机 Linux +全部 conformance（CI 两部分清单 + 新夹具，共 50 个）通过；新夹具逐个对照 0.0.4 确认失败。hello 项目副本：`impo`/`expo` 从第一个字母起 +有 `import`/`export module`；`import hello.` 写盘后 clangd 各线程 20 s 内 0 tick（0.0.4 约 2000）。 + +## 9. 追加：C++26 支持对齐（2026-09-26，用户要求并入 0.0.5） + +调查结论（本机实测，clangd 23.1 + 语义工具包 libc++ 23 / 工具链 clang 22 + libc++ 22）： + +| 问题 | 实测 | 处理 | +|---|---|---| +| 无构建描述的项目默认 `-std=c++23`（MSVC 族却是 `/std:c++latest` 即 C++26），C++26 库名（`std::saturating_add`）缺失 | `inferred-cxx26` 在 0.0.4 上失败 | 默认取读取它的编译器支持的最新标准：语义工具包、GCC ≥ 14、Clang ≥ 17（< 20 写作 `c++2c`）为 C++26，更老的为 C++23；工具包在单元未写标准时也取 C++26 | +| 同一项目混用 C++23 与 C++26：`std` 按代表单元（C++23）构建，C++26 单元 `import std` 直接失败（"C++26 was disabled in precompiled file"），经它导入的一切都丢失 | `compdb-mixed-standards` 在 0.0.4 上失败 | 计划第 5c 步：同一上下文中涉及模块的单元统一为其中最新的标准，普通单元保留自己的；报告 `plan.languageStandard/standardsSeen/standardsRaised`，状态 profile 增加 `standard`（S3，附加字段） | +| 用户看不到当前按哪个标准读 | — | 状态 profile 与报告给出；提升时写一行日志 | +| 契约（P2900）、反射（P2996） | clang 22/23 不认 `pre`/`post`/`contract_assert`（`-fcontracts` 不存在） | 上游能力缺口，不在 mcppls 能补的范围：文档写明；VS Code 注入语法为 `contract_assert` 与合约说明符位置的 `pre`/`post` 着色（纯外观）；服务端对 `contract_assert` 发 keyword 语义 token（clangd 可用时以 clangd 为准） | + +其余 C++26 语言特性（包索引、`= delete("reason")`、占位符 `_`、`static_assert` 消息、`#embed` 等）以 clangd 23.1 的实现为准，mcppls +无需额外处理；标准库部分以构建所用的标准库为准。 + +## 不做的事 + +- 不删除 `-flto` 或其他链接参数（F1）。 +- 不为 UP-13 做规避性改写：只止损，取栈与上游报告在 issue #24 跟踪。 +- 不改写用户的文件（F16 只检测磁盘内容，不修改）。 +- 问题包不自动上传，不含源代码全文与非白名单环境变量（F18）。 +- mcpp 侧的 UP-M2/UP-M3 已合并提交为 mcpp-community/mcpp#699，在 issue #24 跟踪，本方案只做 F10 的适配。 diff --git a/.agents/docs/2026-09-26-issue-23-lto-module-scan.md b/.agents/docs/2026-09-26-issue-23-lto-module-scan.md new file mode 100644 index 0000000..ee7c1bd --- /dev/null +++ b/.agents/docs/2026-09-26-issue-23-lto-module-scan.md @@ -0,0 +1,116 @@ +# Issue #23 分析：Windows/MSVC + LTO 下 clangd 模块扫描失败、clangd 反复退出 + +状态：分析完成（未改代码）· 2026-09-26 · `main` 04b186a（0.0.4）· 捆绑 clangd 23.1.0（clangd/clangd 官方 Windows +发布版）· issue： + +证据来源： + +- 本机（Linux x64）：Clang 22 驱动、`clang-scan-deps`、捆绑 clangd 23.1.0，对照 llvm-project 23.1.0 源码。 +- **Windows 实测**：Sunrisepeak/GalTranslPP 临时 PR #1 的 workflow（windows-2025，VS 18 / MSVC 14.51，mcpp 2026.9.25.1， + LLVM 22.1.8，Qt 6.11.1，vcpkg 22 个 port），跑 mcppls 0.0.4 的 win32-x64 payload。第 4 轮（run 36178100611）复现了 + 报告者的环境：mcpp producer 成功，引擎数据库 **227 条、227 条带 `-flto`、0 条带 `-c`**（报告者也是 227 条）。 + 第 3 轮（run 36164741493）因 CI 的 Qt 不全（`lupdate.exe` 依赖 `Qt6Qml.dll`）producer 失败，意外覆盖了兜底模型路径。 + +## 结论 + +| # | 问题 | 归属 | 结论 | 证据 | +|---|---|---|---|---| +| **M1** | clangd 数据库去掉 `-c`、保留 `-flto`：windows-msvc 目标下驱动报 `LTO requires -fuse-ld=lld`，clangd 的 P1689 扫描失败，**不建任何 BMI** | mcppls | **根因，已确认** | 本机复现 + Windows 实测：可信模型阶段扫描失败 12 次**全部**是这条；210 次 hover 全空 | +| **U1** | clangd 23.1.0（Windows）在"没有 BMI、import 解析不了"的模块单元上 Build AST 时崩溃（`0x80000003`） | clangd 上游，**由 M1 触发** | 已确认；修 M1 即消除 | `GPPDefines.ixx`：原样命令 `--check` 崩溃（E3a），加 `-c` 后 exit 0（E3b）；编辑器会话 20 秒内崩在它上面（E3c） | +| **U2** | clangd 23.1.0（Windows）在 `NormalJsonTranslator.Core.cpp` 等文件上 Build AST 崩溃，**与 `-c`、LTO 无关** | clangd 上游 | 已确认，根因未定位 | 加 `-c`、BMI 正常、跨模块 hover 正常时仍崩（E3d）；Linux 上同形状不崩 | +| **M2** | 扫描器不认 UTF-8 BOM：文件以 BOM + `export module X;` 开头时认不出模块接口 | mcppls | 已确认，**新发现** | 本机：可信模型下 mcppls 模块引擎误报 `module 'm' not found`，去掉 BOM 即消失；兜底模型下还漏掉 `-x c++-module`（CI：3 个 `.ixx`）。GalTranslPP 中 `GPPDefines`、`ProgressBar`、`TerminalController` 受影响 | +| **M3** | 崩溃嫌疑判断不准：按"退出前 10 秒碰过的文件"判定，而 clangd 已打印崩溃文件 | mcppls | 已确认 | E1：clangd 报 `NormalJsonTranslator.Core.cpp`，mcppls 隔离 `GPPDefines.ixx`；前两次崩溃嫌疑为空。**报告者"`GPPDefines.ixx` 被隔离"很可能同属误判** | +| **M4** | producer 尚未回答时的临时模型（`firstOrigin: inferred`）用机器上找到的 MinGW（`x86_64-w64-mingw32`）、不带项目 include 目录喂给 clangd；这一阶段的崩溃耗掉重启额度 | mcppls | 已确认 | E1：5 次退出中 3 次发生在前 18 秒的临时阶段；E4（关 LTO）可信阶段本身正常，仍因临时阶段的崩溃出现 `engine-restart-capped` | +| **M5** | producer 失败时的兜底扫描把 `vcpkg_installed/` 当项目源码（349 条中 166 条），产生伪 `ambiguous-module`（`proxy.v4`） | mcppls | 已确认，影响小 | 第 3 轮 | + +一句话：**报告者看到的"模块全部不可用 + clangd 反复退出 + `GPPDefines.ixx` 被隔离"，是 M1（没有 BMI）→ U1（Windows clangd 在 +无 BMI 的模块单元上崩溃）→ M3（隔离错文件）叠加 M4（临时阶段的崩溃先耗掉额度）的结果**；U2 是独立的上游崩溃；M2 是同一项目 +暴露的独立 bug。 + +## 1. M1：命令是怎么变成这样的、为什么只坏在扫描上 + +- `src/normalize/plan.cpp:220-236`：单元参数来自 `options_arguments`（`semantic.cpp:86` 会加 `-c`）或构建命令，再经 + `translate_gnu`/`translate_msvc`。 +- `src/normalize/gnu.cpp:20,49`：`ALWAYS_STRIP_EXACT` 删掉 `-c`，此后无处补回；`-flto`、`-O3`、`-g` 原样保留。 + `src/normalize/msvc.cpp:33-40` 放行所有 `-f*`，输出同样无 `-c`。所有派生条目（单元 `:473`、游离文件 `:296`、prime + `:499-501`、std `:523-547`、stand-in `:577-581`）都继承。`tests/test_normalize.cpp:70,114` 断言输出无 `-c`——首个版本 + 起就是这样，0.0.3/0.0.4 都受影响，不是回归。kit 路径（`semantic_subset`）不受影响。 +- 驱动检查（`clang/lib/Driver/Driver.cpp:4348-4364`）只在 `FinalPhase == Link`、windows-msvc、开 LTO、`-fuse-ld` 非 lld + 时报 `err_drv_lto_without_lld`。 +- clangd 建 AST/BMI 走 `createInvocation`（`clang/lib/Driver/CreateInvocationFromArgs.cpp:51`），会插入 `-fsyntax-only`， + 不受影响（Windows 日志里 cc1 为 `-O3 -fsyntax-only -flto=full`）；而模块扫描 + （`clangd/ProjectModules.cpp:213-240` → `DependencyScanningTool.cpp:121` 的 `containsError()`）用原命令，直接失败。 + `CompoundProjectModules::getRequiredModules` 直接用扫描结果 → 空 → 不建 BMI；`MODULE_HINTS` 也要扫描确认，同样失败。 + +本机复现（`-###` 不执行任何步骤）： + +| 命令形状（`clang++ -x c++-module a.ixx -###`） | 退出码 | +|---|---| +| `--target=x86_64-pc-windows-msvc -flto` / `-flto=thin` | 1，`LTO requires -fuse-ld=lld` | +| 同上加 `-c`，或加 `-fuse-ld=lld` | 0 | +| `--target=x86_64-linux-gnu -flto` | 0 | + +## 2. Windows 实测（第 4 轮，报告者环境） + +| 实验 | 设置 | 结果 | +|---|---|---| +| E0 | `mcppls report`，不开文件 | 在引擎开始干活前结束，无异常（快照意义有限） | +| E1 | `mcppls serve` 编辑器会话 15 分钟（GPPDefines.ixx、ApiTool.ixx、GPPDefines.cpp、NormalJsonTranslator.Core.cpp、GPPCLI.cpp） | 可信阶段扫描失败 12 次全为 LTO；clangd 90 秒内退出 5 次后被判不可用（`state: unavailable`）；**210 次 hover 全空** | +| E2 | `mcppls check GPPDefines.ixx` | 扫描 LTO 失败 1 次 | +| E3a | clangd `--check GPPDefines.ixx`，mcppls 的数据库原样 | **崩溃 `0x80000003`** | +| E3b | 同上，数据库每条加 `-c` | **exit 0，无扫描失败，无崩溃** | +| E3c | clangd 单独会话，原样数据库 | 第一轮请求前即崩在 `GPPDefines.ixx` | +| E3d | clangd 单独会话，加 `-c` | 无扫描失败；跨模块 hover 生效（`string` → `provided by `）；约 100 秒后崩在 `NormalJsonTranslator.Core.cpp`（U2） | +| E4 | E1 但默认 profile 改为 `fast-release`（无 LTO） | 可信阶段无扫描失败、无崩溃；48 次 hover 有内容（6 次跨模块）；但临时阶段崩溃 2 次、随后 `engine-restart-capped`、文件被隔离与"隔离后仍在处理"的重启（M3/M4） | + +E1 的时间线（mcppls 事件 × clangd 崩溃上下文）： + +| 时间 | 模型阶段 | clangd 报的崩溃文件 | mcppls 的嫌疑 | +|---|---|---|---| +| 19:15:23 | 临时（MinGW，无 include） | `GPPCLI.cpp` | 无 | +| 19:15:27 | 临时 | `GPPCLI.cpp` | 无 | +| 19:15:38 | 临时 | `NormalJsonTranslator.Core.cpp` | **`GPPDefines.ixx`**（错） | +| 19:15:55 | 可信（MSVC + `-flto`） | `ApiTool.ixx` | `ApiTool.ixx` | +| 19:16:45 | 可信 | `NormalJsonTranslator.Core.cpp` | `NormalJsonTranslator.Core.cpp` | + +没有拿到崩溃栈：clangd 只打印了 `Signalled during AST worker action: Build AST` 与 `Exception Code: 0x80000003`，未写 +minidump，Application 事件日志为空。`0x80000003` 在 LLVM 的 Windows 构建中通常意味着内部 `abort()`（LLVM 把 SIGABRT 转成 +陷阱以进入崩溃处理），日志中没有断言或 `LLVM ERROR` 文本。 + +## 3. M2：BOM + +`src/project/scan.cpp` 没有跳过 UTF-8 BOM。第一行是 `module;` 的文件不受影响（后面的 `export module` 仍被识别），第一行就是 +`export module X;` 的文件会被判为非模块。可信模型下 clangd 的数据库用 mcpp 给的 `ide.role`/`provides`（mcpp 构建数据库自带 +这两个字段），所以不影响给 clangd 的命令；但 mcppls 自己的模块引擎按文本扫描，会误报。兜底模型下还会漏掉 `-x c++-module`， +clang 把 `.ixx` 当成链接输入（`'linker' input unused`）。 + +本机复现：一个 mcpp 项目，`src/m.cppm` 写成 BOM + `export module m;`，`mcppls check src/main.cpp` 输出 +`module main.cpp:1: module 'm' not found`；去掉 BOM 后无此行。 + +## 4. 修复建议（未实施） + +| 优先级 | 项 | 做法 | 测试 | +|---|---|---|---| +| P0 | **M1** | 保留删除构建自带的 `-c`/`--precompile`，但在 `translate_gnu`、`translate_msvc`（与 `kit_arguments`）输出末尾补一个 `-c`；所有派生条目随之修正。不删 `-flto`：`-c` 一次关掉所有链接阶段检查，且不动项目的 LTO | `test_normalize.cpp:70,114` 改为断言恰好一个 `-c`；新增 windows-msvc + `-flto` 用例；一个 Linux 可跑的 clangd 级 fixture（本机已证明 windows-msvc 目标在 Linux 上也复现） | +| P0 | **M2** | 扫描器（及模块引擎的文本扫描）跳过开头的 UTF-8 BOM | 单测：BOM + `export module`、BOM + `module;` | +| P1 | **M3** | 解析 clangd 的崩溃上下文（`Signalled during AST worker action …` 后的 `Filename:`），以它为唯一嫌疑；没有上下文时才回退到现有启发式 | 用 mock clangd 输出崩溃上下文的 conformance fixture | +| P1 | **M4** | producer 尚未回答的 mcpp 项目：临时模型不把 MinGW 等"猜来的"工具链喂给 clangd（或至少读取 `mcpp.toml` 的 `[toolchain]` 与 `include_dirs`）；模型从临时切到可信时清零崩溃计数与隔离 | fixture：producer 延迟回答 + 临时阶段崩溃不导致 `engine-restart-capped` | +| P2 | **M5** | 兜底扫描排除包管理器目录（`vcpkg_installed`、`.conan*`、`node_modules`、`target` 等） | 单测 | +| P2 | 诊断 | `engine-exit` 事件与报告记录 clangd 退出码（`Connection::exit_code()` 已有）与崩溃文件；把 `Scanning modules dependencies for X failed: ` 汇总成 issue；崩溃上下文行不受 `LineLimiter` 限流 | — | + +预期效果（按 Windows 实测推断）：M1 修复后可信阶段扫描恢复、BMI 可建、U1 不再触发（E3b/E3d 已证明）；剩下 U2 这类上游崩溃， +由 M3 精确隔离到单个文件、M4 不让临时阶段耗掉额度，clangd 对其余文件保持可用。 + +## 5. 上游（clangd） + +- U1 与 U2 都是 clangd 23.1.0 Windows 官方发布版在 Build AST 时的崩溃，应向 llvm-project 报告，前提是一个最小复现和调用栈。 +- 下一步：在 GalTranslPP 的 CI 上用 Windows SDK 的 `cdb.exe` 挂 clangd 取栈；并用 LLVM 22.1.8 自带的 clangd 对照，判断是否为 + 23.x 回归。U1 可从 `GPPDefines.ixx`（只有 import、无 include）开始缩小。 + +## 6. 给报告者的回复要点 + +- 根因已在 Windows 上复现并定位（M1），下一个版本修复；**无需**关闭 LTO 或加 `-fuse-ld=lld`。 +- `GPPDefines.ixx` 被隔离很可能是误判（M3），与 clangd 在无 BMI 时的崩溃（U1）有关；修复 M1 后这类崩溃在实测中消失。 +- BOM 开头的模块接口（`GPPDefines.ixx` 等）目前会被 mcppls 自身误判（M2），同版本修复。 +- 可选：在 VS Code 设置 `"mcppls.trace.server": "verbose"` 后复现一次并附上日志，便于确认 `NormalJsonTranslator.Core.cpp` + 一类的独立崩溃（U2）在其机器上是否同样出现。 diff --git a/.agents/docs/design.md b/.agents/docs/design.md index 6ffc6c4..62e3c2f 100644 --- a/.agents/docs/design.md +++ b/.agents/docs/design.md @@ -115,6 +115,27 @@ has a deadline and a fallback; an unknown stall isolates one file, not the engin by construction, with the kit as fallback for a failing toolchain `std` (P5). Bounded resources (P6). Visible degradation: the status says which modules, how many files, and why (P7). +**What clangd reads is not only what it is given** (0.0.5, `.agents/docs/2026-09-26-issue-23-fix-plan.md`). +clangd reads a file's imports from disk (UP-14 in issue #24), so every save and watched change is +checked: a file whose text on disk would stall clangd (a module name ending in `.`, an import the +database clangd has read has no unit for) is set aside before clangd builds it and handed back when +the disk or the database makes it safe. Every command clangd is given compiles only (`-c`), so no +link-phase check of the driver can fail its module scan (issue #23). Restarts have a budget per +cause (plan, recovery, crash) and are backed off past it, never refused (refining P3); a person's +own restart and a switch of toolchain, profile or context are never counted. A crash sets aside the +file clangd names in its crash context. A project whose build system was found gets clangd only +with the build tool's model (within its bound), never with a provisional one it would have to +unlearn. + +**Observability.** One occurrence must be enough to see why. clangd logs at `info` into a ring in +memory; crashes, stuck or spinning clangd, files set aside, backed-off restarts and a workaround +whose premise is seen broken each write an incident under the workspace's cache (the newest twenty, +for a week): what led up to it, clangd's log, each file's editor-versus-disk lines, their commands, +clangd's recent states per file, and which of its threads used the CPU. Every change of the engine +database is logged with what changed. A diagnostic bundle (`mcppls.exportBundle`, `mcppls report +--bundle`) zips the report, environment, logs, incidents and engine database, with the user's home, +names and secrets replaced, and is not written at all when any is left; it is never uploaded. + **External programs** (build tools, compiler probes, CMake, git) all go through one runner: each run gets its own process unit, soft and hard deadlines end the unit with everything it started, reads are bounded, and every run is recorded (command, environment source, offline or not, duration, @@ -167,6 +188,12 @@ before anything is published (`docs/92-release.md`). | T1 | Tooling: the server is not split internally; devtools depends on no server code and is the one entry for repository work (`mcpp run -p devtools -- ...`) | | T3 | Cache inspection belongs to the server (`mcppls cache`), since only the server knows its cache layout | | T5 | The specification schema check (`docs/specs/tools/validate.py`) is the one script kept, until a C++ JSON Schema 2020-12 validator exists | +| RD1 | A project whose build system was found gets clangd with the build tool's model only, within the producer's bound (fix plan 2026-09-26 D1) | +| RD2 | clangd's upstream defect behind `import a.` (UP-01) is worked around, not fixed in a clangd of our own, until upstream settles (D2) | +| RD3 | clangd logs at `info` into memory; incidents on disk; nothing ever uploaded (D3, F17, F18) | +| RD4 | The space is a completion trigger only after `import `, dropped by the editor elsewhere, and advertised only to clients known to drop it (D4) | +| RD5 | Reports and bundles are redacted by default; a bundle with anything left is not written (F18) | +| RD6 | Restarts are budgeted per cause and backed off past it, never refused; the person's restart is never counted (F14) | ## 7. Known limits @@ -184,6 +211,13 @@ before anything is published (`docs/92-release.md`). outside the database gets such a command, the one clangd interpolates from its nearest unit, so only a stray file with ill-formed code reaches it. The first-diagnostics guard sets it aside after two minutes (import-hang plan §13). +- clangd 23.1 reads an open file's imports from disk (UP-14): an import typed and not saved is not + built until the save (told as information, `WA-CLANGD-007`), and a file whose disk text would stall + clangd is answered by mcppls's own engine until it is saved again (fix plan F16). The upstream fix + — scanning with the editor's buffer — is tracked in issue #24. +- clangd cannot change its log level while it runs; an incident carries its `info` log, and the + `verbose` one needs the server started at `--log-level debug` (in VS Code, the trace setting at + `verbose`), a restart away. - An mcpp project built for Windows through openkal needs `--target x86_64-windows-gnu`, which no editor setting passes to mcpp yet. - openkal cannot lower a child's scheduling priority, so clangd's cold-start module builds compete diff --git a/.agents/skills/mcppls-contributing/SKILL.md b/.agents/skills/mcppls-contributing/SKILL.md index 992852b..a6a47ac 100644 --- a/.agents/skills/mcppls-contributing/SKILL.md +++ b/.agents/skills/mcppls-contributing/SKILL.md @@ -55,6 +55,25 @@ Conformance needs a payload; `CONTRIBUTING.md` has the exact command and how to - **The version lives in `mcpp.toml`** and is written everywhere else by `mcpp run -p devtools -- version --set`, never by hand. +## Upstream defects (clangd, mcpp, …) + +**Issue #24 is the single register of upstream defects** (pinned, English). Its body is only an +index; **each comment is one problem**, `UP-` (clangd/LLVM) or `UP-M` (mcpp): symptom, +affected versions, upstream status, what mcppls does, when that can go, evidence, TODO. Read it +before calling something "a clangd bug", and before working around one. + +- **Found a new one?** Post one comment in that shape (`unfiled` is a status) and add its row to + the index — then decide what mcppls does. +- **Compensating in code?** It is a registered workaround: an entry in + `src/engine/clangd/workarounds.cpp` (`WA-CLANGD-`: upstream, evidence, `removeWhen`, a canary + where one can exist), and its #24 comment names the ID. A limit mcppls deliberately does not work around + goes in `.agents/docs/design.md` §7 and in #24. +- **Filed or fixed upstream?** Put the link in that comment and the index. Bumping the bundled clangd + (`packaging/payload.lock.json`) means walking #24: run the canaries, remove what they say is gone, + update the comments. +- **Not upstream:** a defect in how mcppls drives an upstream tool (for example the arguments it + generates) is an mcppls bug, fixed here; #24 records only the upstream side of it. + ## Commit messages Lowercase `type(scope): a sentence that says what is now true`, not an imperative. The body is diff --git a/.github/versions.env b/.github/versions.env index 072a8f9..8885b51 100644 --- a/.github/versions.env +++ b/.github/versions.env @@ -4,6 +4,6 @@ # # These are build inputs, not the product's version: that one lives in mcpp.toml and is checked # everywhere else by `mcppls-devtools version --check`. -MCPP_VERSION=2026.9.21.3 +MCPP_VERSION=2026.9.26.1 LLVM_VERSION=22.1.8 XLINGS_VERSION=v2026.8.17.2 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7c47220..45e16d8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -277,35 +277,35 @@ jobs: - platform: linux-x64 part: 1 of 2 os: ubuntu-24.04 - fixtures: mcpp-split mcpp-all-cppm verify-changes mingw mcpp-split-gcc mcpp-watch multi-root mcpp-llvm mcpp-watch@polling mcpp-emit s1-two-sets failure-at-base@vscode generated-module-negotiated typing-import typing-import-spin workaround-canaries + fixtures: mcpp-split mcpp-all-cppm verify-changes mingw mcpp-split-gcc mcpp-watch multi-root mcpp-llvm mcpp-watch@polling mcpp-emit s1-two-sets compdb-lto-msvc failure-at-base@vscode generated-module-negotiated typing-import typing-import-spin workaround-canaries typing-autosave compdb-rejected-command compdb-mixed-standards mcpp-emit-partial - platform: linux-x64 part: 2 of 2 os: ubuntu-24.04 extras: true - fixtures: inferred engine-none module-faults untrusted mcpp-gcc mcpp-gcc@plain mcpp-emit-package-std mcpp-emit-broken mcpp-emit-unavailable mcpp-emit-hang mcpp-emit-needs-download mcpp-emit-watch mcpp-emit-watch@polling cmake-clang cmake-clang-bdb watch-polling payload-corrupt clangd-cannot-load generated-module generated-module@vscode generated-module-old-mcpp generated-module-old-mcpp@neovim module-faults@zed failure-at-base failure-at-base@neovim + fixtures: inferred inferred-bom engine-none module-faults untrusted mcpp-gcc mcpp-gcc@plain mcpp-emit-package-std mcpp-emit-broken mcpp-emit-unavailable mcpp-emit-hang mcpp-emit-needs-download mcpp-emit-watch mcpp-emit-watch@polling cmake-clang cmake-clang-bdb watch-polling payload-corrupt clangd-cannot-load generated-module generated-module@vscode generated-module-old-mcpp generated-module-old-mcpp@neovim module-faults@zed failure-at-base failure-at-base@neovim completion-keywords diagnostic-bundle clangd-crash-context mcpp-emit-wait inferred-cxx26 # generated-module{,-old-mcpp,-negotiated}'s mcpp-mock.json bakes in a POSIX driver path # (${env:HOME}/.mcpp/registry/..., no {exe}); it resolves the same way here as on Linux, so # these run on macOS but are left off win32-x64 below rather than fixed unverified. - platform: darwin-arm64 os: macos-14 extras: true - fixtures: inferred engine-none module-faults untrusted mcpp-llvm mcpp-split mcpp-all-cppm verify-changes mcpp-watch multi-root failure-at-base clangd-cannot-load failure-at-base@zed generated-module generated-module-old-mcpp generated-module-negotiated typing-import typing-import-spin workaround-canaries + fixtures: inferred inferred-bom engine-none module-faults untrusted mcpp-llvm mcpp-split mcpp-all-cppm verify-changes mcpp-watch multi-root failure-at-base clangd-cannot-load failure-at-base@zed generated-module generated-module-old-mcpp generated-module-negotiated typing-import typing-import-spin workaround-canaries completion-keywords diagnostic-bundle typing-autosave clangd-crash-context mcpp-emit-wait inferred-cxx26 # No mcpp on the arm64 runner (its tools are the cross-built ones), so the fixtures that # need no build tool and no compiler of their own: the semantic kit, clangd and the server # on aarch64, with module faults, a corrupt payload and polling included. - platform: linux-arm64 os: ubuntu-24.04-arm cross-tools: true - fixtures: inferred engine-none module-faults untrusted payload-corrupt clangd-cannot-load failure-at-base watch-polling typing-import typing-import-spin workaround-canaries + fixtures: inferred inferred-bom engine-none module-faults untrusted payload-corrupt clangd-cannot-load failure-at-base watch-polling typing-import typing-import-spin workaround-canaries completion-keywords diagnostic-bundle typing-autosave clangd-crash-context inferred-cxx26 - platform: win32-x64 part: 1 of 2 os: windows-2022 - fixtures: mcpp-split mcpp-all-cppm verify-changes mcpp-split-msvc cmake-msvc-std compdb-clangxx-msvc-std multi-root compdb-clang-cl-std mcpp-llvm-msvc failure-at-base@vscode typing-import typing-import-spin workaround-canaries + fixtures: mcpp-split mcpp-all-cppm verify-changes mcpp-split-msvc cmake-msvc-std compdb-clangxx-msvc-std multi-root compdb-clang-cl-std mcpp-llvm-msvc failure-at-base@vscode typing-import typing-import-spin workaround-canaries typing-autosave - platform: win32-x64 part: 2 of 2 os: windows-2022 extras: true - fixtures: inferred engine-none module-faults mcpp-emit-hang inferred-msvc untrusted mingw cmake-msvc cmake-msvc-bdb cmake-clangxx-msvc cmake-clang-cl mcpp-msvc mcpp-watch failure-at-base clangd-cannot-load + fixtures: inferred inferred-bom engine-none module-faults mcpp-emit-hang inferred-msvc untrusted mingw cmake-msvc cmake-msvc-bdb cmake-clangxx-msvc cmake-clang-cl mcpp-msvc mcpp-watch failure-at-base clangd-cannot-load completion-keywords diagnostic-bundle mcpp-emit-wait inferred-cxx26 defaults: run: shell: bash @@ -436,7 +436,9 @@ jobs: *@neovim) fixture_extra+=(--client neovim) ;; *@zed) fixture_extra+=(--client zed) ;; esac + # Issue #23 fix plan F18: the diagnostic bundles the bundle checks exported, kept for the artifact below. PATH="$path_prefix$PATH" "stage/mcppls-conformance$exe" run --server "payload/bin/mcppls$exe" --payload payload \ + --keep-bundles "$RUNNER_TEMP/bundles" \ ${fixture_extra[@]+"${fixture_extra[@]}"} --fixture "conformance/fixtures/${fixture%@*}" "$@" } failed='' @@ -455,6 +457,29 @@ jobs: exit 1 fi + # Issue #23 fix plan F18: the bundles diagnostic-bundle exported on this runner's own account, whose + # name is in every path of it (runneradmin and RUNNER~1 on Windows). The fixture already held each + # to its manifest and found no user name and no home directory in it; here another program's zip + # reader opens them and checks every entry's CRC, and they are kept to look at. + - name: The diagnostic bundles open with another program's zip reader + if: always() && contains(matrix.fixtures, 'diagnostic-bundle') + env: + PYTHONUTF8: '1' + run: | + set -euo pipefail + temp="$RUNNER_TEMP"; python=python3 + if [ "$RUNNER_OS" = Windows ]; then temp=$(cygpath -u "$RUNNER_TEMP"); python=python; fi + shopt -s nullglob + bundles=("$temp"/bundles/*.zip) + [ ${#bundles[@]} -gt 0 ] || { echo "::error::diagnostic-bundle left no bundle"; exit 1; } + for bundle in "${bundles[@]}"; do "$python" -m zipfile -t "$bundle"; "$python" -m zipfile -l "$bundle"; done + - uses: actions/upload-artifact@v7 + if: always() && contains(matrix.fixtures, 'diagnostic-bundle') + with: + name: diagnostic-bundle-${{ matrix.platform }} + path: ${{ runner.temp }}/bundles/*.zip + if-no-files-found: ignore + # The review of changes (overall design 7.4, 10.3, work item RV5): every tools/bench/review fixture's change # reviewed by the payload's server; a missing finding or one a fixture rules out fails the job, and # the precision and recall of each rule are printed. diff --git a/CHANGELOG.md b/CHANGELOG.md index dd2e5e9..2f725ba 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,132 @@ release's notes are that section. Versions are three-part semantic versions, `MAJOR.MINOR.PATCH`, and every editor plugin carries the product version unchanged. +## [0.0.5] — 2026-09-26 + +Issue #23 is fixed: modules are built again for projects compiled with LTO for the MSVC ABI. An +autosave of a half-typed `import` no longer stalls clangd, restarts can no longer leave it stuck, a +project mixing C++23 and C++26 reads both, and one command exports everything a problem report needs, +with your name, paths and secrets replaced. +The analysis of #23 with its Windows measurements is `.agents/docs/2026-09-26-issue-23-lto-module-scan.md`; +the plan, its decisions and what was measured is `.agents/docs/2026-09-26-issue-23-fix-plan.md`. +clangd's own defects behind all of this are registered in issue #24. + +### Engine + +- **Modules are built again under LTO for the MSVC ABI (issue #23).** clangd's module scan failed on + every unit with the driver's `LTO requires -fuse-ld=lld`, so no module was built, imports were not + found, and on Windows clangd then crashed until it was given up on. Every command mcppls gives + clangd now compiles only (`-c`), which no link-phase check can fail; `-flto` and the rest of your + command stay as they are. clangd rebuilds its module cache once, the first time a project is opened. +- **An autosave of a half-typed import no longer stalls clangd.** clangd reads a file's imports from + its text on disk, not from the editor, so `import hello.` saved by `files.autoSave` spun a clangd + worker at a full core (it crashed clangd on Windows), past the 0.0.4 workaround, and every request + for the file waited behind it until clangd was restarted; an import of a module that does not exist + yet, saved, deadlocked it the same way. + - Every save and watched change is checked: such a file is answered by mcppls's own engine until it + is saved again, and the status says so as a problem of the code being typed, staying *ready*. + - A build clangd began on it just before is given 1.5 s to finish, else clangd restarts without it. + - A module nothing provides that a save put on disk gets its stand-in within a second. + - Measured on the hello project: 0 CPU for every clangd thread over 20 s with `import hello.` on + disk, where 0.0.4 used a full core. +- **A crash sets aside the file clangd names.** clangd says which file it crashed on; that one is set + aside, instead of whatever was asked about or edited in the last ten seconds (in #23, the wrong + file). The exit code, the file and, on Windows, the exception code are in the report and the status. +- **Restarts are backed off, never refused.** Each reason has its own budget of three restarts in ten + minutes: the database changing, recovering a clangd that stopped answering, and clangd exiting. Past + it, the next one waits 1, 2, 4, then 8 minutes, so a stuck clangd always comes back; in 0.0.4 it + stayed stuck until the window aged out. + - **C++ Modules: Restart clangd** restarts it at once and is never counted. + - Switching the toolchain, the profile or the context is not counted either. +- **Fewer restarts while typing.** A stand-in coming or going no longer restarts clangd, nor does a + changed command of a file clangd does not have, and a restart the database asks for waits two seconds + so the changes that follow share it. The stand-in given to a module clangd could not find is no + longer taken for a new provider, which dropped it again and restarted clangd. A change to a file's + imports is planned once the file has been quiet for two seconds. +- **clangd starts with the build tool's model.** A project whose build system was found no longer + gives clangd a model guessed from its sources while the build tool runs, which clangd then had to + unlearn (in #23, three crashes on it before the real model came). mcppls's own engine answers + module features meanwhile, for up to a minute. +- **A build tool's partial answer is used.** mcpp now describes every workspace member it can plan, + with an `error` naming each one it could not (S2 0.3.0; mcpp-community/mcpp#699). Before, one + member's failure lost the whole database, which is why issue #23's CI read GalTranslPP from scanned + sources. The rest is used, and the status names the missing member by its build description + (`producer-partial`). +- **A command clangd rejects is said so.** When clangd's module scan fails on a command, the status + names the first rejection in the compiler's words, as an environment problem; a missing header, as + the project's. +- A source saved with a UTF-8 byte order mark still declares its module. When no build tool describes + a project, `vcpkg_installed/`, `vcpkg/`, Conan and xmake caches and vendored vcpkg packages are not + scanned, so their modules no longer show up as ambiguous. + +### Diagnostics + +- A directive missing its `;` is reported on the directive, not on the code after it + (`WA-CLANGD-006`). +- An import typed and not saved yet, of a module in the project, is information ("module 'X' is in + the project; clangd loads it once the file is saved"), not a `module not found` error + (`WA-CLANGD-007`). + +### C++26 + +- **A C++26 file can import what a C++23 one built.** A module's BMI is only imported under the + standard it was built with, so in a project mixing C++23 and C++26 the C++26 files lost `import std` + and every module ("C++26 was disabled in precompiled file"). The module units of a context are now + read with one standard, the newest they name; plain units keep their own, and the status profile + names the standard (`standard`, S3). +- **Sources nothing describes are read as C++26** with the semantic kit, GCC 14 and later, and Clang 17 + and later (C++23 before), so C++26 library names such as `std::saturating_add` are there. +- Contracts (P2900) and reflection (P2996) are not in clang 23 and show clang's errors; VS Code colors + `contract_assert`, and `pre` and `post` as contract specifiers. + +### Completion and coloring + +- **A space after `import ` or `export import ` opens the module list** in VS Code; spaces anywhere + else never reach the server. Setting `mcppls.completion.triggerOnSpace` (default on). Other editors + can opt in with `initializationOptions.completion.triggerOnSpace: true`. +- **Module keywords come from mcppls**, merged with clangd's: `import`, `export import`, `module;`, + `export module`, `module` and `module :private;`, each where it can start a declaration, and still + there when clangd is stuck or restarting. Accepting `import` opens the module list. No name is + suggested after `export module `. +- The `export` of every export declaration (`export namespace`, `export {`, `export int f()`) is + colored like the one of `export module`. + +### Reports and diagnostics bundle + +- **C++ Modules: Export Diagnostic Bundle** (`mcppls report --bundle out.zip`, `workspace/executeCommand` + `mcppls.exportBundle`): one zip with the report, the environment, recent server logs, incidents and + the engine database, written locally and never uploaded. + - Your home directory becomes `~`, your user and host names `` and ``, and tokens, + passwords, keys and e-mail addresses ``. + - If anything is left after that, the bundle is not written at all, and the editor offers to retry + with project paths hidden. + - Source files are never included; crash dumps only on request. +- The diagnostic report is redacted the same way by default (`cxxModules/report` `redact`, S3-5.5-3). +- **Incidents.** A crash, a stuck or spinning clangd, a file set aside, a restart held back and a + broken workaround premise each leave a directory under the workspace's cache (`incidents/`, the + newest twenty, for a week). It holds clangd's latest log (clangd now logs at `info`, into memory + only), the files' editor-versus-disk lines and which clangd thread was busy. +- Every change of the engine database is logged with what changed. The status buttons say what they + do (Restart clangd, Export Diagnostic Bundle). + +### Build + +- The macOS server runs on macOS 11 again (it said 14.0), built with mcpp 2026.9.26.1. + +### Known limits + +- clangd's own defects are worked around, not fixed: `import a.` (UP-01, whose upstream fix is + deferred), imports read from disk (UP-14), and a Windows crash on some units with correct commands + (UP-13). The last one is contained: the file clangd names is set aside. Issue #24 tracks each. + +### Testing + +- New conformance fixtures, each failing on 0.0.4: `typing-autosave`, `clangd-crash-context`, + `compdb-rejected-command`, `mcpp-emit-wait`, `compdb-lto-msvc`, `inferred-bom`, `inferred-cxx26`, + `compdb-mixed-standards`, `mcpp-emit-partial`, `completion-keywords` and `diagnostic-bundle`. +- `diagnostic-code` checks take a line, a severity and codes that must be absent; `completion-contains` + takes a trigger character. + ## [0.0.4] — 2026-09-25 Typing an `import` no longer freezes the editor, the status bar says whose problem it is, and diff --git a/conformance/README.md b/conformance/README.md index 5651073..2f0f08d 100644 --- a/conformance/README.md +++ b/conformance/README.md @@ -23,7 +23,7 @@ checks fail at once with that reason instead of each waiting out its timeout. | Fixture | What it covers | |---|---| | `inferred` | Loose module sources, no build system and no compiler: the semantic kit provides libc++ semantics (design 13.5) | -| `engine-none` | The `inferred` project with `--engine none` (overall design 5.6): no core engine, so mcppls's own engine alone answers module navigation, hover, outline, import completion and module diagnostics; the status names `none` as the core engine and lists `mcppls` in `engines` (S3-4-5, S3-4-6). `inferred` checks the same fields with clangd, and starts a second server on the same workspace and cache, which must report the `shared-workspace` notice (design 6.3). Its S5 checks: module descriptions and interface summaries, exported symbols found without clangd, `unavailable` for references, and a file written to disk read by the next query | +| `engine-none` | The `inferred` project with `--engine none` (overall design 5.6): no core engine, so mcppls's own engine alone answers module navigation, hover, outline, import completion, module-syntax keywords and module diagnostics, and a space off an import line is answered with nothing (S3-6.2); the status names `none` as the core engine and lists `mcppls` in `engines` (S3-4-5, S3-4-6). `inferred` checks the same fields with clangd, and starts a second server on the same workspace and cache, which must report the `shared-workspace` notice (design 6.3). Its S5 checks: module descriptions and interface summaries, exported symbols found without clangd, `unavailable` for references, and a file written to disk read by the next query | | `verify-changes` | `mcpp-split`'s project in a git repository, for S5's verification after an edit: snippets checked in place (passing, failing, and leaving no unsaved content behind), a partition interface renamed on disk that breaks the implementation unit using it, the working tree's changes from git, and the restored file passing again once its importer is built again | | `untrusted` | An mcpp package in an untrusted workspace: nothing is executed, the kit answers, the status is `degraded` with the reason | | `mcpp-gcc` | mcpp with GCC 16, described by mcpp's own `emit build-database` (mcpp 2026.9.15.1): level 3 once the S1 library has structured mcpp's level 2 document, GCC arguments translated for clangd (P1), libstdc++'s `std` from its manifest, a test that imports the package's module across sets, and the workspace unchanged | @@ -34,6 +34,7 @@ checks fail at once with that reason instead of each waiting out its timeout. | `mcpp-emit` | mcpp's `emit build-database --format json` (mcpp-community/mcpp#636), simulated by `mcppls-mock-mcpp` from `mcpp-mock.json`, which records what mcpp 2026.9.15.1 prints for the project (paths as `${root}` and `${env:HOME}`): a level 2 document without `ide.options`, one set per package plus `hello:test` and `mcpp:std`, each seeing every other set. The S1 library completes it to a level 3 model; a test imports the package's module across sets, and the workspace stays unchanged. The simulated fixtures cover what a real mcpp cannot be made to do on demand | | `mcpp-emit-package-std` | The same with `std` and `std.compat` provided by translation units of `mcpp:std` from a dependency package instead of by the toolchain's manifest, built in an mcpp std cache directory that does not exist yet | | `mcpp-emit-broken` | The same mcpp answering with an error: the status carries mcpp's own diagnostic, sources are scanned meanwhile, and nothing is configured in its place (the mock's `build` would leave a `compile_commands.json` that `workspace-unchanged` sees) | +| `mcpp-emit-partial` | S2 0.3.0 (S2-3.4-12, S2-3.4-13; mcpp-community/mcpp#699, fix plan 2026-09-26 F10): the producer answers with every member it could plan and an `error` diagnostic whose `path` names the one it could not, exiting 1. The rest is used (the model is mcpp's, not scanned sources) and the status names the missing part (`producer-partial`) | | `mcpp-emit-unavailable` | A project whose `.xlings.json` asks for an mcpp that is not installed: xlings answers every mcpp command in its place and runs nothing, and the status carries xlings's explanation (`mcpp-no-database`) while sources are scanned | | `mcpp-emit-watch` | The inputs mcpp names in `watch` (S2 5, S2-5-1): writing one loads the model again, and an unchanged answer is recognized without rebuilding the index; when mcpp then fails the last model is kept, `degraded` with `model-stale` (S2-5-9), until it answers again. CI also runs it as `mcpp-emit-watch@polling`, with `--no-dynamic-watch` | | `mingw` | A compile database for `x86_64-windows-gnu`: MinGW-w64 GCC semantics through `--sysroot` (P2) | @@ -46,9 +47,13 @@ checks fail at once with that reason instead of each waiting out its timeout. | `cmake-clang-cl` | CMake 4.4 modules built by clang-cl (P6), the first CMake that scans clang-cl module sources | | `compdb-clangxx-msvc-std` | clang++ for the MSVC ABI with `import std`, built by the fixture's own script (P5) | | `compdb-clang-cl-std` | clang-cl with `import std`, built by the fixture's own script (P6) | +| `compdb-lto-msvc` | Issue #23: a `compile_commands.json` of clang++ for the MSVC ABI with `-flto` (the runner's `prepare compdb-lto-msvc`, no configuration file choosing lld). Every command the server gives clangd carries `-c`, so clangd's module scan does not stop at the driver's `LTO requires -fuse-ld=lld`: the module is built, the importer's import resolves and a hover across it answers. The driver raises that for the windows-msvc target on any host, so this runs on Linux | | `mcpp-msvc` | mcpp with `msvc@system` and `import std`, described by mcpp's own `emit build-database`: level 3, a test across sets, the workspace unchanged | | `mcpp-llvm-msvc` | mcpp's default Windows toolchain, LLVM for `x86_64-windows-msvc`, with the MSVC STL (P5), described the same way | | `inferred-msvc` | Loose module sources on a machine with Visual Studio: MSVC STL semantics without a build system (design 9.3, D27) | +| `inferred-bom` | Issue #23: loose module sources saved with a UTF-8 byte order mark. The scanners skip the mark, so `export module greet;` after it declares the module: no false `unresolved-module`, no stand-in, the import navigates to the module and a hover across it answers | +| `inferred-cxx26` | C++26 alignment (fix plan 2026-09-26 §9): sources nothing describes are read with the newest standard their compiler takes, C++26 for the semantic kit; C++26 language features in a module's interface and its importer, `import std` and `std::saturating_add` under C++26, and `standard` in the status profile. 0.0.4 read them as C++23 | +| `compdb-mixed-standards` | C++26 alignment: a `compile_commands.json` (`prepare compdb-mixed-standards`) whose module and a user of std are C++23 and whose application importing both is C++26. A BMI is only imported under the standard it was built with (0.0.4: "C++26 was disabled in precompiled file"); the module units of a context are read with the newest standard they name, and the report says which and how many were raised | | `inferred-no-sdk` | macOS with the Command Line Tools and Xcode hidden: degraded with `sdk-missing` and its install command, a file importing `std` answered at once, module-level features (usable plan W5, U7) | | `inferred-discover` | The `inferred` project with compiler discovery on, on clean machines: a Linux container without a compiler and Windows with Visual Studio hidden (usable plan W5) | | `self-mcpp` | The mcpp repository at a fixed commit, about 170 modules (nightly, W8). Its `.xlings.json` asks for mcpp 2026.9.21.1, which xlings runs inside it | @@ -66,7 +71,13 @@ checks fail at once with that reason instead of each waiting out its timeout. | `clangd-cannot-load` | 0.0.3 plan B1: its `prepare` step puts a stand-in clangd in the workspace (mcppls-mock-mcpp with an `unavailable` config) that writes a loader's message, a `GLIBCXX` version not found, to standard error and exits 1; `--clangd` points the server at it. `initialize` must be answered within 20 s (it used to wait for good), the status must reach `error` with issue `engine-incompatible`, and mcppls's own module features must work | | `typing-import` | Import-hang plan §8: `import hello.greet;` in `main.cpp` and `export module hello.greet;` in its interface typed one key at a time, through `import hello.` and `export module hello.`, which clangd 23.1 never finishes building (WA-CLANGD-001); again with every step saved, as autosave does. Every request is answered within 5 s, the status never turns `degraded`, hover works right after, and nothing restarts clangd or is set aside | | `typing-import-spin` | The same typing with WA-CLANGD-001 turned off (`--disable-workaround`), so clangd really spins (Linux, macOS; on Windows the same text crashes it instead): a spin is found within its 20 s budget (event `engine-spin`), the file set aside with that text remembered and clangd restarted, a crash is restarted as before (`engine-exit`), and either way features come back while typing goes on (import-hang plan §4). If a clangd update removes the defect, its T2 check fails as well | +| `typing-autosave` | Fix plan 2026-09-26 F16, F13, F11, F12 (the hello project with `files.autoSave`): a new import line typed below the others, every step saved to disk, staying on `import hello.` for twelve seconds and then on a module nothing provides. clangd reads a file's imports from disk (UP-14), so 0.0.4 left every request for the file unanswered; the file is now set aside before clangd builds it and handed back once its disk text, or the database clangd read, makes it safe, with an incident recording the broken premise and no restart but for a spin. A directive missing its `;` is reported on the directive (WA-CLANGD-006) and an import only in the unsaved buffer is information (WA-CLANGD-007) | +| `clangd-crash-context` | Fix plan 2026-09-26 F3 (issue #23), Linux and macOS: a stand-in clangd (`prepare clangd-crash-context`) prints a crash context naming a file nobody opened while `main.cpp` is typed in, then dies. Only the named file is set aside, the exit is reported with it and the status names it; 0.0.4 set aside `main.cpp` | +| `compdb-rejected-command` | Fix plan 2026-09-26 F6: a `compile_commands.json` whose commands carry a value the compiler rejects (`-std=c++99999`, `prepare compdb-rejected-command`), as issue #23's did: every module scan fails, and the status says the command was rejected, as an environment issue in the compiler's words | +| `mcpp-emit-wait` | Fix plan 2026-09-26 F4 (D1): the producer hangs past the ten seconds after which scanned sources are planned, to its 20 s bound; mcppls's own engine answers meanwhile and clangd is given no plan until the producer answers or is given up | +| `completion-keywords` | Fix plan 2026-09-26 F9 and F15, as VS Code (`client-info`): the space is a completion trigger character; a space after `import ` or `export import ` opens the module list from mcppls's own index, and one typed anywhere else, or after `import ` or `export module `, is answered at once with nothing. The module-syntax keywords are offered where each can begin a declaration and not inside a function body, merged with clangd's answer, and on their own within 1.5 s for a file clangd cannot answer for yet (a new file waiting for clangd to read a database that has it); the report counts what the space trigger cost (S3-6.2-1 to S3-6.2-5) | | `workaround-canaries` | Import-hang plan §9: one `clangd-check` per registered workaround with a canary, run against the payload's clangd. A failure here means a clangd update fixed that defect and the workaround it names can be removed | +| `diagnostic-bundle` | Issue #23 fix plan F18: the diagnostic bundle an editor exports (`mcppls.exportBundle`) holds what its manifest says, digest for digest, within its 25 MB cap, and no file of it, nor `cxxModules/report`, names the user or the home directory the server runs with, the home written into the client's log it is sent included (S3-5.5-3); asked to hide project paths, not the workspace either. On a CI runner that is `runneradmin`, its `RUNNER~1` and `C:\Users\runneradmin` on Windows, `/home/runner` and `/Users/runner` elsewhere | | `payload-corrupt` | Its `prepare` step copies the payload the runner was given and truncates clangd in the copy (usable plan W9.4); `server-arguments` then points `--payload` at that broken copy, and status must reach `error` with issue `payload-corrupt` | | `multi-root` | Two workspace folders (usable plan W9.1): an `inferred` root and an mcpp-built `mcpp-llvm` root (level 3, from mcpp's own build database), each getting its own project model and clangd, each `cxxModules/status` telling them apart by `project.root` | @@ -125,6 +136,8 @@ scenario names, are relative to the fixture's own root, never to a specific work `initializationOptions` (over whatever `--client` profile set), so a fixture can ask for something `--client` does not, such as `{"semanticTokens": {"moduleType": true}}` (design doc 2026-09-25 K/§7). +`"client-info"` on the scenario is the `clientInfo` the runner sends in `initialize` (none otherwise): +what a server tells VS Code can differ from what it tells other clients (fix plan 2026-09-26 F9). `"initialize-within": SECONDS` on the scenario fails the run when `initialize` is answered later than that (the runner itself waits up to 120 s): a server that answers eventually is not enough where the point is that it answers at once (`clangd-cannot-load`). @@ -156,11 +169,13 @@ always has been. | `responds` | a request (`method`, default `textDocument/definition`) at `at` is answered, empty answers included, within the check's time | | `module-cache-reused` | every file clangd published for `module` (default `std`) before the server started is still there unchanged, and none was added (SC4); passes on a cold start unless `--expect-warm` | | `diagnostics-empty` | the file's diagnostics, after the engine has published them, contain no errors | -| `diagnostic-code` | a diagnostic with code `expect` is published for the file | +| `diagnostic-code` | a diagnostic with code `expect` is published for the file; with `"line"` (0-based) it starts on that line, with `"severity"` it has that severity, and none of the codes in `"absent"` is published with it (fix plan 2026-09-26 F11, F12) | | `definition` / `declaration` | a location ends with `expect` | | `definition-any` | there is at least one location | | `hover-contains` | the hover text contains `expect`, or any one of them when `expect` is a list | -| `completion-contains` | a completion label starts with `expect`; `insert: [line, text]` adds a line first, `edit` changes another open buffer without saving it | +| `completion-contains` | a completion label starts with `expect` (with a list, every one does; with `"exact": true`, a label is it), and none is one of `"absent"`; `insert: [line, text]` adds a line first, `edit` changes another open buffer without saving it; `"trigger"` sends the request as typing that character asked for it (`context.triggerKind` 2) | +| `completion-empty` | the completion (with `"trigger"` as above) has no items, within `"within-ms"` when given (fix plan 2026-09-26 F9: a space off an import line) | +| `capabilities` | the server capabilities `initialize` answered with meet `"expect"` (expectations as for `report`) | | `references-span` | the references include every path in `expect` | | `document-symbol-contains` | the outline has a top-level symbol named `expect` | | `semantic-tokens` | `textDocument/semanticTokens/full` (or `/range`, with `"range"`) for `"file"` (optionally with an unsaved `"text"`), decoded with the legend `initialize` gave, has every entry of `"expect"` (`{"line", "text", "type", "modifiers"?}`; `"modifiers"` is a list, and optional) among its tokens (design doc 2026-09-25 K/§7) | @@ -175,11 +190,12 @@ always has been. | `type-text` | line `line` of `file` takes each of `steps` in turn, `interval-ms` apart (default 120), the whole buffer sent each time; after each, `request` (default `textDocument/documentSymbol`) is answered within `answer-within` seconds (default 5); with `save`, each step is also written to disk and reported as saved and changed, as autosave does; fails when the status turned to a state listed in `states-never` meanwhile (import-hang plan §8) | | `clangd-check` | the runner's own clangd (`--clangd`, else the payload's) run with `--check` on `file` does not finish (`expect: "hangs"`: not finished after `seconds`, default 10, or crashed) or finishes normally (`"finishes"`); a workaround's canary expects its defect, and fails with `says` once a clangd update fixed it (import-hang plan §9) | | `report` | robustness design O3: `cxxModules/report` meets `"expect"`, retried within the check's time like an `mcp`/`cli` result (a plan or an engine may still be on its way) | +| `bundle` | issue #23 fix plan F18: `workspace/executeCommand` `mcppls.exportBundle` with `"arguments"` (and a client log naming the home directory) answers with the path of a zip under 25 MB whose `manifest.json` lists every other file with its SHA-256, which has every file of `"expect-files"`, and in which no file names the server's home directory (either separator), a distinctive user name (`USER`, `USERNAME`, `LOGNAME`) or its 8.3 form, or, with `"forbid-workspace": true`, the workspace; neither may `cxxModules/report` (the workspace aside) | An expectation of `mcp`, `cli` and `report` names a JSON pointer in `"path"`, where a `*` segment stands for every element of an array, and one of `"equals"` (a value the pointer names equals it), `"contains"` (a string -contains it, or an array has an element that includes all its members), `"min-items"`, `"max-items"`, `"exists"` or -`"absent"`; it holds when any value the pointer names satisfies it. `"each-contains"` is the one that +contains it, or an array has an element that includes all its members), `"min-items"`, `"max-items"`, `"at-least"` +(a number at least it), `"exists"` or `"absent"`; it holds when any value the pointer names satisfies it. `"each-contains"` is the one that every value the pointer names must satisfy instead (each is a string containing it), and it holds when the pointer names none. diff --git a/conformance/fixtures/clangd-crash-context/scenario.json b/conformance/fixtures/clangd-crash-context/scenario.json new file mode 100644 index 0000000..8b48c10 --- /dev/null +++ b/conformance/fixtures/clangd-crash-context/scenario.json @@ -0,0 +1,135 @@ +{ + "name": "clangd-crash-context", + "description": "Fix plan 2026-09-26 F3 (issue #23): clangd names the file it crashed on. Its `prepare` step puts a stand-in clangd in the workspace that runs the payload's clangd and, 25 seconds in, prints a crash context naming src/crasher.cpp -- a file nobody opened -- the way clangd 23.1 did on Windows, and kills it; later starts are the real clangd. Meanwhile the editor types in src/main.cpp. 0.0.4 set aside what it had been given or asked about in the last ten seconds, main.cpp, and not the file clangd named; now only src/crasher.cpp is set aside, the exit is reported with its crash context, and main.cpp keeps clangd once it is back.", + "prepare": [ + [ + "{conformance}", + "prepare", + "clangd-crash-context", + "{payload}" + ] + ], + "server-arguments": [ + "--clangd", + "{workspace}/stand-in/clangd", + "--no-discover" + ], + "checks": [ + { + "id": "S1", + "kind": "status", + "state": "ready", + "engine-name": "clangd" + }, + { + "id": "T1-editing-when-it-crashes", + "kind": "type-text", + "file": "src/main.cpp", + "line": 2, + "steps": [ + "i", + "in", + "int", + "int x", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "" + ], + "interval-ms": 500, + "answer-within": 15 + }, + { + "id": "R1-the-file-it-named", + "kind": "report", + "timeout": 30, + "expect": [ + { + "path": "/roots/0/engines/*/details/lastExit/crashFile", + "contains": "crasher.cpp" + }, + { + "path": "/roots/0/engines/*/details/lastExit/crashAction", + "equals": "Build AST" + }, + { + "path": "/roots/0/engines/*/details/filesSetAside", + "min-items": 1 + }, + { + "path": "/roots/0/engines/*/details/filesSetAside/*", + "each-contains": "crasher.cpp" + }, + { + "path": "/roots/0/events/*/kind", + "equals": "engine-exit" + } + ] + }, + { + "id": "T2-main-keeps-clangd", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 4, + 31 + ], + "expect": "greet", + "timeout": 90 + }, + { + "id": "R2-status-names-it", + "kind": "status", + "issue-code": "engine-crashed", + "issue-message": "crasher.cpp" + } + ] +} diff --git a/conformance/fixtures/clangd-crash-context/src/crasher.cpp b/conformance/fixtures/clangd-crash-context/src/crasher.cpp new file mode 100644 index 0000000..4e702de --- /dev/null +++ b/conformance/fixtures/clangd-crash-context/src/crasher.cpp @@ -0,0 +1,4 @@ +import std; +import hello.greet; + +int crash_here() { return 1; } diff --git a/conformance/fixtures/clangd-crash-context/src/greet/detail.cppm b/conformance/fixtures/clangd-crash-context/src/greet/detail.cppm new file mode 100644 index 0000000..56cb780 --- /dev/null +++ b/conformance/fixtures/clangd-crash-context/src/greet/detail.cppm @@ -0,0 +1,5 @@ +export module hello.greet:detail; +import std; +export namespace hello::detail { + std::string prefix() { return "Hello, "; } +} diff --git a/conformance/fixtures/clangd-crash-context/src/greet/greet.cppm b/conformance/fixtures/clangd-crash-context/src/greet/greet.cppm new file mode 100644 index 0000000..21a6536 --- /dev/null +++ b/conformance/fixtures/clangd-crash-context/src/greet/greet.cppm @@ -0,0 +1,7 @@ +export module hello.greet; +export import :detail; +import std; + +export namespace hello { + std::string greet(std::string_view who) { return detail::prefix() + std::string(who); } +} diff --git a/conformance/fixtures/clangd-crash-context/src/main.cpp b/conformance/fixtures/clangd-crash-context/src/main.cpp new file mode 100644 index 0000000..71169db --- /dev/null +++ b/conformance/fixtures/clangd-crash-context/src/main.cpp @@ -0,0 +1,7 @@ +import std; +import hello.greet; + +int main(int argc, char* argv[]) { + std::println("{}", hello::greet("mcpp")); + return 0; +} diff --git a/conformance/fixtures/compdb-lto-msvc/scenario.json b/conformance/fixtures/compdb-lto-msvc/scenario.json new file mode 100644 index 0000000..cbe70e6 --- /dev/null +++ b/conformance/fixtures/compdb-lto-msvc/scenario.json @@ -0,0 +1,40 @@ +{ + "name": "compdb-lto-msvc", + "description": "Issue #23, fix plan F1: a compile_commands.json of a project built with LTO for the MSVC ABI (clang++ --target=x86_64-pc-windows-msvc -flto -c, written by the runner's `prepare compdb-lto-msvc`). Every command the server gives clangd must carry -c: without it clangd's module dependency scan fails on the driver's `LTO requires -fuse-ld=lld`, no module is built, and the importer reports its import as not found and hovers across it answer nothing. The driver raises that for the windows-msvc target on any host, so this runs on Linux.", + "prepare": [ + [ + "{conformance}", + "prepare", + "compdb-lto-msvc", + "{env:CONFORMANCE_CLANGXX|clang++}" + ] + ], + "checks": [ + { + "id": "S1", + "kind": "status", + "source": "compile-commands", + "profile-kind": "build-toolchain" + }, + { + "id": "L1-the-module-is-built", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 3, + 12 + ], + "expect": "answer_of_everything", + "timeout": 60 + }, + { + "id": "L2-the-import-resolves", + "kind": "diagnostics-empty", + "file": "src/main.cpp" + }, + { + "id": "W1", + "kind": "workspace-unchanged" + } + ] +} diff --git a/conformance/fixtures/compdb-lto-msvc/src/answer.cppm b/conformance/fixtures/compdb-lto-msvc/src/answer.cppm new file mode 100644 index 0000000..2a393fb --- /dev/null +++ b/conformance/fixtures/compdb-lto-msvc/src/answer.cppm @@ -0,0 +1,3 @@ +export module answer; + +export int answer_of_everything() { return 42; } diff --git a/conformance/fixtures/compdb-lto-msvc/src/main.cpp b/conformance/fixtures/compdb-lto-msvc/src/main.cpp new file mode 100644 index 0000000..7eb63a7 --- /dev/null +++ b/conformance/fixtures/compdb-lto-msvc/src/main.cpp @@ -0,0 +1,5 @@ +import answer; + +int main() { + return answer_of_everything() == 42 ? 0 : 1; +} diff --git a/conformance/fixtures/compdb-mixed-standards/scenario.json b/conformance/fixtures/compdb-mixed-standards/scenario.json new file mode 100644 index 0000000..70e2b77 --- /dev/null +++ b/conformance/fixtures/compdb-mixed-standards/scenario.json @@ -0,0 +1,17 @@ +{ + "name": "compdb-mixed-standards", + "description": "C++26 alignment (fix plan 2026-09-26 §9): a compile_commands.json (the runner's `prepare compdb-mixed-standards`) whose module `core` and a user of std are C++23 while the application importing both is C++26. A BMI can only be imported under the standard it was built with, so 0.0.4 built std as C++23 and the C++26 application lost everything that came through it (\"C++26 was disabled in precompiled file\"). The module units of a context are now read with one standard, the newest they name, and the report says which and how many were raised.", + "prepare": [ + ["{conformance}", "prepare", "compdb-mixed-standards", "{env:CONFORMANCE_CLANGXX|clang++}"] + ], + "checks": [ + { "id": "S1", "kind": "status", "source": "compile-commands" }, + { "id": "D1-the-cxx26-application", "kind": "diagnostics-empty", "file": "src/app.cpp", "timeout": 90 }, + { "id": "D2-the-cxx23-module", "kind": "diagnostics-empty", "file": "src/core.cppm", "timeout": 60 }, + { "id": "D3-the-cxx23-user-of-std", "kind": "diagnostics-empty", "file": "src/legacy.cpp", "timeout": 60 }, + { "id": "R1-one-standard", "kind": "report", "expect": [ + { "path": "/roots/0/plan/languageStandard", "equals": "c++26" }, + { "path": "/roots/0/plan/standardsRaised", "at-least": 1 }, + { "path": "/roots/0/project/profile/standard", "equals": "c++26" } ] } + ] +} diff --git a/conformance/fixtures/compdb-mixed-standards/src/app.cpp b/conformance/fixtures/compdb-mixed-standards/src/app.cpp new file mode 100644 index 0000000..922d65b --- /dev/null +++ b/conformance/fixtures/compdb-mixed-standards/src/app.cpp @@ -0,0 +1,8 @@ +import std; +import core; + +int main() { + auto [n, _] = std::pair { core_name().size(), 0 }; + std::println("{}", n + 1); + return 0; +} diff --git a/conformance/fixtures/compdb-mixed-standards/src/core.cppm b/conformance/fixtures/compdb-mixed-standards/src/core.cppm new file mode 100644 index 0000000..0c8d0d8 --- /dev/null +++ b/conformance/fixtures/compdb-mixed-standards/src/core.cppm @@ -0,0 +1,4 @@ +export module core; +import std; + +export std::string core_name() { return "core"; } diff --git a/conformance/fixtures/compdb-mixed-standards/src/legacy.cpp b/conformance/fixtures/compdb-mixed-standards/src/legacy.cpp new file mode 100644 index 0000000..be79390 --- /dev/null +++ b/conformance/fixtures/compdb-mixed-standards/src/legacy.cpp @@ -0,0 +1,3 @@ +import std; + +int legacy() { return static_cast(std::string("legacy").size()); } diff --git a/conformance/fixtures/compdb-rejected-command/scenario.json b/conformance/fixtures/compdb-rejected-command/scenario.json new file mode 100644 index 0000000..8a0bace --- /dev/null +++ b/conformance/fixtures/compdb-rejected-command/scenario.json @@ -0,0 +1,47 @@ +{ + "name": "compdb-rejected-command", + "description": "Fix plan 2026-09-26 F6: a compile_commands.json whose commands carry an option value the compiler rejects (`-std=c++99999`) (written by the runner's `prepare compdb-rejected-command`), the kind of command issue #23's was. clangd's module scan fails on every unit; the status now says so as an environment issue, in the driver's own words, where 0.0.4 said nothing and the reason was one line among thousands of the log.", + "prepare": [ + [ + "{conformance}", + "prepare", + "compdb-rejected-command", + "{env:CONFORMANCE_CLANGXX|clang++}" + ] + ], + "checks": [ + { + "id": "S1", + "kind": "status", + "source": "compile-commands" + }, + { + "id": "O1-open", + "kind": "open", + "file": "src/main.cpp" + }, + { + "id": "F1-in-the-report", + "kind": "report", + "timeout": 30, + "expect": [ + { + "path": "/roots/0/engines/*/details/scanFailures/count", + "at-least": 1 + }, + { + "path": "/roots/0/engines/*/details/scanFailures/firstReason", + "contains": "c++99999" + } + ] + }, + { + "id": "F2-the-status-says-why", + "kind": "status", + "issue-code": "module-scan-failed", + "issue-category": "environment", + "issue-message": "rejected the compile command", + "timeout": 60 + } + ] +} diff --git a/conformance/fixtures/compdb-rejected-command/src/answer.cppm b/conformance/fixtures/compdb-rejected-command/src/answer.cppm new file mode 100644 index 0000000..2a393fb --- /dev/null +++ b/conformance/fixtures/compdb-rejected-command/src/answer.cppm @@ -0,0 +1,3 @@ +export module answer; + +export int answer_of_everything() { return 42; } diff --git a/conformance/fixtures/compdb-rejected-command/src/main.cpp b/conformance/fixtures/compdb-rejected-command/src/main.cpp new file mode 100644 index 0000000..7eb63a7 --- /dev/null +++ b/conformance/fixtures/compdb-rejected-command/src/main.cpp @@ -0,0 +1,5 @@ +import answer; + +int main() { + return answer_of_everything() == 42 ? 0 : 1; +} diff --git a/conformance/fixtures/completion-keywords/scenario.json b/conformance/fixtures/completion-keywords/scenario.json new file mode 100644 index 0000000..8fcb1e1 --- /dev/null +++ b/conformance/fixtures/completion-keywords/scenario.json @@ -0,0 +1,226 @@ +{ + "name": "completion-keywords", + "description": "Fix plan 2026-09-26 F9 and F15 (decisions D4, D5), as VS Code: a space is a trigger character; a space typed after `import` opens the module list from mcppls's own index, and one typed anywhere else is answered at once with nothing, never reaching clangd. The module-syntax keywords (`import`, `export import`, `module;`, `export module`, `module`, `module :private;`) come from mcppls where each can begin a declaration, merged with clangd's answer, and on their own when clangd cannot answer for the file yet: a file just opened waits for clangd to read a database that has it, and its keywords do not.", + "server-arguments": [ + "--no-discover" + ], + "client-info": { + "name": "Visual Studio Code", + "version": "1.105.0" + }, + "checks": [ + { + "id": "S1", + "kind": "status", + "source": "inferred", + "state": "ready", + "engine-name": "clangd" + }, + { + "id": "C0-space-is-a-trigger", + "kind": "capabilities", + "expect": [ + { + "path": "/completionProvider/triggerCharacters", + "contains": " " + }, + { + "path": "/completionProvider/triggerCharacters", + "contains": "." + } + ] + }, + { + "id": "T0-clangd-answers", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 4, + 31 + ], + "expect": "greet", + "timeout": 90 + }, + { + "id": "K5-not-in-a-body", + "kind": "completion-contains", + "file": "src/main.cpp", + "insert": [ + 4, + " i" + ], + "at": [ + 4, + 5 + ], + "expect": [ + "int" + ], + "absent": [ + "import", + "export module", + "module;" + ] + }, + { + "id": "K1-import", + "kind": "completion-contains", + "file": "src/main.cpp", + "insert": [ + 2, + "i" + ], + "at": [ + 2, + 1 + ], + "expect": [ + "import", + "int" + ], + "absent": [ + "export import", + "module;" + ], + "exact": true + }, + { + "id": "K2-module-declarations", + "kind": "completion-contains", + "file": "src/scratch.cppm", + "text": "// a new unit\ne", + "at": [ + 1, + 1 + ], + "expect": [ + "export module" + ], + "absent": [ + "export import" + ], + "exact": true + }, + { + "id": "K3-export-import", + "kind": "completion-contains", + "file": "src/iface.cppm", + "text": "export module scratch.iface;\nimport std;\nexport i", + "at": [ + 2, + 8 + ], + "expect": [ + "export import" + ], + "absent": [ + "export module" + ], + "exact": true + }, + { + "id": "K4-private-fragment", + "kind": "completion-contains", + "file": "src/iface.cppm", + "text": "export module scratch.iface;\nimport std;\nexport int f();\nmod", + "at": [ + 3, + 3 + ], + "expect": [ + "module :private;" + ], + "absent": [ + "module;", + "export module" + ], + "exact": true + }, + { + "id": "P1-space-elsewhere", + "kind": "completion-empty", + "file": "src/space.cpp", + "text": "int x = ", + "at": [ + 0, + 8 + ], + "trigger": " ", + "within-ms": 2000 + }, + { + "id": "P2-space-after-import", + "kind": "completion-contains", + "file": "src/space.cpp", + "text": "import std;\nimport ", + "at": [ + 1, + 7 + ], + "trigger": " ", + "expect": [ + "hello.greet" + ] + }, + { + "id": "P3-space-after-export-import", + "kind": "completion-contains", + "file": "src/space.cppm", + "text": "export module space;\nexport import ", + "at": [ + 1, + 14 + ], + "trigger": " ", + "expect": [ + "hello.greet" + ] + }, + { + "id": "P4-two-spaces", + "kind": "completion-empty", + "file": "src/space.cpp", + "text": "import ", + "at": [ + 0, + 8 + ], + "trigger": " ", + "within-ms": 2000 + }, + { + "id": "P5-export-module-names-nothing", + "kind": "completion-empty", + "file": "src/space.cppm", + "text": "export module ", + "at": [ + 0, + 14 + ], + "trigger": " ", + "within-ms": 2000 + }, + { + "id": "R1-costs-counted", + "kind": "report", + "expect": [ + { + "path": "/roots/0/completion/spaceTrigger/count", + "at-least": 5 + }, + { + "path": "/roots/0/completion/spaceTrigger/passed", + "at-least": 2 + }, + { + "path": "/roots/0/completion/spaceTrigger/maxMicros", + "exists": true + }, + { + "path": "/roots/0/completion/keywordsWithoutEngine", + "at-least": 1 + } + ] + } + ] +} diff --git a/conformance/fixtures/completion-keywords/src/greet/detail.cppm b/conformance/fixtures/completion-keywords/src/greet/detail.cppm new file mode 100644 index 0000000..56cb780 --- /dev/null +++ b/conformance/fixtures/completion-keywords/src/greet/detail.cppm @@ -0,0 +1,5 @@ +export module hello.greet:detail; +import std; +export namespace hello::detail { + std::string prefix() { return "Hello, "; } +} diff --git a/conformance/fixtures/completion-keywords/src/greet/greet.cppm b/conformance/fixtures/completion-keywords/src/greet/greet.cppm new file mode 100644 index 0000000..21a6536 --- /dev/null +++ b/conformance/fixtures/completion-keywords/src/greet/greet.cppm @@ -0,0 +1,7 @@ +export module hello.greet; +export import :detail; +import std; + +export namespace hello { + std::string greet(std::string_view who) { return detail::prefix() + std::string(who); } +} diff --git a/conformance/fixtures/completion-keywords/src/main.cpp b/conformance/fixtures/completion-keywords/src/main.cpp new file mode 100644 index 0000000..71169db --- /dev/null +++ b/conformance/fixtures/completion-keywords/src/main.cpp @@ -0,0 +1,7 @@ +import std; +import hello.greet; + +int main(int argc, char* argv[]) { + std::println("{}", hello::greet("mcpp")); + return 0; +} diff --git a/conformance/fixtures/diagnostic-bundle/scenario.json b/conformance/fixtures/diagnostic-bundle/scenario.json new file mode 100644 index 0000000..146b559 --- /dev/null +++ b/conformance/fixtures/diagnostic-bundle/scenario.json @@ -0,0 +1,16 @@ +{ + "name": "diagnostic-bundle", + "description": "Issue #23 fix plan F18: the diagnostic bundle an editor exports holds what its manifest says, within its size cap, and neither it nor the report names the user or the home directory; with project paths hidden, not the workspace either.", + "server-arguments": ["--no-discover"], + "checks": [ + { "id": "S1", "kind": "status", "source": "inferred", "profile-kind": "semantic-kit", "state": "ready" }, + { "id": "C2", "kind": "definition", "file": "src/main.cpp", "at": [4, 31], "expect": "src/greet/greet.cppm" }, + { "id": "B1-bundle", "kind": "bundle", "timeout": 60, + "expect-files": ["manifest.json", "report.json", "environment.json", "logs/client.log", "engine/root-1/plan.json", "engine/root-1/compile_commands.json"] }, + { "id": "B2-bundle-project-paths-hidden", "kind": "bundle", "timeout": 60, "forbid-workspace": true, + "arguments": { "hideProjectPaths": true, "noSourceExcerpts": true }, + "expect-files": ["manifest.json", "report.json", "environment.json"] }, + { "id": "B3-bundle-from-the-command-line", "kind": "bundle", "via": "cli", "timeout": 120, + "expect-files": ["manifest.json", "report.json", "environment.json", "engine/root-1/compile_commands.json"] } + ] +} diff --git a/conformance/fixtures/diagnostic-bundle/src/greet/detail.cppm b/conformance/fixtures/diagnostic-bundle/src/greet/detail.cppm new file mode 100644 index 0000000..56cb780 --- /dev/null +++ b/conformance/fixtures/diagnostic-bundle/src/greet/detail.cppm @@ -0,0 +1,5 @@ +export module hello.greet:detail; +import std; +export namespace hello::detail { + std::string prefix() { return "Hello, "; } +} diff --git a/conformance/fixtures/diagnostic-bundle/src/greet/greet.cppm b/conformance/fixtures/diagnostic-bundle/src/greet/greet.cppm new file mode 100644 index 0000000..21a6536 --- /dev/null +++ b/conformance/fixtures/diagnostic-bundle/src/greet/greet.cppm @@ -0,0 +1,7 @@ +export module hello.greet; +export import :detail; +import std; + +export namespace hello { + std::string greet(std::string_view who) { return detail::prefix() + std::string(who); } +} diff --git a/conformance/fixtures/diagnostic-bundle/src/main.cpp b/conformance/fixtures/diagnostic-bundle/src/main.cpp new file mode 100644 index 0000000..71169db --- /dev/null +++ b/conformance/fixtures/diagnostic-bundle/src/main.cpp @@ -0,0 +1,7 @@ +import std; +import hello.greet; + +int main(int argc, char* argv[]) { + std::println("{}", hello::greet("mcpp")); + return 0; +} diff --git a/conformance/fixtures/engine-none/scenario.json b/conformance/fixtures/engine-none/scenario.json index bdccff0..f6312f2 100644 --- a/conformance/fixtures/engine-none/scenario.json +++ b/conformance/fixtures/engine-none/scenario.json @@ -9,6 +9,10 @@ { "id": "M2-hover", "kind": "hover-contains", "file": "src/main.cpp", "at": [1, 9], "expect": "module hello.greet" }, { "id": "M3-outline", "kind": "document-symbol-contains", "file": "src/greet/greet.cppm", "expect": "hello.greet" }, { "id": "M4-import-completion", "kind": "completion-contains", "file": "src/scratch.cpp", "text": "import hel", "at": [0, 10], "expect": "hello.greet" }, + { "id": "M4-keywords", "kind": "completion-contains", "file": "src/keywords.cpp", "text": "i", "at": [0, 1], "expect": ["import"], "exact": true }, + { "id": "M4-export-import", "kind": "completion-contains", "file": "src/keywords.cppm", "text": "export module keywords;\nexport i", "at": [1, 8], "expect": ["export import"], "exact": true }, + { "id": "M4-space-elsewhere", "kind": "completion-empty", "file": "src/keywords.cpp", "text": "int x = ", "at": [0, 8], "trigger": " ", "within-ms": 2000 }, + { "id": "M4-space-after-import", "kind": "completion-contains", "file": "src/keywords.cpp", "text": "import ", "at": [0, 7], "trigger": " ", "expect": "hello.greet" }, { "id": "M5-unresolved", "kind": "diagnostic-code", "file": "src/unresolved.cpp", "text": "import missing.module;\nint f();\n", "expect": "unresolved-module" }, { "id": "M6-graph", "kind": "module-graph-contains", "expect": "hello.greet:detail" }, { "id": "Q4-module", "kind": "mcp", "tool": "cxx_module", "arguments": { "name": "hello.greet" }, diff --git a/conformance/fixtures/inferred-bom/scenario.json b/conformance/fixtures/inferred-bom/scenario.json new file mode 100644 index 0000000..2d2bafe --- /dev/null +++ b/conformance/fixtures/inferred-bom/scenario.json @@ -0,0 +1,60 @@ +{ + "name": "inferred-bom", + "description": "Issue #23, fix plan F2: loose module sources saved with a UTF-8 byte order mark, as editors on Windows can save them. The scanners skip the mark: `export module greet;` after it is the module's declaration, so mcppls reports no false `unresolved-module`, plans no stand-in for greet, and clangd builds the real module.", + "server-arguments": [ + "--no-discover" + ], + "checks": [ + { + "id": "S1", + "kind": "status", + "source": "inferred", + "profile-kind": "semantic-kit", + "state": "ready" + }, + { + "id": "B1-no-false-unresolved-module", + "kind": "diagnostics-empty", + "file": "src/main.cpp" + }, + { + "id": "B2-import-goes-to-the-module", + "kind": "definition", + "file": "src/main.cpp", + "at": [ + 1, + 9 + ], + "expect": "src/greet.cppm", + "timeout": 30 + }, + { + "id": "B3-hover-across-the-module", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 4, + 14 + ], + "expect": "greet_answer", + "timeout": 60 + }, + { + "id": "B4-graph", + "kind": "module-graph-contains", + "expect": "greet", + "timeout": 30 + }, + { + "id": "B5-outline", + "kind": "document-symbol-contains", + "file": "src/greet.cppm", + "expect": "greet", + "timeout": 30 + }, + { + "id": "W1", + "kind": "workspace-unchanged" + } + ] +} diff --git a/conformance/fixtures/inferred-bom/src/greet.cppm b/conformance/fixtures/inferred-bom/src/greet.cppm new file mode 100644 index 0000000..c66030a --- /dev/null +++ b/conformance/fixtures/inferred-bom/src/greet.cppm @@ -0,0 +1,3 @@ +export module greet; + +export int greet_answer() { return 42; } diff --git a/conformance/fixtures/inferred-bom/src/main.cpp b/conformance/fixtures/inferred-bom/src/main.cpp new file mode 100644 index 0000000..b055abb --- /dev/null +++ b/conformance/fixtures/inferred-bom/src/main.cpp @@ -0,0 +1,6 @@ +// Saved with a byte order mark, the way editors on Windows can save a file. +import greet; + +int main() { + return greet_answer() == 42 ? 0 : 1; +} diff --git a/conformance/fixtures/inferred-cxx26/scenario.json b/conformance/fixtures/inferred-cxx26/scenario.json new file mode 100644 index 0000000..2ba9794 --- /dev/null +++ b/conformance/fixtures/inferred-cxx26/scenario.json @@ -0,0 +1,52 @@ +{ + "name": "inferred-cxx26", + "description": "C++26 alignment (fix plan 2026-09-26 §9): sources nothing describes are read with the newest standard the compiler reading them takes, C++26 for the semantic kit (clang 23, libc++ 23), where 0.0.4 read them as C++23 and C++26 library names were missing (`std::saturating_add`). C++26 language features in a module's interface and in its importer, `import std` under C++26, and the standard in the status. (Contracts, P2900, are not in clang 23: `contract_assert` is colored by the VS Code grammar only, and code using contracts shows clang's errors.)", + "server-arguments": [ + "--no-discover" + ], + "checks": [ + { + "id": "S1", + "kind": "status", + "source": "inferred", + "state": "ready" + }, + { + "id": "D1-interface-under-cxx26", + "kind": "diagnostics-empty", + "file": "src/sat.cppm", + "timeout": 90 + }, + { + "id": "D2-importer-under-cxx26", + "kind": "diagnostics-empty", + "file": "src/main.cpp", + "timeout": 90 + }, + { + "id": "H1-across-the-module", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 8, + 32 + ], + "expect": "clamp_add", + "timeout": 60 + }, + { + "id": "R1-the-standard-is-said", + "kind": "report", + "expect": [ + { + "path": "/roots/0/project/profile/standard", + "equals": "c++26" + }, + { + "path": "/roots/0/plan/languageStandard", + "equals": "c++26" + } + ] + } + ] +} diff --git a/conformance/fixtures/inferred-cxx26/src/main.cpp b/conformance/fixtures/inferred-cxx26/src/main.cpp new file mode 100644 index 0000000..4e8df09 --- /dev/null +++ b/conformance/fixtures/inferred-cxx26/src/main.cpp @@ -0,0 +1,12 @@ +import std; +import sat; + +void old_api() = delete("use clamp_add"); // C++26: = delete("reason") (P2573) + +int main() { + auto [a, _] = std::pair { 1, 2 }; // C++26: placeholder variables (P2169) + auto [b, _] = std::pair { 3, 4 }; + first_of x = clamp_add(a, b); + std::println("{}", x); + return 0; +} diff --git a/conformance/fixtures/inferred-cxx26/src/sat.cppm b/conformance/fixtures/inferred-cxx26/src/sat.cppm new file mode 100644 index 0000000..4566566 --- /dev/null +++ b/conformance/fixtures/inferred-cxx26/src/sat.cppm @@ -0,0 +1,8 @@ +export module sat; +import std; + +// C++26: pack indexing (P2662) in a module's interface. +export template using first_of = Ts...[0]; + +// C++26: saturation arithmetic (P0543), under its C++26 name. +export inline int clamp_add(int a, int b) { return std::saturating_add(a, b); } diff --git a/conformance/fixtures/inferred/scenario.json b/conformance/fixtures/inferred/scenario.json index b12a6f0..67853c0 100644 --- a/conformance/fixtures/inferred/scenario.json +++ b/conformance/fixtures/inferred/scenario.json @@ -29,7 +29,8 @@ { "line": 1, "text": "import", "type": "keyword" }, { "line": 1, "text": "detail", "type": "module", "modifiers": ["partition"] }, { "line": 2, "text": "import", "type": "keyword" }, - { "line": 2, "text": "std", "type": "module" } + { "line": 2, "text": "std", "type": "module" }, + { "line": 4, "text": "export", "type": "keyword" } ] } ] } diff --git a/conformance/fixtures/mcpp-emit-partial/mcpp-mock.json b/conformance/fixtures/mcpp-emit-partial/mcpp-mock.json new file mode 100644 index 0000000..a1087d7 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-partial/mcpp-mock.json @@ -0,0 +1,348 @@ +{ + "database": { + "ide": { + "generator": { + "name": "mcpp", + "version": "2026.9.15.1" + }, + "profile-version": "0.2.0", + "toolchains": { + "llvm-22.1.8-x86_64-unknown-linux-gnu": { + "config-files": [], + "driver": "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/clang++", + "family": "clang", + "stdlib": { + "name": "libc++", + "version": "22.1.8" + }, + "target": "x86_64-unknown-linux-gnu", + "version": "22.1.8" + } + } + }, + "revision": 0, + "sets": [ + { + "baseline-arguments": [ + "-std=c++23", + "-fmodule-file=std=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.pcm", + "-fmodule-file=std.compat=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.compat.pcm", + "-fprebuilt-module-path=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache", + "-O0", + "-g", + "--no-default-config", + "-nostdinc++", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/x86_64-unknown-linux-gnu/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/include", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-linux-headers/5.11.1/include" + ], + "family-name": "hello", + "ide": { + "configuration": "dev", + "kind": "executable", + "toolchain": "llvm-22.1.8-x86_64-unknown-linux-gnu" + }, + "name": "hello", + "translation-units": [ + { + "arguments": [ + "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/clang++", + "-std=c++23", + "-fmodule-file=std=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.pcm", + "-fmodule-file=std.compat=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.compat.pcm", + "-fprebuilt-module-path=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache", + "-O0", + "-g", + "--no-default-config", + "-nostdinc++", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/x86_64-unknown-linux-gnu/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/include", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-linux-headers/5.11.1/include", + "-c", + "${root}/src/greet/detail.cppm", + "-o", + "${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/obj/detail.m.o" + ], + "ide": { + "role": "module-partition-interface" + }, + "local-arguments": [], + "object": "${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/obj/detail.m.o", + "private": false, + "provides": { + "hello.greet:detail": "" + }, + "requires": [ + "std" + ], + "source": "${root}/src/greet/detail.cppm", + "work-directory": "${root}" + }, + { + "arguments": [ + "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/clang++", + "-std=c++23", + "-fmodule-file=std=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.pcm", + "-fmodule-file=std.compat=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.compat.pcm", + "-fprebuilt-module-path=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache", + "-O0", + "-g", + "--no-default-config", + "-nostdinc++", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/x86_64-unknown-linux-gnu/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/include", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-linux-headers/5.11.1/include", + "-c", + "${root}/src/greet/greet.cppm", + "-o", + "${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/obj/greet.m.o" + ], + "ide": { + "role": "module-interface" + }, + "local-arguments": [], + "object": "${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/obj/greet.m.o", + "private": false, + "provides": { + "hello.greet": "" + }, + "requires": [ + "hello.greet:detail", + "std" + ], + "source": "${root}/src/greet/greet.cppm", + "work-directory": "${root}" + }, + { + "arguments": [ + "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/clang++", + "-std=c++23", + "-fmodule-file=std=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.pcm", + "-fmodule-file=std.compat=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.compat.pcm", + "-fprebuilt-module-path=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache", + "-O0", + "-g", + "--no-default-config", + "-nostdinc++", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/x86_64-unknown-linux-gnu/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/include", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-linux-headers/5.11.1/include", + "-c", + "${root}/src/main.cpp", + "-o", + "${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/obj/main.o" + ], + "ide": { + "role": "non-module" + }, + "local-arguments": [], + "object": "${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/obj/main.o", + "private": false, + "provides": {}, + "requires": [ + "std", + "hello.greet" + ], + "source": "${root}/src/main.cpp", + "work-directory": "${root}" + } + ], + "visible-sets": [ + "hello:test", + "mcpp:std" + ] + }, + { + "baseline-arguments": [ + "-std=c++23", + "-fmodule-file=std=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.pcm", + "-fmodule-file=std.compat=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.compat.pcm", + "-fprebuilt-module-path=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache", + "-O0", + "-g", + "--no-default-config", + "-nostdinc++", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/x86_64-unknown-linux-gnu/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/include", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-linux-headers/5.11.1/include" + ], + "family-name": "hello", + "ide": { + "configuration": "dev", + "kind": "test", + "toolchain": "llvm-22.1.8-x86_64-unknown-linux-gnu" + }, + "name": "hello:test", + "translation-units": [ + { + "arguments": [ + "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/clang++", + "-std=c++23", + "-fmodule-file=std=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.pcm", + "-fmodule-file=std.compat=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache/std.compat.pcm", + "-fprebuilt-module-path=${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/pcm.cache", + "-O0", + "-g", + "--no-default-config", + "-nostdinc++", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/x86_64-unknown-linux-gnu/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/include", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-linux-headers/5.11.1/include", + "-c", + "${root}/tests/greet_test.cpp", + "-o", + "${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/obj/greet_test.o" + ], + "ide": { + "role": "non-module" + }, + "local-arguments": [], + "object": "${env:HOME}/.mcpp/cache/build-database/00ee0ef06d348b25/target/x86_64-linux-gnu/0ba60b7727c1b097/obj/greet_test.o", + "private": false, + "provides": {}, + "requires": [ + "std", + "hello.greet" + ], + "source": "${root}/tests/greet_test.cpp", + "work-directory": "${root}" + } + ], + "visible-sets": [ + "hello", + "mcpp:std" + ] + }, + { + "baseline-arguments": [ + "-std=c++23", + "-Wno-reserved-module-identifier", + "--no-default-config", + "-nostdinc++", + "-stdlib=libc++", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/x86_64-unknown-linux-gnu/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/include", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-linux-headers/5.11.1/include" + ], + "family-name": "mcpp:std", + "ide": { + "configuration": "dev", + "kind": "library", + "toolchain": "llvm-22.1.8-x86_64-unknown-linux-gnu" + }, + "name": "mcpp:std", + "translation-units": [ + { + "arguments": [ + "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/clang++", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--no-default-config", + "-nostdinc++", + "-stdlib=libc++", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/x86_64-unknown-linux-gnu/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/include", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-linux-headers/5.11.1/include", + "--precompile", + "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/share/libc++/v1/std.cppm", + "-o", + "pcm.cache/std.pcm" + ], + "ide": { + "role": "module-interface" + }, + "local-arguments": [ + "--precompile", + "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/share/libc++/v1/std.cppm", + "-o", + "pcm.cache/std.pcm" + ], + "object": "${env:HOME}/.mcpp/build-cache/v1/std/c6af27b46156bf73/std.o", + "private": false, + "provides": { + "std": "" + }, + "requires": [], + "source": "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/share/libc++/v1/std.cppm", + "work-directory": "${env:HOME}/.mcpp/build-cache/v1/std/c6af27b46156bf73" + }, + { + "arguments": [ + "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/clang++", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--no-default-config", + "-nostdinc++", + "-stdlib=libc++", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/include/x86_64-unknown-linux-gnu/c++/v1", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/include", + "-isystem${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-linux-headers/5.11.1/include", + "-fmodule-file=std=${env:HOME}/.mcpp/build-cache/v1/std/c6af27b46156bf73/pcm.cache/std.pcm", + "--precompile", + "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/share/libc++/v1/std.compat.cppm", + "-o", + "${env:HOME}/.mcpp/build-cache/v1/std/c6af27b46156bf73/pcm.cache/std.compat.pcm" + ], + "ide": { + "role": "module-interface" + }, + "local-arguments": [ + "-fmodule-file=std=${env:HOME}/.mcpp/build-cache/v1/std/c6af27b46156bf73/pcm.cache/std.pcm", + "--precompile", + "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/share/libc++/v1/std.compat.cppm", + "-o", + "${env:HOME}/.mcpp/build-cache/v1/std/c6af27b46156bf73/pcm.cache/std.compat.pcm" + ], + "object": "${env:HOME}/.mcpp/build-cache/v1/std/c6af27b46156bf73/std.compat.o", + "private": false, + "provides": { + "std.compat": "" + }, + "requires": [ + "std" + ], + "source": "${env:HOME}/.mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/share/libc++/v1/std.compat.cppm", + "work-directory": "${env:HOME}/.mcpp/build-cache/v1/std/c6af27b46156bf73" + } + ], + "visible-sets": [ + "hello", + "hello:test" + ] + } + ], + "version": 1 + }, + "watch": [ + "${env:HOME}/.mcpp/config.toml", + "mcpp.lock", + "mcpp.toml", + "src/**/*.S", + "src/**/*.asm", + "src/**/*.c", + "src/**/*.cc", + "src/**/*.cpp", + "src/**/*.cppm", + "src/**/*.s", + "tests/**/*.cpp" + ], + "requireOffline": true, + "diagnostics": [ + { + "code": "MCPP_MEMBER_PLAN_FAILED", + "severity": "error", + "source": "mcpp", + "path": "tools/updater/mcpp.toml", + "message": "member 'updater' could not be planned: its build program failed (lupdate: Qt6Qml.dll not found)" + } + ] +} diff --git a/conformance/fixtures/mcpp-emit-partial/mcpp.toml b/conformance/fixtures/mcpp-emit-partial/mcpp.toml new file mode 100644 index 0000000..a29d7a2 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-partial/mcpp.toml @@ -0,0 +1,8 @@ +[package] +name = "hello" +version = "0.1.0" +description = "Conformance fixture: mcpp-emit" +license = "Apache-2.0" + +[toolchain] +default = "llvm@22.1.8" diff --git a/conformance/fixtures/mcpp-emit-partial/scenario.json b/conformance/fixtures/mcpp-emit-partial/scenario.json new file mode 100644 index 0000000..e0bbfb5 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-partial/scenario.json @@ -0,0 +1,44 @@ +{ + "name": "mcpp-emit-partial", + "description": "S2 0.3.0 (S2-3.4-12, S2-3.4-13; mcpp-community/mcpp#699, fix plan 2026-09-26 F10): mcpp answers with the database of every member it could plan and an `error` diagnostic whose `path` names the one it could not (here a member whose build program failed, as GalTranslPP's Updater did in issue #23's CI), exiting 1. The rest of the document is used -- the project is mcpp's, not scanned sources -- and the status names the part that is missing, by its build description file. Simulated by mcppls-mock-mcpp.", + "server-arguments": [ + "--mcpp", + "{runner-dir}/mcppls-mock-mcpp{exe}" + ], + "checks": [ + { + "id": "P1-the-rest-is-used", + "kind": "status", + "source": "mcpp", + "level": 3, + "state": "degraded", + "issue-code": "producer-partial", + "issue-message": "tools/updater/mcpp.toml", + "timeout": 60 + }, + { + "id": "P2-features-from-it", + "kind": "definition", + "file": "src/main.cpp", + "at": [ + 4, + 31 + ], + "expect": "src/greet/greet.cppm" + }, + { + "id": "P3-in-the-report", + "kind": "report", + "expect": [ + { + "path": "/roots/0/project/issues/*/code", + "equals": "producer-partial" + }, + { + "path": "/roots/0/project/source", + "equals": "mcpp" + } + ] + } + ] +} diff --git a/conformance/fixtures/mcpp-emit-partial/src/greet/detail.cppm b/conformance/fixtures/mcpp-emit-partial/src/greet/detail.cppm new file mode 100644 index 0000000..56cb780 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-partial/src/greet/detail.cppm @@ -0,0 +1,5 @@ +export module hello.greet:detail; +import std; +export namespace hello::detail { + std::string prefix() { return "Hello, "; } +} diff --git a/conformance/fixtures/mcpp-emit-partial/src/greet/greet.cppm b/conformance/fixtures/mcpp-emit-partial/src/greet/greet.cppm new file mode 100644 index 0000000..21a6536 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-partial/src/greet/greet.cppm @@ -0,0 +1,7 @@ +export module hello.greet; +export import :detail; +import std; + +export namespace hello { + std::string greet(std::string_view who) { return detail::prefix() + std::string(who); } +} diff --git a/conformance/fixtures/mcpp-emit-partial/src/main.cpp b/conformance/fixtures/mcpp-emit-partial/src/main.cpp new file mode 100644 index 0000000..71169db --- /dev/null +++ b/conformance/fixtures/mcpp-emit-partial/src/main.cpp @@ -0,0 +1,7 @@ +import std; +import hello.greet; + +int main(int argc, char* argv[]) { + std::println("{}", hello::greet("mcpp")); + return 0; +} diff --git a/conformance/fixtures/mcpp-emit-partial/tests/greet_test.cpp b/conformance/fixtures/mcpp-emit-partial/tests/greet_test.cpp new file mode 100644 index 0000000..c1632f7 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-partial/tests/greet_test.cpp @@ -0,0 +1,6 @@ +import std; +import hello.greet; + +int main() { + return hello::greet("test").empty() ? 1 : 0; +} diff --git a/conformance/fixtures/mcpp-emit-wait/mcpp-mock.json b/conformance/fixtures/mcpp-emit-wait/mcpp-mock.json new file mode 100644 index 0000000..0dbe655 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-wait/mcpp-mock.json @@ -0,0 +1,7 @@ +{ + "hang": { + "seconds": 45, + "holdPipe": true, + "exitAtOnce": false + } +} diff --git a/conformance/fixtures/mcpp-emit-wait/mcpp.toml b/conformance/fixtures/mcpp-emit-wait/mcpp.toml new file mode 100644 index 0000000..f84b886 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-wait/mcpp.toml @@ -0,0 +1,8 @@ +[package] +name = "hello" +version = "0.1.0" +description = "Conformance fixture: mcpp-emit-hang" +license = "Apache-2.0" + +[toolchain] +default = "llvm@22.1.8" diff --git a/conformance/fixtures/mcpp-emit-wait/scenario.json b/conformance/fixtures/mcpp-emit-wait/scenario.json new file mode 100644 index 0000000..cae6fc5 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-wait/scenario.json @@ -0,0 +1,14 @@ +{ + "name": "mcpp-emit-wait", + "description": "Fix plan 2026-09-26 F4 (decision D1): a project whose build system was detected gets clangd with the build tool's model, not the one scanned from its sources that stands in meanwhile. The producer (mcppls-mock-mcpp) hangs past the ten seconds after which scanned sources are planned, and is ended at its 20 s bound: until then mcppls's own engine answers what it can and clangd is given no plan at all, where 0.0.4 gave clangd the provisional plan -- in issue #23 a Windows clangd crashed three times on it before the real model came, and was given up on. After the bound, clangd starts with what there is.", + "server-arguments": ["--mcpp", "{runner-dir}/mcppls-mock-mcpp{exe}", "--producer-timeout", "20"], + "checks": [ + { "id": "W1-module-features-while-it-waits", "kind": "definition", "file": "src/main.cpp", "at": [1, 9], "expect": "src/greet/greet.cppm", "timeout": 15 }, + { "id": "W2-clangd-waits-for-the-producer", "kind": "report", "timeout": 15, "expect": [ + { "path": "/roots/0/events/*/kind", "equals": "engine-waits-for-producer" }, + { "path": "/roots/0/engines/*/details/generation", "equals": 1 } ] }, + { "id": "W3-clangd-once-the-producer-is-given-up", "kind": "hover-contains", "file": "src/main.cpp", "at": [4, 31], "expect": "greet", "timeout": 90 }, + { "id": "W4-the-model-is-scanned-sources-not-nothing", "kind": "status", "source": "inferred", "state": "degraded" }, + { "id": "W5-nothing-was-written-into-the-project", "kind": "workspace-unchanged" } + ] +} diff --git a/conformance/fixtures/mcpp-emit-wait/src/greet/detail.cppm b/conformance/fixtures/mcpp-emit-wait/src/greet/detail.cppm new file mode 100644 index 0000000..56cb780 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-wait/src/greet/detail.cppm @@ -0,0 +1,5 @@ +export module hello.greet:detail; +import std; +export namespace hello::detail { + std::string prefix() { return "Hello, "; } +} diff --git a/conformance/fixtures/mcpp-emit-wait/src/greet/greet.cppm b/conformance/fixtures/mcpp-emit-wait/src/greet/greet.cppm new file mode 100644 index 0000000..21a6536 --- /dev/null +++ b/conformance/fixtures/mcpp-emit-wait/src/greet/greet.cppm @@ -0,0 +1,7 @@ +export module hello.greet; +export import :detail; +import std; + +export namespace hello { + std::string greet(std::string_view who) { return detail::prefix() + std::string(who); } +} diff --git a/conformance/fixtures/mcpp-emit-wait/src/main.cpp b/conformance/fixtures/mcpp-emit-wait/src/main.cpp new file mode 100644 index 0000000..71169db --- /dev/null +++ b/conformance/fixtures/mcpp-emit-wait/src/main.cpp @@ -0,0 +1,7 @@ +import std; +import hello.greet; + +int main(int argc, char* argv[]) { + std::println("{}", hello::greet("mcpp")); + return 0; +} diff --git a/conformance/fixtures/typing-autosave/scenario.json b/conformance/fixtures/typing-autosave/scenario.json new file mode 100644 index 0000000..9b11a83 --- /dev/null +++ b/conformance/fixtures/typing-autosave/scenario.json @@ -0,0 +1,240 @@ +{ + "name": "typing-autosave", + "description": "Fix plan 2026-09-26 F16, F13, F11, F12, F17 (the hello project, with files.autoSave afterDelay). Every step is also written to disk and reported as saved, as autosave does, and a new line below the imports stays on `import hello.` for twelve seconds, as in the hello project: clangd reads a file's imports from disk (UP-14), so WA-CLANGD-001's rewrite of the text it is given does not reach there, and 0.0.4 spun a clangd worker at a full core until clangd was restarted, every request for the file waiting behind it. The file is now set aside before clangd builds it, answered by mcppls's own engine, and handed back once the disk is fixed, with no restart and an incident recording the broken premise. Typing a module nothing provides, saved and left quiet long enough to get a stand-in, then changed back, restarts nothing. A directive missing its ';' is reported on the directive (WA-CLANGD-006), and an import only in the unsaved buffer is information, not an error (WA-CLANGD-007).", + "server-arguments": [ + "--no-discover" + ], + "checks": [ + { + "id": "S1", + "kind": "status", + "source": "inferred", + "state": "ready", + "engine-name": "clangd" + }, + { + "id": "T0-before", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 4, + 31 + ], + "expect": "greet", + "timeout": 90 + }, + { + "id": "T1-type-to-the-dot-saved", + "kind": "type-text", + "file": "src/main.cpp", + "line": 2, + "steps": [ + "i", + "im", + "imp", + "impo", + "impor", + "import", + "import ", + "import h", + "import he", + "import hel", + "import hell", + "import hello", + "import hello." + ], + "interval-ms": 150, + "answer-within": 5, + "save": true, + "states-never": [ + "error" + ] + }, + { + "id": "T2-stay-on-the-dot-saved", + "kind": "type-text", + "file": "src/main.cpp", + "line": 2, + "steps": [ + "import hello.", + "import hello." + ], + "interval-ms": 6000, + "answer-within": 5, + "save": true, + "states-never": [ + "error" + ] + }, + { + "id": "T3-disk-seen", + "kind": "report", + "timeout": 20, + "expect": [ + { + "path": "/roots/0/events/*/kind", + "equals": "disk-unsafe" + }, + { + "path": "/roots/0/engines/*/details/filesUnsafeOnDisk", + "min-items": 1 + }, + { + "path": "/roots/0/events/*/detail/kind", + "equals": "workaround-premise" + } + ] + }, + { + "id": "T4-keep-typing-saved", + "kind": "type-text", + "file": "src/main.cpp", + "line": 2, + "steps": [ + "import hello.e", + "import hello.ex", + "import hello.ext", + "import hello.extr", + "import hello.extra", + "import hello.extra;" + ], + "interval-ms": 150, + "answer-within": 5, + "save": true, + "states-never": [ + "error" + ] + }, + { + "id": "T5-features-back", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 4, + 31 + ], + "expect": "greet", + "timeout": 30 + }, + { + "id": "T6-nothing-provides-it", + "kind": "type-text", + "file": "src/main.cpp", + "line": 2, + "steps": [ + "import hello.n", + "import hello.no", + "import hello.nos", + "import hello.nosu", + "import hello.nosuch", + "import hello.nosuch;" + ], + "interval-ms": 150, + "answer-within": 5, + "save": true, + "states-never": [ + "error" + ] + }, + { + "id": "T7-quiet-long-enough-for-a-stand-in", + "kind": "type-text", + "file": "src/main.cpp", + "line": 2, + "steps": [ + "import hello.nosuch;", + "import hello.nosuch;" + ], + "interval-ms": 4000, + "answer-within": 5, + "save": true, + "states-never": [ + "error" + ] + }, + { + "id": "T8-back-to-the-module", + "kind": "type-text", + "file": "src/main.cpp", + "line": 2, + "steps": [ + "import hello.nosuc", + "import hello.", + "import hello", + "import", + "", + "" + ], + "interval-ms": 400, + "answer-within": 5, + "save": true, + "states-never": [ + "error" + ] + }, + { + "id": "T9-features-after", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 4, + 31 + ], + "expect": "greet", + "timeout": 30 + }, + { + "id": "T10-nothing-left-aside", + "kind": "report", + "timeout": 30, + "expect": [ + { + "path": "/roots/0/engines/*/details/filesUnsafeOnDisk", + "max-items": 0 + }, + { + "path": "/roots/0/engines/*/details/filesSetAside", + "max-items": 0 + }, + { + "path": "/roots/0/events/*/kind", + "equals": "file-handed-back" + } + ] + }, + { + "id": "T11-no-restart-but-for-a-spin", + "kind": "report", + "timeout": 10, + "expect": [ + { + "path": "/roots/0/engines/*/details/restarts/*/reason", + "each-contains": "would not finish" + } + ], + "only-on": [ + "linux", + "macos" + ] + }, + { + "id": "D1-missing-semicolon-on-its-directive", + "kind": "diagnostic-code", + "file": "src/main.cpp", + "text": "import std;\nimport hello.greet\n\nint main(int argc, char* argv[]) {\n std::println(\"{}\", hello::greet(\"mcpp\"));\n return 0;\n}\n", + "expect": "expected_semi_after_module_or_import", + "line": 1 + }, + { + "id": "D2-unsaved-import-is-information", + "kind": "diagnostic-code", + "file": "src/main.cpp", + "text": "import std;\nimport hello.greet;\nimport hello.extra;\n\nint main(int argc, char* argv[]) {\n std::println(\"{} {}\", hello::greet(\"mcpp\"), hello::extra());\n return 0;\n}\n", + "expect": "unsaved-import", + "severity": 3, + "absent": [ + "module_not_found" + ] + } + ] +} diff --git a/conformance/fixtures/typing-autosave/src/extra/extra.cppm b/conformance/fixtures/typing-autosave/src/extra/extra.cppm new file mode 100644 index 0000000..fa31d1d --- /dev/null +++ b/conformance/fixtures/typing-autosave/src/extra/extra.cppm @@ -0,0 +1,5 @@ +export module hello.extra; + +export namespace hello { + int extra() { return 42; } +} diff --git a/conformance/fixtures/typing-autosave/src/greet/detail.cppm b/conformance/fixtures/typing-autosave/src/greet/detail.cppm new file mode 100644 index 0000000..56cb780 --- /dev/null +++ b/conformance/fixtures/typing-autosave/src/greet/detail.cppm @@ -0,0 +1,5 @@ +export module hello.greet:detail; +import std; +export namespace hello::detail { + std::string prefix() { return "Hello, "; } +} diff --git a/conformance/fixtures/typing-autosave/src/greet/greet.cppm b/conformance/fixtures/typing-autosave/src/greet/greet.cppm new file mode 100644 index 0000000..21a6536 --- /dev/null +++ b/conformance/fixtures/typing-autosave/src/greet/greet.cppm @@ -0,0 +1,7 @@ +export module hello.greet; +export import :detail; +import std; + +export namespace hello { + std::string greet(std::string_view who) { return detail::prefix() + std::string(who); } +} diff --git a/conformance/fixtures/typing-autosave/src/main.cpp b/conformance/fixtures/typing-autosave/src/main.cpp new file mode 100644 index 0000000..71169db --- /dev/null +++ b/conformance/fixtures/typing-autosave/src/main.cpp @@ -0,0 +1,7 @@ +import std; +import hello.greet; + +int main(int argc, char* argv[]) { + std::println("{}", hello::greet("mcpp")); + return 0; +} diff --git a/conformance/traceability.json b/conformance/traceability.json index 255f80b..bb99848 100644 --- a/conformance/traceability.json +++ b/conformance/traceability.json @@ -683,6 +683,33 @@ "contains": "!spec::effects_acceptable(effects->second)" } ], + "S2-3.4-12": [ + { + "validate": "S2 schema accepts a partial document: data and an error naming a path" + }, + { + "validate": "S2 schema rejects a diagnostic path that is not a string" + }, + { + "script": "src/bin/mockmcpp.cpp", + "contains": "return partial ? 1 : 0;" + } + ], + "S2-3.4-13": [ + { + "check": "mcpp-emit-partial/P1-the-rest-is-used" + }, + { + "check": "mcpp-emit-partial/P2-features-from-it" + }, + { + "test": "tests/test_spec.cpp: a single-document envelope is read, and only a database answers" + }, + { + "script": "src/project/mcpp.cpp", + "contains": "enriched.issues.emplace_back(\"producer-partial\"" + } + ], "S2-4-1": [ { "manual": "A producer requirement for mcpp (mcpp-community/mcpp#636); the simulated producer runs no build at all." @@ -994,6 +1021,24 @@ "manual": "The VS Code extension only shows the report as a document (editors/vscode/src/commands.ts, collectReport); no feature reads it." } ], + "S3-5.5-3": [ + { + "check": "diagnostic-bundle/B1-bundle" + }, + { + "test": "tests/test_bundle.cpp: a home directory is ~ in every spelling of a POSIX path" + }, + { + "test": "tests/test_bundle.cpp: a Windows profile is ~ with either separator, escaped, encoded, from WSL and by its 8.3 name" + }, + { + "test": "tests/test_bundle.cpp: other people's profile directories get their own placeholder, the same one every time" + }, + { + "script": "src/server/session.cpp", + "contains": "reply_(id, redact ? bundle::redact_report(full_report_()) : full_report_());" + } + ], "S3-6-1": [ { "test": "tests/test_server.cpp: merging" @@ -1032,6 +1077,74 @@ "test": "tests/test_tokens.cpp: clangd's legend, including its own duplicates, maps by name" } ], + "S3-6.2-1": [ + { + "test": "tests/test_completion.cpp: the space gate passes an import directive's keyword and one blank, and nothing else" + }, + { + "check": "completion-keywords/P1-space-elsewhere" + }, + { + "check": "completion-keywords/P4-two-spaces" + }, + { + "check": "engine-none/M4-space-elsewhere" + } + ], + "S3-6.2-2": [ + { + "test": "tests/test_completion.cpp: the space is advertised to VS Code and its forks, or to a client that asks" + }, + { + "script": "src/server/session.cpp", + "contains": "if (orchestrator::completion::space_trigger_wanted(clientParams_)) orchestrator::completion::add_space_trigger" + } + ], + "S3-6.2-3": [ + { + "test": "tests/test_completion.cpp: the space is advertised to VS Code and its forks, or to a client that asks" + }, + { + "check": "completion-keywords/C0-space-is-a-trigger" + } + ], + "S3-6.2-4": [ + { + "test": "tests/test_completion.cpp: module keywords where a declaration can begin, filtered by what was typed" + }, + { + "test": "tests/test_completion.cpp: no keywords inside braces, comments, literals, or the middle of a word" + }, + { + "test": "tests/test_completion.cpp: keywords merge with the core engine's answer without duplicates" + }, + { + "check": "completion-keywords/K1-import" + }, + { + "check": "completion-keywords/K3-export-import" + }, + { + "check": "completion-keywords/K4-private-fragment" + }, + { + "check": "completion-keywords/K5-not-in-a-body" + } + ], + "S3-6.2-5": [ + { + "check": "completion-keywords/K2-module-declarations" + }, + { + "check": "completion-keywords/R1-costs-counted" + }, + { + "check": "engine-none/M4-keywords" + }, + { + "check": "engine-none/M4-export-import" + } + ], "S4-3-1": [ { "validate": "S4 schema rejects unknown kit-version" diff --git a/docs/20-projects.md b/docs/20-projects.md index 84f68c3..02a8bad 100644 --- a/docs/20-projects.md +++ b/docs/20-projects.md @@ -76,6 +76,27 @@ each module is. `import std` resolves, and so do the modules of your own code. D a standard library that is not the one you build with, so they can differ — the status bar says the profile is a semantic kit rather than a build toolchain. +## Which C++ standard, and C++26 + +The standard is the build's: whatever `-std=` (or `/std:`) a unit's command says, mcppls gives +clangd; `/std:c++latest` is C++26. Two rules go further. + +- **One standard per context for module units.** A module's BMI can only be imported under the + standard it was built with — `import std` in a C++26 file fails outright when `std` was built as + C++23 ("C++26 was disabled in precompiled file"). So the units of one context that import, provide + or belong to a module are read with the newest standard among them, and a plain unit keeps its own; + the log says when some were raised, and the report's `plan.languageStandard`, + `plan.standardsSeen` and `plan.standardsRaised` say which. The status profile names the standard. +- **Sources nothing describes are read with the newest standard their compiler takes**: C++26 with + the semantic kit (clang 23, libc++ 23), GCC 14 and later, and Clang 17 and later (spelled `c++2c` + before Clang 20); C++23, the oldest standard with `import std`, for older compilers. + +What C++26 gives you is clangd 23.1's: pack indexing, `= delete("reason")`, placeholder `_`, +`static_assert` messages, `#embed`, variadic friends and the rest of what clang 23 implements, and +the C++26 library of the standard library you build with (the kit's is libc++ 23). **Contracts +(P2900) and reflection (P2996) are not in clang 23**: code using them shows clang's errors even +where GCC compiles it; VS Code still colors `contract_assert`, `pre` and `post`. + ## An untrusted workspace No build tool and no compiler is run at all — VS Code's workspace trust is respected before anything diff --git a/docs/30-settings.md b/docs/30-settings.md index 8a470c9..e9fcda4 100644 --- a/docs/30-settings.md +++ b/docs/30-settings.md @@ -12,6 +12,7 @@ | `mcppls.ai.enabled` | `false` (default) | Whether the model-backed half of change review may be used. Off means the server makes no model calls | | `mcppls.detectConflicts` | `true` (default) | Offer once to turn off another C++ extension's language features in this workspace, and say so when one becomes active later | | `mcppls.semanticTokens.modules` | `true` (default) | Color `import`, `module`, `export` and module names from the server's semantic tokens. Off: only the grammar's colors | +| `mcppls.completion.triggerOnSpace` | `true` (default) | Show the module list as soon as a space is typed after `import` or `export import`. A space anywhere else never reaches the server. Other editors ask for the same with `initializationOptions.completion.triggerOnSpace` | | `mcppls.trace.server` | `off` (default), `messages`, `verbose` | Log the LSP traffic to the C++ Modules output channel (at Trace level); `verbose` adds the server's debug log (at Debug level). Set the channel's log level to see them | ## Commands diff --git a/docs/50-troubleshooting.md b/docs/50-troubleshooting.md index e24067b..ca56206 100644 --- a/docs/50-troubleshooting.md +++ b/docs/50-troubleshooting.md @@ -21,6 +21,37 @@ carrying what almost every question turns out to need: Log files outlive the editor: the report names the path, and they are kept under the cache directory with timestamps. +The report is made to be shared: your home directory is `~` in it, your user and machine names are +`` and ``, and anything that looks like a secret (a token, a password, an API key, an +e-mail address) is ``. The project's own paths are kept — they are what it is read for. + +**C++ Modules: Export Diagnostic Bundle** goes further: one zip, written under the cache directory's +`bundles/` (the newest five are kept) and never uploaded, with everything a problem usually needs — + +| In the bundle | What it is | +|---|---| +| `report.json` | The report above | +| `environment.json` | System, editor and extension versions, the other C/C++ extensions, your mcppls settings, the payload, the toolchains found, and a few environment variables (`PATH`, `LANG`, `LC_*`, `MCPP_*`, `XLINGS_*`) — no other | +| `logs/` | The server's logs of the last three sessions and any other of the last day, and the extension's own log | +| `incidents/` | What the server wrote down when clangd crashed, hung or was set aside | +| `engine/` | The database clangd was given, and the plan behind it | +| `manifest.json` | Every file with its size and SHA-256, and how many replacements each redaction rule made | + +— with the same replacements in every file. Before anything is written, the bundle is searched for +your home directory, user name and host name in every spelling; if any is left, **no bundle is +written** and the message says which file, and *Retry with Project Paths Hidden* replaces the +project's paths too. Source files are never included; an incident carries only the lines it is +about. Other editors run the same command as `workspace/executeCommand` `mcppls.exportBundle`, and +on the command line: + +```bash +mcppls report --bundle problem.zip --root path/to/project # --hide-project-paths, --no-source-excerpts +``` + +Crash dumps are left out unless asked for (`--include-dumps`): they hold memory, which cannot be +redacted. `--no-redact` keeps everything as it is, for looking at a problem on your own machine; it +is not offered in the editor. + ## Symptoms **Nothing works — no go-to-definition anywhere.** Look at `project.source` in the report. If it is @@ -60,6 +91,22 @@ and every later version of the file waits behind it: typing any dotted import we text. mcppls 0.0.4 gives clangd the line with `;` right after the dot instead, which clangd reports at once (workaround `WA-CLANGD-001`); `engines[].details.workarounds` in the report lists it. +**While typing an import with autosave on, the file loses clangd for a few seconds.** clangd reads a +file's imports from its text on disk, not from the editor (0.0.4 and earlier could stall there for +good: every request for the file unanswered after `import hello.` was autosaved). When a save puts +something on disk clangd would stall on — a module name ending in `.`, or an import of a module the +project does not have (yet) — the file is answered by mcppls's own engine until it is saved again; +the status lists it as `file-unsafe-on-disk`, category `code`, with the reason, and stays *ready*. +A module nothing provides gets a stand-in within a second of the save, and the file goes back to +clangd once clangd has read the database with it, about six seconds later. + +**"Import directive must end with a ';'" on the wrong line, or "module X not found" for an import you +just typed.** clangd reports a directive missing its `;` on the code after it; mcppls moves the +diagnostic back onto the directive (workaround `WA-CLANGD-006`). An import typed and not saved yet is +not built by clangd until the file is saved (it reads imports from disk); while the module is in the +project, that is an information-level "module 'X' is in the project; clangd loads it once the file is +saved", not an error (`WA-CLANGD-007`). + **"clangd would not finish main.cpp".** A file's build ran past its budget — five times its own last build, never under 20 s — while the editor waited on it: clangd will not finish it, busy or not. The file is answered by mcppls's own engine, with module-level features, until its text @@ -80,10 +127,54 @@ confirms it in the background. `project.firstOrigin` says which happened. If it `producer`, the fingerprint is not matching — the report's `project.producerRun` and the build files' timestamps are where to look. -**clangd keeps restarting.** `engines[].restarts` and the `events` journal. Restarts are spaced out -and capped at three in ten minutes, after which the status says so (`engine-restart-capped`) and -mcppls's own engine answers what a restart would have tried to fix; a module that does not compile -is never a reason to restart. A burst usually means the compile arguments are changing under it. +**clangd keeps restarting.** `engines[].restarts`, `engines[].details.restartBudget` and the `events` +journal. Each reason has its own budget of three restarts in ten minutes: the engine database +changing (`plan`), recovering a clangd that stopped answering or spun (`recovery`), and clangd +exiting (`crash`). Past it, the next restart of that kind waits one, two, four, then eight minutes +— it is backed off, not refused, so a stuck clangd always comes back — and the status says so +(`engine-restart-capped`) with a **Restart clangd** button (other editors: `workspace/executeCommand` `mcppls.restartEngine`), which restarts +at once and is never counted. Switching the toolchain, the profile or the context is never counted +either, and a module that does not compile is never a reason to restart. Every change of the engine +database is logged with what it changed (`engine database changed: … compiled otherwise (main.cpp: +argument 3: -O0 -> -O2)`), so a burst names its cause. + +**"clangd crashed while building NormalJsonTranslator.Core.cpp".** clangd names the file it crashed +on (its crash context), and that file is set aside: answered by mcppls's own engine while clangd is +restarted without it. `engines[].details.lastExit` in the report has the exit code, the file, what +clangd was doing and, on Windows, the exception code. Five exits in five minutes and clangd is given +up until the next server start; the status offers **Export Diagnostic Bundle**. + +**"mcpp could not describe tools/updater/mcpp.toml".** The build tool described the rest of the +workspace and said which part it could not (a member whose build program failed, say); that part's +files are read with what the rest gives them, and the rest works as usual (`producer-partial`, S2 +0.3.0). Fix what the message names, and the next reload describes it too. + +**"clangd rejected the compile command for module scanning".** Before it builds a module, clangd +scans each unit of the database for its imports, with that unit's compile command; a command the +compiler driver rejects fails the scan, and no module is built — issue #23's `LTO requires +-fuse-ld=lld` was one. The status names the first rejection in the driver's own words (category +`environment`); `engines[].details.scanFailures` counts them. A header the command cannot find is +told the same way (category `project`). A scan of a file you are typing fails all the time and is +only counted. + +**"C++26 was disabled in precompiled file".** A module was built with one C++ standard and imported +under another; clang refuses that. mcppls reads the module units of a context with one standard, +the newest they name (`plan.languageStandard` in the report, `standard` in the status profile), so +this should only come from a module clangd built before 0.0.5 — it rebuilds on the next change — or +from the build itself mixing standards, which the build tool's own compiler will refuse too. + +**Right after opening a project, only module-level features for up to a minute.** A project whose +build system was found gets clangd once its build tool has described it — up to a minute, the +build tool's own limit — rather than with a guess from its sources that clangd would then have to +unlearn; mcppls's own engine answers module navigation, hover on imports and `import` completion +meanwhile. A second session starts from the cached model at once. + +**Where the server writes down what went wrong.** A crash, a stuck or spinning clangd, a file set +aside, a restart held back and a broken workaround premise each leave an *incident*: a directory +under the workspace's cache (`incidents/-/`, the newest twenty, for a week) with +what led up to it, clangd's latest log lines (clangd logs at `info` into memory, never to the default +log), each file's lines where the editor's text and the disk's differ, and which of clangd's threads +used the CPU. The diagnostic bundle carries them. **"clangd stopped making progress; it was restarted".** clangd left a request unanswered, answered nothing else meanwhile, and used next to no CPU for five seconds: it was waiting for something that @@ -103,5 +194,6 @@ payload. An editor that starts `mcppls` itself can give it a clangd of your own, ## Filing a bug -Attach the diagnostic report. It names paths on your machine, so read it first — it carries no -environment variable values and no file contents, by design. +Attach the diagnostic bundle (**C++ Modules: Export Diagnostic Bundle**, or `mcppls report --bundle`), +or at least the diagnostic report. Both have your user name, home directory, host name and secrets +replaced, and neither carries the contents of your files; read them before attaching all the same. diff --git a/docs/specs/CHANGELOG.md b/docs/specs/CHANGELOG.md index d7741f2..ffcde2c 100644 --- a/docs/specs/CHANGELOG.md +++ b/docs/specs/CHANGELOG.md @@ -2,6 +2,42 @@ Changes to the specifications in this directory. Each specification is versioned independently. +## 2026-09-26 — S3: the standard a profile reads with + +`SemanticProfile` gains `standard` (optional): the C++ standard the context's module units are read +with, for example `"c++26"`. A BMI can only be imported under the standard it was built with, so a +server reads the module units of one context with one standard; this field says which. Additive: +protocol version stays 1. + +## 2026-09-26 — S3: completion of module syntax + +Added section 6.2. A server may make a space a completion trigger character, for the module names +after `import`; if it does, a space-triggered request anywhere but right after `import ` or +`export import ` is answered at once, empty, without the semantic engine (S3-6.2-1), and the space is +advertised only to a client that asks for it (`initializationOptions.completion.triggerOnSpace`) or +that the server knows drops the other spaces itself; never to one that declared `false` (S3-6.2-2, +S3-6.2-3). Servers offer the module-syntax keywords where each can begin a declaration, merged with +the semantic engine's result, and without it when it does not answer in time (S3-6.2-4, S3-6.2-5). +All additive: protocol version stays 1. + +## 2026-09-26 — S3: a report names no one + +`cxxModules/report` takes `redact` (optional, default `true`): a server replaces the user's home +directory, the user's and the machine's names and recognizable secrets in its report with +placeholders, the same placeholder for the same original, and keeps the project's own paths +(S3-5.5-3). `redact: false` gets the report as before. Additive: protocol version stays 1. + +## 2026-09-26 — S2 0.3.0: a partial single-document answer + +In single-document mode a document may carry `data` together with `error` diagnostics. It describes +everything except what those diagnostics name; the producer names each part it could not describe +(a workspace member or a package) in the diagnostic's new optional `path` field, and a consumer uses +the rest of the document (S2-3.4-12, S2-3.4-13). `data` is present when the command described the +workspace in whole or in part; a command without `data` has still failed (S2-3.4-11). A consumer +written for 0.2.0 reads such a document as a success with errors, which is the intended reading. +mcpp implements it with mcpp-community/mcpp#699 (in #702); before, one workspace member's planning +failure used to remove every member's sets. + ## 2026-09-25 — S3: issue categories, the degraded hold, module syntax in semantic tokens Added `category` (optional) to `CxxModulesIssue`: `code`, `engine`, `environment` or `project`, diff --git a/docs/specs/README.md b/docs/specs/README.md index b054966..efe1612 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -5,7 +5,7 @@ This directory holds the normative specifications that let C++ named modules be | Spec | Title | Version | Status | Schema | |---|---|---|---|---| | [S1](s1-build-database.md) | C++ Build Database: IDE Profile | profile-version 0.2.0 | Draft | [s1-build-database.schema.json](schema/s1-build-database.schema.json) | -| [S2](s2-discovery.md) | Build Database Discovery Protocol | 0.2.0 | Draft | [s2-discovery.schema.json](schema/s2-discovery.schema.json) | +| [S2](s2-discovery.md) | Build Database Discovery Protocol | 0.3.0 | Draft | [s2-discovery.schema.json](schema/s2-discovery.schema.json) | | [S3](s3-lsp-extensions.md) | Language Server Protocol Extensions for C++ Modules | protocol version 1 | Draft | TypeScript interfaces in the text | | [S4](s4-semantic-kit.md) | Semantic Kit | kit-version 1 | Draft | [s4-kit.schema.json](schema/s4-kit.schema.json) | | [S5](s5-semantic-query.md) | Semantic Queries for C++ Code | 0.1.0 | Draft | TypeScript interfaces in the text | diff --git a/docs/specs/s2-discovery.md b/docs/specs/s2-discovery.md index 82690ec..266c504 100644 --- a/docs/specs/s2-discovery.md +++ b/docs/specs/s2-discovery.md @@ -3,7 +3,7 @@ | | | |---|---| | Specification | S2 | -| Version | 0.2.0 | +| Version | 0.3.0 | | Status | Draft | | Schema | [`schema/s2-discovery.schema.json`](schema/s2-discovery.schema.json) | | Examples | [`examples/s2-request.json`](examples/s2-request.json), [`examples/s2-messages.jsonl`](examples/s2-messages.jsonl), [`examples/s2-envelope.json`](examples/s2-envelope.json) | @@ -92,11 +92,13 @@ A producer that already prints machine-readable envelopes offers discovery as on | `kind` | string | MUST | A name ending in `.build-database`, for example `mcpp.build-database`. S2-3.4-2 | | `kindVersion` | integer | MUST | `1`. S2-3.4-3 | | `effects` | string[] | MUST | What running the command did, for example `read-project`. S2-3.4-4 | -| `data` | object | conditional MUST | Present when the command succeeded: `database` (object, MUST), the S1 document; `watch` (string[], MUST), as in section 3.3; `inputs-fingerprint` (string, SHOULD), a digest of the inputs `watch` names. S2-3.4-5, S2-3.4-6, S2-3.4-7, S2-3.4-8 | -| `diagnostics` | object[] | MUST | Each with `code`, `severity` (`error`, `warning` or `note`) and `message`. S2-3.4-9 | +| `data` | object | conditional MUST | Present when the command described the workspace in whole or in part: `database` (object, MUST), the S1 document; `watch` (string[], MUST), as in section 3.3; `inputs-fingerprint` (string, SHOULD), a digest of the inputs `watch` names. S2-3.4-5, S2-3.4-6, S2-3.4-7, S2-3.4-8 | +| `diagnostics` | object[] | MUST | Each with `code`, `severity` (`error`, `warning` or `note`) and `message`; `path` (string, MAY) names the file the diagnostic concerns, relative to the workspace root. S2-3.4-9 | The consumer writes nothing to the command's standard input. Before running it, the consumer reads the producer's protocol description, ` --protocol-version`: a JSON object whose `kinds` maps kind names to versions and whose `commands` maps command names to the `effects` they may have. A consumer **MUST** use single-document mode only when `kinds` contains the build-database kind, and **MUST** run the command only when the workspace is trusted and the listed effects are acceptable. A command without `data` has failed; its `diagnostics` say why. S2-3.4-10, S2-3.4-11 +A document may carry `data` together with `error` diagnostics. It describes everything except what those diagnostics name: the producer **MUST** name each part it could not describe, such as a workspace member or a package, in an `error` diagnostic whose `path` is that part's build description file, and a consumer **SHOULD** use the rest of the document and report the errors. The command's exit status is non-zero in this case. S2-3.4-12, S2-3.4-13 + In this mode the producer does not write a database file. The consumer keeps the document where it keeps its own state. Example: [`examples/s2-envelope.json`](examples/s2-envelope.json). diff --git a/docs/specs/s3-lsp-extensions.md b/docs/specs/s3-lsp-extensions.md index 7f9bd0d..931488b 100644 --- a/docs/specs/s3-lsp-extensions.md +++ b/docs/specs/s3-lsp-extensions.md @@ -88,6 +88,7 @@ interface SemanticProfile { compiler?: string; // e.g. "gcc 16.1.0"; absent for a semantic kit stdlib: string; // e.g. "libstdc++ 16.1.0" or "libc++ 23.1.0" target: string; // e.g. "x86_64-linux-gnu" + standard?: string; // the C++ standard the context's module units are read with, e.g. "c++26" } interface CxxModulesIssue { @@ -209,7 +210,7 @@ After answering, the server rewrites the engine's input for the new context and Direction: client → server. What a report of a problem needs, gathered by the server for a person or a bug report. ```ts -// Params: {} +interface CxxModulesReportParams { redact?: boolean } // default true interface CxxModulesReport { generatedAt: string; // UTC, ISO 8601 server: { name: string; version: string; platform: string; uptimeSeconds: number; logLevel: string; logFile: string }; @@ -223,12 +224,15 @@ A server **SHOULD** answer at once with what it knows rather than wait for its e The content of each `roots` entry is the server's own and may change between server versions: a client **MUST NOT** base features on it. S3-5.5-2 +A report is made to be shared, so unless `redact` is `false` a server **SHOULD** replace in it the user's home directory (by `~`), the user's and the machine's names and anything it recognizes as a secret (by placeholders such as `` and ``), in every spelling a path takes in it, and use the same placeholder for the same original throughout. S3-5.5-3 Paths of the project itself are kept: they are what a report is read for. + ## 6. Module features through standard LSP | Feature | Standard message | Answered by | |---|---|---| | Go to the primary interface unit or partition declaration from a module name | `textDocument/definition` | the server's module index | | Import completion: module names and partitions of the same module | `textDocument/completion` | the server's module index | +| Module-syntax keywords where a declaration can begin (6.2) | `textDocument/completion` | the server, merged with the semantic engine's result | | Module-name hover: providers, role, semantic profile | `textDocument/hover` | the server's module index | | Module declaration as a top-level outline node | `textDocument/documentSymbol` | merged with the semantic engine's result | | Search by module name | `workspace/symbol` | merged with the semantic engine's result | @@ -253,6 +257,23 @@ interface CxxModulesInitializationOptions { A module name is sent with the token type `module`, and a partition name also with the modifier `partition`, only to a client that declared `moduleType: true`; to any other client a server **MUST** send module and partition names as `namespace`, so that a theme that knows only the standard types still colors them. A server **MUST NOT** add module-syntax tokens for a client that declared `modules: false`. The legend is the server's: it maps the core engine's token types and modifiers into it by name. S3-6.1-2, S3-6.1-3 +### 6.2 Completion of module syntax + +A space typed after `import` is where a person expects the module names. A server **MAY** add `" "` to `completionProvider.triggerCharacters` for it. A server that does **MUST** answer a completion request triggered by a space (`context.triggerKind` 2, `context.triggerCharacter` `" "`) whose line, up to the position, is anything but optional white space, an optional `export` and white space, `import` and exactly one white-space character, at once with an empty result, without giving it to the semantic engine. S3-6.2-1 + +Every space typed anywhere reaches the server as such a request, so a client that is sent the trigger should drop the others itself before they are sent. A server **MUST NOT** add `" "` for a client that declared `completion.triggerOnSpace: false`, and **SHOULD NOT** add it for a client that did not declare `true`, unless the server knows that client drops them. S3-6.2-2, S3-6.2-3 + +```ts +// Client → server: InitializeParams.initializationOptions +interface CxxModulesInitializationOptions { + completion?: { + triggerOnSpace?: boolean; // true: a space after `import` triggers completion; false: never + }; +} +``` + +A semantic engine may offer some module-syntax keywords, not their combined forms, and nothing while it cannot answer for a file. Where a module declaration or an import declaration can begin, a server **SHOULD** offer the keywords that can appear there — `import`; `export import` in a module interface unit; `module;` before anything else in the file; `export module` and `module` in a file with no module declaration; `module :private;` in a primary module interface unit without one — merged with the semantic engine's result without duplicate labels, and **SHOULD** still offer them when the semantic engine gives no result in time. S3-6.2-4, S3-6.2-5 No module name is offered after `export module`: the name is being declared, not referred to. + ## 7. Versioning The protocol version is an integer. Version 1 is defined by this document. A later version adds optional fields and new messages only; a change that is not backward compatible requires a new method prefix. diff --git a/docs/specs/schema/s2-discovery.schema.json b/docs/specs/schema/s2-discovery.schema.json index 228cc07..f3c02c1 100644 --- a/docs/specs/schema/s2-discovery.schema.json +++ b/docs/specs/schema/s2-discovery.schema.json @@ -202,6 +202,10 @@ }, "message": { "type": "string" + }, + "path": { + "type": "string", + "description": "The file the diagnostic concerns, relative to the workspace root (section 3.4)." } } } diff --git a/docs/specs/tools/validate.py b/docs/specs/tools/validate.py index 1466e89..f7b3aff 100644 --- a/docs/specs/tools/validate.py +++ b/docs/specs/tools/validate.py @@ -252,6 +252,13 @@ def s1_level3(name, doc): envelope = copy.deepcopy(base_envelope); envelope.pop("data") envelope["diagnostics"] = [{"code": "E_TOOLCHAIN", "severity": "error", "message": "no compiler"}] validate("S2 schema accepts a failed command without data", s2, envelope) +# S2 0.3.0: a document that describes the workspace in part carries data and an error naming what it left out. +envelope = copy.deepcopy(base_envelope) +envelope["diagnostics"] = [{"code": "MCPP_MEMBER_PLAN_FAILED", "severity": "error", "message": "member updater could not be planned", + "path": "tools/updater/mcpp.toml"}] +validate("S2 schema accepts a partial document: data and an error naming a path", s2, envelope) +envelope["diagnostics"][0]["path"] = 7 +validate("S2 schema rejects a diagnostic path that is not a string", s2, envelope, expect_valid=False) s2_request_only = Draft202012Validator({"$ref": "#/$defs/request", "$defs": load(root / "schema" / "s2-discovery.schema.json")["$defs"]}) validate("S2 request definition rejects a request carrying kind", s2_request_only, diff --git a/docs/zh-CN/20-projects.md b/docs/zh-CN/20-projects.md index bedf4c9..d4423e3 100644 --- a/docs/zh-CN/20-projects.md +++ b/docs/zh-CN/20-projects.md @@ -40,6 +40,15 @@ mcpp 给出的文档里列出了每个翻译单元、它的模块角色、它的 这时内置的**语义工具包**接管：libc++ 编译成 modules，附带一份说明每个模块位置的清单。`import std` 能解析，你自己代码里的模块也能解析。诊断信息来自一个和你实际构建所用不同的标准库，所以可能会有出入——状态栏会说明当前的语义来自语义工具包，而不是构建工具链。 +## 用哪个 C++ 标准，以及 C++26 + +标准以构建为准：一个单元的命令里写的 `-std=`（或 `/std:`）是什么，mcppls 就交给 clangd 什么；`/std:c++latest` 即 C++26。另有两条规则： + +- **同一上下文中的模块单元用同一个标准。** 模块的 BMI 只能在构建它时所用的标准下导入——`std` 按 C++23 构建时，C++26 文件里的 `import std` 会直接失败（"C++26 was disabled in precompiled file"）。因此同一上下文里导入、提供或属于某个模块的单元，统一按其中最新的标准来读，不涉及模块的普通单元保留自己的标准；有单元被提升时日志会说明，报告中的 `plan.languageStandard`、`plan.standardsSeen`、`plan.standardsRaised` 给出具体情况，状态中的 profile 也会写明所用标准。 +- **没有任何构建描述的源文件，按读取它们的编译器所支持的最新标准来读**：语义工具包（clang 23、libc++ 23）、GCC 14 及以上、Clang 17 及以上为 C++26（Clang 20 之前写作 `c++2c`）；更老的编译器为 C++23，即支持 `import std` 的最低标准。 + +C++26 能用到什么，取决于 clangd 23.1：包索引（pack indexing）、`= delete("reason")`、占位变量 `_`、`static_assert` 自定义消息、`#embed`、可变参数友元，以及 clang 23 已实现的其余特性；标准库部分取决于你构建所用的标准库（语义工具包为 libc++ 23）。**契约（P2900）和反射（P2996）clang 23 尚未实现**：使用它们的代码即使 GCC 能编译，clangd 里也会报错；VS Code 仍会为 `contract_assert`、`pre`、`post` 着色。 + ## 不受信任的工作区 不会运行任何构建工具，也不会运行编译器——VS Code 的工作区信任机制优先于一切。语义由语义工具包提供，状态会说明原因。 diff --git a/docs/zh-CN/30-settings.md b/docs/zh-CN/30-settings.md index 6474691..a554f1e 100644 --- a/docs/zh-CN/30-settings.md +++ b/docs/zh-CN/30-settings.md @@ -14,6 +14,7 @@ | `mcppls.ai.enabled` | `false`（默认） | 是否启用变更审查里依赖模型的那部分。关闭时服务端不发起任何模型调用 | | `mcppls.detectConflicts` | `true`（默认） | 在此工作区中提议关闭另一个 C++ 扩展的语言功能（只提议一次），之后又有冲突扩展启用时会提示 | | `mcppls.semanticTokens.modules` | `true`（默认） | 用服务端的语义 token 给 `import`、`module`、`export` 和模块名上色。关闭后只用语法文件的颜色 | +| `mcppls.completion.triggerOnSpace` | `true`（默认） | 在 `import` 或 `export import` 后输入空格时立即弹出模块列表；其他位置的空格不会发给服务端。其他编辑器用 `initializationOptions.completion.triggerOnSpace` 开启同样的行为 | | `mcppls.trace.server` | `off`（默认）, `messages`, `verbose` | 把 LSP 通信记录到 C++ Modules 输出通道（Trace 级别）；`verbose` 还会打开服务端的 debug 日志（Debug 级别）。要看到它们，需把该输出通道的日志级别调到对应级别 | ## 命令 diff --git a/docs/zh-CN/50-troubleshooting.md b/docs/zh-CN/50-troubleshooting.md index 3f09262..80cecf3 100644 --- a/docs/zh-CN/50-troubleshooting.md +++ b/docs/zh-CN/50-troubleshooting.md @@ -21,6 +21,27 @@ 日志文件不会随编辑器关闭而消失：报告里写明路径，日志按时间戳保存在缓存目录下。 +报告本来就是为了给别人看的：其中你的主目录写作 `~`，用户名和主机名写作 ``、``，看起来像密钥的内容（token、密码、API key、邮箱）写作 ``。工程自己的路径保留——看报告要的正是它们。 + +**C++ Modules: Export Diagnostic Bundle** 更进一步：生成一个 zip，写在缓存目录的 `bundles/` 下（保留最新的 5 个），**从不上传**，里面是排查问题通常需要的全部内容—— + +| 问题包里的文件 | 内容 | +|---|---| +| `report.json` | 上面的报告 | +| `environment.json` | 系统、编辑器和插件的版本、其他 C/C++ 插件、你的 mcppls 设置、payload、探测到的工具链，以及少数几个环境变量（`PATH`、`LANG`、`LC_*`、`MCPP_*`、`XLINGS_*`），其他的一概不收 | +| `logs/` | 服务端最近三次会话以及最近一天内其他会话的日志，还有插件自己的日志 | +| `incidents/` | clangd 崩溃、卡住或文件被搁置时服务端记下的现场 | +| `engine/` | 交给 clangd 的数据库，以及生成它的计划 | +| `manifest.json` | 每个文件的大小和 SHA-256，以及每条脱敏规则各替换了多少处 | + +——每个文件都做同样的替换。写出之前，会在整个问题包里按各种写法搜索你的主目录、用户名和主机名；只要还有残留，**就不写出问题包**，提示会说明是哪个文件，并可以选择 *Retry with Project Paths Hidden*，把工程路径也一并替换。源文件从不打包；事故记录只带与问题相关的那几行。其他编辑器用 `workspace/executeCommand` 的 `mcppls.exportBundle` 执行同一操作，命令行则是： + +```bash +mcppls report --bundle problem.zip --root path/to/project # 可加 --hide-project-paths、--no-source-excerpts +``` + +崩溃转储（dump）默认不包含（需要时用 `--include-dumps`）：它装的是内存，无法脱敏。`--no-redact` 保留一切原样，只用于在你自己的机器上排查；编辑器里不提供这个选项。 + ## 常见症状 **所有功能失效，任何位置都无法跳转到定义。** 看报告里的 `project.source`。如果一个用了构建系统的项目里它是 `inferred`，说明构建工具没有给出答复；原因在 `project.issues` 和 `toolRuns` 的最后一条里。常见原因是构建工具需要下载东西：这时状态栏会提议在你的终端里运行它。 @@ -35,6 +56,10 @@ **输入 `import` 时编辑器卡死（0.0.3 及更早版本）。** clangd 23.1 遇到模块名以 `.` 结尾、而且 `.` 就在行尾的文件（`import hello.`、`export module a.`）时永远处理不完，这个文件之后的所有版本都排在它后面等待；而输入任何带点的模块名都会经过这个状态。mcppls 0.0.4 改为把这一行在点后补上 `;` 再交给 clangd，clangd 会立即报告这个错误（规避措施 `WA-CLANGD-001`）；报告里的 `engines[].details.workarounds` 会列出它。 +**开着自动保存输入 import 时，这个文件有几秒钟不走 clangd。** clangd 从磁盘上的文本读取一个文件的 import，而不是从编辑器里读（0.0.4 及更早版本会就此永久卡住：`import hello.` 被自动保存后，这个文件的所有请求都得不到应答）。一次保存把 clangd 会卡住的内容写到磁盘上时——以 `.` 结尾的模块名，或者 import 了项目里（还）没有的模块——这个文件改由 mcppls 自己的引擎应答，直到再次保存；状态里以 `file-unsafe-on-disk`（类别 `code`）列出它并说明原因，状态仍是 *ready*。没有任何单元提供的模块会在保存后一秒内得到替身单元，等 clangd 读到带替身的数据库（大约六秒后），文件就交还给 clangd。 + +**“Import directive must end with a ';'” 标在了别的行上，或刚输入的 import 报 “module X not found”。** clangd 把缺少 `;` 的指令报在它后面的代码上；mcppls 会把这条诊断移回指令所在行（规避措施 `WA-CLANGD-006`）。刚输入、还没保存的 import，clangd 要等文件保存后才会构建（它从磁盘读取 import）；只要这个模块在项目里，这时给出的是信息级提示 “module 'X' is in the project; clangd loads it once the file is saved”，而不是错误（`WA-CLANGD-007`）。 + **“clangd would not finish main.cpp”。** 某个文件的构建超出了预算——该文件上次构建耗时的五倍，最少 20 秒——而编辑器还在等它：不管 clangd 忙不忙，它都不会完成这个文件了。这个文件改由 mcppls 自己的引擎应答（提供模块层面的功能），直到它的文本发生变化（让 clangd 卡住的那份文本永远不会再交给它），同时立即重启一个不带这个文件的 clangd。`events` 日志里有一条带具体数字的 `engine-spin`。 **某个规避措施还需要吗？** `--disable-workaround WA-CLANGD-`（可重复）可以关掉一个；日志开头几行会列出正在使用的规避措施。每个规避措施在一致性测试里都有一个对应的检测项（`workaround-canaries`），clangd 更新修好了对应缺陷后，这个检测项就会失败。 @@ -43,7 +68,19 @@ **每次启动都很慢。** 第二次会话应该很快：模型连同构建工具所读一切内容的指纹一起被缓存，与之匹配的会话会立即套用计划，并在后台确认。`project.firstOrigin` 会说明发生了哪种情况。如果它一直是 `producer`，说明指纹没有匹配上——该看报告里的 `project.producerRun` 和构建文件的时间戳。 -**clangd 反复重启。** 看 `engines[].restarts` 和 `events` 日志。重启之间会拉开间隔，而且十分钟内最多三次；超过之后状态会说明（`engine-restart-capped`），原本想靠重启解决的部分改由 mcppls 自己的引擎应答。模块编译不过从来不是重启的理由。如果重启密集出现，通常说明编译参数一直在变。 +**clangd 反复重启。** 看 `engines[].restarts`、`engines[].details.restartBudget` 和 `events` 日志。每种原因各有十分钟三次的重启额度：引擎数据库变化（`plan`）、恢复停止应答或空转的 clangd（`recovery`）、clangd 退出（`crash`）。用完之后，同类的下一次重启依次等待一、二、四、八分钟——是退避而不是拒绝，所以卡住的 clangd 总能恢复——状态会说明（`engine-restart-capped`）并提供 **Restart clangd** 按钮（其他编辑器用 `workspace/executeCommand` `mcppls.restartEngine`），它立即重启且从不计入额度。切换工具链、profile 或 context 也从不计入，模块编译不过从来不是重启的理由。引擎数据库的每次变化都会记入日志并写明改了什么（`engine database changed: … compiled otherwise (main.cpp: argument 3: -O0 -> -O2)`），重启密集时能直接看到原因。 + +**“clangd crashed while building NormalJsonTranslator.Core.cpp”。** clangd 会说明它在哪个文件上崩溃（崩溃上下文），隔离的就是这个文件：它改由 mcppls 自己的引擎应答，同时重启一个不带它的 clangd。报告里的 `engines[].details.lastExit` 有退出码、文件、clangd 当时在做什么，Windows 上还有异常码。五分钟内退出五次，clangd 在下次服务启动前不再使用；状态会提供 **Export Diagnostic Bundle**。 + +**“mcpp could not describe tools/updater/mcpp.toml”。** 构建工具描述了工作区的其余部分，并说明了它没能描述的那一部分（例如某个成员的构建程序失败）；那部分的文件按其余部分提供的信息来读，其余部分照常工作（`producer-partial`，S2 0.3.0）。修好消息里指出的问题，下次重新加载就会一并描述它。 + +**“clangd rejected the compile command for module scanning”。** clangd 在构建模块之前，会用数据库里每个单元自己的编译命令扫描它的 import；编译器驱动拒绝的命令会让扫描失败，模块也就一个都不会构建——issue #23 的 `LTO requires -fuse-ld=lld` 就是这种情况。状态会用驱动的原话写出第一处拒绝（类别 `environment`），`engines[].details.scanFailures` 记录次数。命令找不到头文件时也这样说明（类别 `project`）。正在输入的文件扫描失败是常态，只计数。 + +**“C++26 was disabled in precompiled file”。** 某个模块用一种 C++ 标准构建，却在另一种标准下被导入；clang 会拒绝。mcppls 对同一上下文中的模块单元统一按其中最新的标准来读（报告里的 `plan.languageStandard`、状态 profile 里的 `standard`），所以这条错误只应来自 0.0.5 之前 clangd 构建的模块（下次改动时会重建），或者构建本身就混用了标准——那样构建工具自己的编译器也会拒绝。 + +**刚打开项目时，最长一分钟内只有模块层面的功能。** 识别出构建系统的项目，要等构建工具描述完项目（最长一分钟，即构建工具自身的时限）才把它交给 clangd，而不是先给 clangd 一个从源码猜出来、之后还要推翻的模型；这期间由 mcppls 自己的引擎提供模块跳转、import 上的悬停和 `import` 补全。第二次会话会直接从缓存的模型开始。 + +**服务端把出错的现场记在哪里。** 崩溃、clangd 卡住或空转、文件被隔离、重启被推迟、某个规避措施的前提被发现不成立，每一种都会留下一份“事故”：工作区缓存下的一个目录（`incidents/-<类型>/`，保留最近二十份、一周内），里面有事发前的经过、clangd 最近的日志（clangd 以 `info` 级别记录到内存，从不写进默认日志）、每个相关文件在编辑器与磁盘上不同的那几行，以及 clangd 哪个线程在占用 CPU。诊断包会带上它们。 **“clangd stopped making progress; it was restarted”。** clangd 有请求一直没答，期间也没答任何别的请求，并且五秒内几乎没用 CPU：它在等一个不会来的东西，而不是在编译（编译会一直占着一个核，这种情况不会被打断）。`events` 日志里有一条带具体数字的 `engine-stuck`。已经观察到 clangd 23.1 在某个模块的源文件一秒内被改两次之后出现这种情况。在 Windows 上服务端读不到 clangd 的 CPU 时间，所以检测不到；clangd 不再应答的文件仍会被逐个搁置。 @@ -51,4 +88,4 @@ ## 提交 bug 报告 -附上诊断报告。它会写出你机器上的路径，所以先读一遍再附——按设计，它不包含任何环境变量的值，也不包含文件内容。 +附上问题包（**C++ Modules: Export Diagnostic Bundle**，或 `mcppls report --bundle`），至少也附上诊断报告。两者都已替换你的用户名、主目录、主机名和密钥，也都不含你的文件内容；附上之前仍请先看一遍。 diff --git a/editors/claude-code/.claude-plugin/marketplace.json b/editors/claude-code/.claude-plugin/marketplace.json index bbe46bf..96be841 100644 --- a/editors/claude-code/.claude-plugin/marketplace.json +++ b/editors/claude-code/.claude-plugin/marketplace.json @@ -11,7 +11,7 @@ "displayName": "C++ Modules Language Server", "source": "./mcppls-lsp", "description": "Registers mcppls as the language server for C and C++ sources, including C++20/23 named modules.", - "version": "0.0.4", + "version": "0.0.5", "author": { "name": "Sunrisepeak", "url": "https://github.com/Sunrisepeak/mcpp-language-server" diff --git a/editors/claude-code/mcppls-lsp/.claude-plugin/plugin.json b/editors/claude-code/mcppls-lsp/.claude-plugin/plugin.json index 172091e..c68b430 100644 --- a/editors/claude-code/mcppls-lsp/.claude-plugin/plugin.json +++ b/editors/claude-code/mcppls-lsp/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "mcppls-lsp", "displayName": "C++ Modules Language Server", - "version": "0.0.4", + "version": "0.0.5", "description": "Registers mcppls as the language server for C and C++ sources, including C++20/23 named modules, and its MCP tools (symbols, references, modules, verification, review). Replaces clangd-lsp for a project; do not enable both at once.", "author": { "name": "Sunrisepeak", diff --git a/editors/clion/gradle.properties b/editors/clion/gradle.properties index 351542a..90e75c0 100644 --- a/editors/clion/gradle.properties +++ b/editors/clion/gradle.properties @@ -2,7 +2,7 @@ # ones within the same major line; sinceBuild/untilBuild in plugin.xml is what actually gates it. platformType = CL platformVersion = 2025.2 -pluginVersion = 0.0.4 +pluginVersion = 0.0.5 org.gradle.jvmargs = -Xmx2g # The IDE ships the Kotlin standard library; bundling a second copy in the plugin is what JetBrains # asks plugins not to do. diff --git a/editors/nvim/README.md b/editors/nvim/README.md index 27368f0..aecebba 100644 --- a/editors/nvim/README.md +++ b/editors/nvim/README.md @@ -113,6 +113,15 @@ vim.api.nvim_set_hl(0, '@lsp.mod.partition', { italic = true }) -- only the par vim.api.nvim_set_hl(0, '@lsp.type.keyword', { link = 'Keyword' }) -- import / module / export ``` +## Completing `import` + +The server completes module names after `import ` and offers the module keywords (`import`, +`export import`, `module;`, `export module`, `module`, `module :private;`) where each can begin a +declaration. It does not make a space a trigger character for Neovim, since every space typed +would then ask for a completion; to have the module list open on the space after `import`, ask for +it with `init_options = { completion = { triggerOnSpace = true } }` — the server answers the other +spaces with nothing, at once. + ## Commands and statusline | Command | | diff --git a/editors/vscode/README.md b/editors/vscode/README.md index 1d6e182..32dbc2b 100644 --- a/editors/vscode/README.md +++ b/editors/vscode/README.md @@ -35,7 +35,9 @@ Customize the color the standard way: `editor.semanticTokenColorCustomizations.r | C++ Modules: Restart Language Server | Restart the server and clangd | | C++ Modules: Show Logs | Open the log | | C++ Modules: Install Command Line Tools | Run `xcode-select --install` (macOS only) | -| C++ Modules: Collect Diagnostic Report | Open the server's status and this extension's version, settings and other installed C++ extensions as JSON, ready to copy or attach to an issue | +| C++ Modules: Collect Diagnostic Report | Open the server's status and this extension's version, settings and other installed C++ extensions as JSON, ready to copy or attach to an issue; your user name, home directory, host name and secrets are replaced | +| C++ Modules: Export Diagnostic Bundle | Write one zip with the report, the environment, the logs of the last sessions, the incidents and the engine database, redacted the same way and checked before it is written; never uploaded | +| C++ Modules: Restart clangd | Restart clangd alone, now, whatever its restart budget says | | C++ Modules: Run the Build Tool in a Terminal | Run the project's build command (`mcpp build` or the CMake configure step) in your own terminal, where a proxy or credentials you set by hand actually are | | C++ Modules: Turn Off Other C++ Language Features | Turn off the language features of other active C++ extensions, in this workspace or everywhere (user settings) | | C++ Modules: Restore Other C++ Language Features | Put back whatever the command above (or the one-time question) last changed, in the same scope | @@ -59,6 +61,7 @@ All settings are optional. | `mcppls.ai.enabled` | `false` | Show the review commands | | `mcppls.detectConflicts` | `true` | Offer once to turn off other C++ extensions' language features in the workspace, and notice again if one becomes active later | | `mcppls.semanticTokens.modules` | `true` | Module keywords and names from the server's semantic tokens; turn off to use only your own grammar or tree-sitter colors for module syntax | +| `mcppls.completion.triggerOnSpace` | `true` | Show the module list as soon as a space is typed after `import` or `export import`; a space anywhere else never reaches the server | | `mcppls.trace.server` | `off` | Trace the language server protocol in the log | | `mcppls.buildTool` | `offline` | How mcppls may run the project's build tool (mcpp, CMake) to learn how it is built: `offline` runs it without the network, offering to run it in a terminal when it needs a download; `online` lets it reach the network, with up to ten minutes; `off` never runs it, using only the cache or scanned sources | | `mcppls.toolEnvironment` | `auto` | Which environment build tools are started in: `auto` reads the login shell's environment once in the background on Linux and macOS (Windows always matches the editor); `editor` always uses the editor process's own environment | diff --git a/editors/vscode/package-lock.json b/editors/vscode/package-lock.json index e714e74..834fca4 100644 --- a/editors/vscode/package-lock.json +++ b/editors/vscode/package-lock.json @@ -1,12 +1,12 @@ { "name": "mcpp-language-server", - "version": "0.0.4", + "version": "0.0.5", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "mcpp-language-server", - "version": "0.0.4", + "version": "0.0.5", "license": "Apache-2.0", "dependencies": { "vscode-languageclient": "^10.1.1" diff --git a/editors/vscode/package.json b/editors/vscode/package.json index 3dfda9c..db0c7bb 100644 --- a/editors/vscode/package.json +++ b/editors/vscode/package.json @@ -2,7 +2,7 @@ "name": "mcpp-language-server", "displayName": "C++ Modules Language Server", "description": "C++20/23 named modules that just work: go to definition, completion, hover and references across modules for any compiler, with clangd and a standard library kit built in. (mcppls)", - "version": "0.0.4", + "version": "0.0.5", "publisher": "sunrisepeak", "license": "Apache-2.0", "icon": "icon.png", @@ -155,6 +155,16 @@ "title": "Collect Diagnostic Report", "category": "C++ Modules" }, + { + "command": "mcppls.exportDiagnosticBundle", + "title": "Export Diagnostic Bundle", + "category": "C++ Modules" + }, + { + "command": "mcppls.restartClangd", + "title": "Restart clangd", + "category": "C++ Modules" + }, { "command": "mcppls.runBuildToolInTerminal", "title": "Run the Build Tool in a Terminal", @@ -221,6 +231,11 @@ "default": true, "description": "Module keywords and names from the server's semantic tokens. Turn this off to use only your own grammar or tree-sitter colors for module syntax." }, + "mcppls.completion.triggerOnSpace": { + "type": "boolean", + "default": true, + "markdownDescription": "Show the module list as soon as a space is typed after `import` or `export import`. A space anywhere else never asks the server for anything." + }, "mcppls.trace.server": { "type": "string", "enum": [ diff --git a/editors/vscode/src/commands.ts b/editors/vscode/src/commands.ts index 0f74a60..059c9ef 100644 --- a/editors/vscode/src/commands.ts +++ b/editors/vscode/src/commands.ts @@ -1,8 +1,12 @@ -// The commands: select context, show module graph, restart, show logs, collect a diagnostic report. +// The commands: select context, show module graph, restart, show logs, collect a diagnostic report, +// export a diagnostic bundle, restart clangd. +import * as os from 'os'; import * as vscode from 'vscode'; import type { LanguageClient } from 'vscode-languageclient/node'; +import { SETTABLE_CANDIDATES, UNSETTABLE_CANDIDATES } from './conflictCandidates'; import { restoreOtherCppFeatures, turnOffOtherCppFeatures } from './conflicts'; +import { redactJson, Who } from './redact'; import { describeProfile, SemanticProfile } from './status'; export interface ServerAccess { @@ -11,6 +15,18 @@ export interface ServerAccess { restart(): Promise; showLogs(): void; log(line: string): void; + // The extension's own latest log lines, oldest first (the server keeps its own log in a file). + recentLog(): string[]; +} + +function whoAmI(): Who { + let user = ''; + try { + user = os.userInfo().username; + } catch { + // No user database entry: the home directory still goes. + } + return { home: os.homedir(), user }; } interface ProtocolRange { @@ -158,29 +174,65 @@ function withTimeout(promise: Thenable, milliseconds: number, what: string }); } +function extensionVersion(): string | undefined { + const extension = vscode.extensions.getExtension('sunrisepeak.mcpp-language-server'); + return (extension?.packageJSON as { version?: string } | undefined)?.version; +} + +function mcpplsSettings(): Record { + const settings = vscode.workspace.getConfiguration('mcppls'); + return { + compiler: settings.get('compiler'), + semanticKit: settings.get('semanticKit'), + engine: settings.get('engine'), + buildTool: settings.get('buildTool'), + toolEnvironment: settings.get('toolEnvironment'), + semanticTokensModules: settings.get('semanticTokens.modules'), + traceServer: settings.get('trace.server'), + aiEnabled: settings.get('ai.enabled'), + detectConflicts: settings.get('detectConflicts'), + }; +} + +// The other C/C++ extensions a report of a problem needs to know about: installed and enabled, active, +// and for those with a setting for it, whether their language features are turned off. +function otherCppExtensions(): Record[] { + const described: Record[] = []; + const describe = (extensionId: string, featuresOff?: boolean) => { + const extension = vscode.extensions.getExtension(extensionId); + if (!extension) return; + described.push({ + id: extensionId, + version: (extension.packageJSON as { version?: string } | undefined)?.version, + active: extension.isActive, + ...(featuresOff === undefined ? {} : { languageFeaturesOff: featuresOff }), + }); + }; + for (const candidate of SETTABLE_CANDIDATES) { + describe(candidate.extensionId, vscode.workspace.getConfiguration(candidate.section).get(candidate.key) === candidate.disabledValue); + } + for (const candidate of UNSETTABLE_CANDIDATES) { + describe(candidate.extensionId); + } + describe('mcpp-community.mcpp-vscode'); + return described; +} + // robustness design O3: what a bug report needs, in one document a person can read, copy or save. The // server's part (cxxModules/report) comes with the extension's own: versions, settings, other C++ extensions. +// The server redacts its part (S3-5.5-3); the extension's own is redacted here by the same rules. async function collectReport(access: ServerAccess): Promise { const client = access.runningClient(); - const extension = vscode.extensions.getExtension('sunrisepeak.mcpp-language-server'); - const settings = vscode.workspace.getConfiguration('mcppls'); - const report: Record = { + const report: Record = redactJson({ extension: { - version: (extension?.packageJSON as { version?: string } | undefined)?.version, + version: extensionVersion(), vscode: vscode.version, platform: `${process.platform}-${process.arch}`, - otherCppExtensions: ['ms-vscode.cpptools', 'llvm-vs-code-extensions.vscode-clangd', 'mcpp-community.mcpp-vscode'] - .filter((id) => vscode.extensions.getExtension(id) !== undefined), - settings: { - compiler: settings.get('compiler'), - semanticKit: settings.get('semanticKit'), - engine: settings.get('engine'), - aiEnabled: settings.get('ai.enabled'), - detectConflicts: settings.get('detectConflicts'), - }, + otherCppExtensions: otherCppExtensions(), + settings: mcpplsSettings(), }, workspaceFolders: (vscode.workspace.workspaceFolders ?? []).map((folder) => folder.uri.fsPath), - }; + }, whoAmI()); if (!client) { report.server = 'not running'; } else if (!declaresModules(client.initializeResult?.capabilities)) { @@ -197,15 +249,95 @@ async function collectReport(access: ServerAccess): Promise { const document = await vscode.workspace.openTextDocument({ language: 'json', content }); await vscode.window.showTextDocument(document, { preview: false }); const choice = await vscode.window.showInformationMessage( - 'C++ Modules: the diagnostic report is open. It names paths on this machine; attach it to an issue as it is or after editing.', - 'Copy to Clipboard', 'Show Logs'); + 'C++ Modules: the diagnostic report is open. Your user name, home directory, host name and anything that looks like a ' + + 'secret were replaced; the project\'s own paths are kept. Export Diagnostic Bundle packs it with the logs and the environment.', + 'Copy to Clipboard', 'Export Diagnostic Bundle', 'Show Logs'); if (choice === 'Copy to Clipboard') { await vscode.env.clipboard.writeText(content); + } else if (choice === 'Export Diagnostic Bundle') { + await exportDiagnosticBundle(access); } else if (choice === 'Show Logs') { access.showLogs(); } } +interface BundleWritten { + path: string; + bytes: number; + redactions?: Record; +} + +function sizeText(bytes: number): string { + return bytes >= 1024 * 1024 ? `${(bytes / (1024 * 1024)).toFixed(1)} MB` : `${Math.max(1, Math.round(bytes / 1024))} KB`; +} + +// Issue #23 fix plan F18: one zip with what a report of a problem needs -- the server's report, the +// environment, the logs of the last sessions, the incidents, the engine databases -- written by the +// server with user names, paths and secrets replaced, and never uploaded. When the server's check +// finds something its rules left, nothing is written, and hiding the project's paths too is offered. +export async function exportDiagnosticBundle(access: ServerAccess, hideProjectPaths = false): Promise { + const client = access.runningClient(); + if (!client) { + void vscode.window.showWarningMessage( + 'C++ Modules: the language server is not running. `mcppls report --bundle ` in a terminal writes the same bundle.'); + return; + } + const argument = { + hideProjectPaths, + client: { + extension: { version: extensionVersion(), vscode: vscode.version, platform: `${process.platform}-${process.arch}`, remote: vscode.env.remoteName ?? null }, + otherCppExtensions: otherCppExtensions(), + settings: mcpplsSettings(), + log: access.recentLog().join('\n'), + }, + }; + let written: BundleWritten; + try { + written = await vscode.window.withProgress( + { location: vscode.ProgressLocation.Notification, title: 'C++ Modules: writing a diagnostic bundle' }, + () => withTimeout(client.sendRequest('workspace/executeCommand', { command: 'mcppls.exportBundle', arguments: [argument] }), + 120000, 'mcppls.exportBundle')); + } catch (error) { + const message = errorText(error); + access.log(`mcppls.exportBundle failed: ${message}`); + const offer = hideProjectPaths ? [] : ['Retry with Project Paths Hidden']; + const choice = await vscode.window.showWarningMessage(`C++ Modules: no diagnostic bundle was written. ${message}`, ...offer, 'Show Logs'); + if (choice === 'Retry with Project Paths Hidden') { + await exportDiagnosticBundle(access, true); + } else if (choice === 'Show Logs') { + access.showLogs(); + } + return; + } + access.log(`diagnostic bundle written: ${written.path} (${written.bytes} bytes)`); + const choice = await vscode.window.showInformationMessage( + `C++ Modules: diagnostic bundle written (${sizeText(written.bytes)}). Your user name, home directory, host name and secrets were ` + + 'replaced; nothing was uploaded. Attach it to an issue if you choose to.', + 'Reveal in Folder', 'Copy Path'); + if (choice === 'Reveal in Folder') { + await vscode.commands.executeCommand('revealFileInOS', vscode.Uri.file(written.path)); + } else if (choice === 'Copy Path') { + await vscode.env.clipboard.writeText(written.path); + } +} + +// Issue #23 fix plan F14: clangd restarted now, whatever its restart budget says; the server counts it +// as the user's own restart, not against that budget. Nothing else about the session changes. +async function restartClangd(access: ServerAccess): Promise { + const client = access.runningClient(); + if (!client) { + void vscode.window.showWarningMessage('C++ Modules: the language server is not running.'); + return; + } + try { + await client.sendRequest('workspace/executeCommand', { command: 'mcppls.restartEngine', arguments: [] }); + access.log('clangd restart requested'); + } catch (error) { + access.log(`mcppls.restartEngine failed: ${errorText(error)}`); + void vscode.window.showWarningMessage(`C++ Modules: clangd could not be restarted: ${errorText(error)}`); + } +} + // Build description design 4.4. The server runs the build tool offline, so a project whose // dependencies are not on the machine yet cannot be described without a download. That download is // the user's to start, and it is offered in their own terminal for a reason: a proxy set by hand in @@ -276,6 +408,8 @@ export function registerCommands(context: vscode.ExtensionContext, access: Serve vscode.commands.registerCommand('mcppls.restartServer', () => access.restart()), vscode.commands.registerCommand('mcppls.showLogs', () => access.showLogs()), vscode.commands.registerCommand('mcppls.collectReport', () => collectReport(access)), + vscode.commands.registerCommand('mcppls.exportDiagnosticBundle', () => exportDiagnosticBundle(access)), + vscode.commands.registerCommand('mcppls.restartClangd', () => restartClangd(access)), vscode.commands.registerCommand('mcppls.runBuildToolInTerminal', () => runBuildToolInTerminal(access)), vscode.commands.registerCommand('mcppls.turnOffOtherCppFeatures', () => turnOffOtherCppFeatures(context, access.log)), vscode.commands.registerCommand('mcppls.restoreOtherCppFeatures', () => restoreOtherCppFeatures(context, access.log)), diff --git a/editors/vscode/src/completionGate.ts b/editors/vscode/src/completionGate.ts new file mode 100644 index 0000000..dab3246 --- /dev/null +++ b/editors/vscode/src/completionGate.ts @@ -0,0 +1,19 @@ +// Fix plan 2026-09-26 F9 (decision D4, layer 1): the server takes a space as a completion trigger +// character, for the module list after `import`. VS Code would then ask on every space typed +// anywhere; this is what drops those requests before anything is sent. The same rule as the +// server's own gate (src/orchestrator/completion.cppm, is_import_line_prefix), in plain TypeScript +// so it runs without VS Code in the unit tests. + +// `^\s*(export\s+)?import\s$`: an import directive's keyword and exactly one blank after it, with +// nothing typed after that. +const IMPORT_LINE_PREFIX = /^[ \t\v\f]*(?:export[ \t\v\f]+)?import[ \t\v\f]$/; + +export function isImportLinePrefix(prefix: string): boolean { + return IMPORT_LINE_PREFIX.test(prefix); +} + +// Whether a completion VS Code asks for because `triggerCharacter` was typed at `character` of a +// line should reach the server. Only a space is ever dropped. +export function sendTriggeredCompletion(triggerCharacter: string | undefined, lineText: string, character: number): boolean { + return triggerCharacter !== ' ' || isImportLinePrefix(lineText.slice(0, character)); +} diff --git a/editors/vscode/src/extension.ts b/editors/vscode/src/extension.ts index f2b52f3..7e5dd40 100644 --- a/editors/vscode/src/extension.ts +++ b/editors/vscode/src/extension.ts @@ -26,6 +26,7 @@ import { } from 'vscode-languageclient/node'; import { CommandLineToolsController, withInstallCommandFallback } from './commandLineTools'; import { registerCommands, reloadBuildDescription } from './commands'; +import { sendTriggeredCompletion } from './completionGate'; import { checkConflicts, ConflictCheck, watchForNewConflicts } from './conflicts'; import { resolveLaunch } from './payload'; import { ServerLogLevel, ServerLogRouter } from './serverLog'; @@ -42,6 +43,7 @@ const CLIENT_ID = 'mcppls'; const CLIENT_NAME = 'C++ Modules'; const RESTART_WINDOW_MS = 3 * 60 * 1000; const MAX_RESTARTS = 4; +const RECENT_LOG_LINES = 1000; export interface TestApi { waitForState(state: ModuleState | readonly ModuleState[], timeoutMs: number): Promise; @@ -119,6 +121,7 @@ class ServerHost implements vscode.Disposable { readonly serverLog = new ServerLogRouter(); private restarts: number[] = []; private queue: Promise = Promise.resolve(); + private readonly recent: string[] = []; constructor( private readonly context: vscode.ExtensionContext, @@ -144,7 +147,18 @@ class ServerHost implements vscode.Disposable { } log(line: string): void { - this.output().appendLine(`[${new Date().toLocaleTimeString()}] ${line}`); + const stamped = `[${new Date().toLocaleTimeString()}] ${line}`; + this.output().appendLine(stamped); + // A diagnostic bundle carries the extension's own log (issue #23 fix plan F18); the output + // channel cannot be read back, so its latest lines are kept here too. + this.recent.push(stamped); + if (this.recent.length > RECENT_LOG_LINES) { + this.recent.splice(0, this.recent.length - RECENT_LOG_LINES); + } + } + + recentLog(): string[] { + return [...this.recent]; } runningClient(): LanguageClient | undefined { @@ -259,6 +273,20 @@ class ServerHost implements vscode.Disposable { modules: configuration.get('semanticTokens.modules', true), moduleType: true, }, + // Fix plan 2026-09-26 F9: a space after `import` opens the module list. The server + // advertises the space as a trigger character to this client unless this is off. + completion: { + triggerOnSpace: configuration.get('completion.triggerOnSpace', true), + }, + }, + middleware: { + // Fix plan 2026-09-26 F9 (D4 layer 1): of the completions a typed space asks for, only + // the one after `import` or `export import` is sent; every other is answered here, with + // nothing, before it costs a message. + provideCompletionItem: (document, position, context, token, next) => + sendTriggeredCompletion(context.triggerCharacter, document.lineAt(position.line).text, position.character) + ? next(document, position, context, token) + : [], }, errorHandler: { error: () => ({ action: ErrorAction.Continue, handled: true }), @@ -489,6 +517,7 @@ export function activate(context: vscode.ExtensionContext): TestApi { restart: () => host.restart(), showLogs: () => host.output().show(true), log: (line: string) => host.log(line), + recentLog: () => host.recentLog(), }; registerCommands(context, serverAccess); @@ -496,7 +525,8 @@ export function activate(context: vscode.ExtensionContext): TestApi { vscode.workspace.onDidChangeConfiguration((event) => { if (event.affectsConfiguration('mcppls.compiler') || event.affectsConfiguration('mcppls.semanticKit') || event.affectsConfiguration('mcppls.engine') || event.affectsConfiguration('mcppls.buildTool') - || event.affectsConfiguration('mcppls.toolEnvironment') || event.affectsConfiguration('mcppls.semanticTokens.modules')) { + || event.affectsConfiguration('mcppls.toolEnvironment') || event.affectsConfiguration('mcppls.semanticTokens.modules') + || event.affectsConfiguration('mcppls.completion.triggerOnSpace')) { void host.restart(); } }), diff --git a/editors/vscode/src/redact.ts b/editors/vscode/src/redact.ts new file mode 100644 index 0000000..dfb4fb3 --- /dev/null +++ b/editors/vscode/src/redact.ts @@ -0,0 +1,58 @@ +// The extension's own part of a diagnostic report (workspace folders, versions, settings), redacted +// the way the server redacts its part (issue #23 fix plan F18, S3-5.5-3): the home directory as `~`, +// the user's name as ``. The server's report and bundle go through the server's own rules; +// this covers only the few fields the extension adds itself. Imports nothing from VS Code, so the +// plain-Node unit tests can load it. + +export interface Who { + home: string; // os.homedir() + user: string; // os.userInfo().username +} + +// Names too short or too common to replace wherever they stand ("runner", "admin"): replaced only +// where they name a directory, the same rule as the server's. +const COMMON = new Set(['admin', 'administrator', 'user', 'users', 'guest', 'root', 'test', 'runner', 'build', 'developer', 'home', 'public', + 'default', 'shared', 'ubuntu', 'docker', 'vagrant', 'owner', 'work', 'workspace', 'code', 'server', 'client', 'data', 'temp', 'demo']); + +export function distinctiveName(name: string): boolean { + return name.length >= 4 && !COMMON.has(name.toLowerCase()); +} + +function escapeRegExp(text: string): string { + return text.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + +// The home directory with either separator, doubled backslashes (JSON) and either case, as a whole +// path: "/home/speak" is not in "/home/speaker". +function homePattern(home: string): RegExp | undefined { + const segments = home.split(/[\\/]+/).filter((segment) => segment.length > 0); + if (segments.length === 0 || (segments.length === 1 && /^[A-Za-z]:$/.test(segments[0]))) { + return undefined; + } + const drive = /^[A-Za-z]:$/.test(segments[0]); + const body = segments.map(escapeRegExp).join('[\\\\/]+'); + const source = drive ? body : `[\\\\/]+${body}`; + return new RegExp(`${source}(?![A-Za-z0-9_])`, 'gi'); +} + +export function redactText(text: string, who: Who): string { + let out = text; + const home = homePattern(who.home); + if (home) { + out = out.replace(home, '~'); + } + if (who.user.length > 0) { + const name = escapeRegExp(who.user); + const pattern = distinctiveName(who.user) + ? new RegExp(`(?'); + } + return out; +} + +// A JSON value redacted through its text; its structure is kept, since no placeholder has a quote +// or a backslash. +export function redactJson(value: T, who: Who): T { + return JSON.parse(redactText(JSON.stringify(value), who)) as T; +} diff --git a/editors/vscode/src/status.ts b/editors/vscode/src/status.ts index a471cf7..5889cfd 100644 --- a/editors/vscode/src/status.ts +++ b/editors/vscode/src/status.ts @@ -88,7 +88,9 @@ function barFor(state: ModuleState | 'starting', detail: string | undefined): const PULSE_MS = 900; const SHOW_LOGS: vscode.Command = { title: 'Show Logs', command: 'mcppls.showLogs' }; -const COLLECT_REPORT: vscode.Command = { title: 'Collect Report', command: 'mcppls.collectReport' }; +// Issue #23 fix plan F18: what a limited state without a fix of its own offers is the bundle a report of the +// problem needs, which also carries the report. +const EXPORT_BUNDLE: vscode.Command = { title: 'Export Diagnostic Bundle', command: 'mcppls.exportDiagnosticBundle' }; const RESTART: vscode.Command = { title: 'Restart', command: 'mcppls.restartServer' }; const BUSY_STATES: readonly ModuleState[] = ['starting', 'loading', 'preparing']; @@ -240,10 +242,10 @@ export class StatusController implements vscode.Disposable { ? vscode.LanguageStatusSeverity.Warning : vscode.LanguageStatusSeverity.Information; const withCommand = issues.find((issue) => issue.command !== undefined); - // A limited state without a fix of its own offers the report a bug report needs (robustness design O4). + // A limited state without a fix of its own offers what a bug report needs (robustness design O4). this.item.command = withCommand?.command ? { title: withCommand.command.title, command: withCommand.command.command, arguments: withCommand.command.arguments } - : status.state === 'degraded' || status.state === 'error' ? COLLECT_REPORT : SHOW_LOGS; + : status.state === 'degraded' || status.state === 'error' ? EXPORT_BUNDLE : SHOW_LOGS; // The status bar says the one thing that matters now, shortened when there is a fuller // version in the tooltip; the item behind `{}` keeps the rest. diff --git a/editors/vscode/syntaxes/mcppls-modules.tmLanguage.json b/editors/vscode/syntaxes/mcppls-modules.tmLanguage.json index c520df1..79be5cf 100644 --- a/editors/vscode/syntaxes/mcppls-modules.tmLanguage.json +++ b/editors/vscode/syntaxes/mcppls-modules.tmLanguage.json @@ -1,12 +1,15 @@ { "$schema": "https://raw.githubusercontent.com/martinring/tmlanguage/master/tmlanguage.json", "name": "mcppls C++ Modules", - "comment": "WA-VSCODE-001 (see src/workarounds.ts): VS Code's built-in cpp grammar defines a module_import rule but nothing includes it, so import/module/export and module names get no scope on their own. This injection grammar covers only the module syntax the built-in grammar misses; its capture-group scopes match mcpp-community.mcpp-vscode's own module grammar exactly, so installing both extensions is harmless. Anchored at the start of the line (leading whitespace allowed) so 'x = import;' and 'obj.import(1)' are never matched -- only a line that itself begins with (an optional 'export' then) 'module' or 'import' is. Each pattern accepts an incomplete, still-being-typed name (for example 'import hello.' colors 'import' and 'hello', leaving the trailing dot unscoped) because nothing here requires a trailing ';' or an end of line.", + "comment": "WA-VSCODE-001 (see src/workarounds.ts): VS Code's built-in cpp grammar defines a module_import rule but nothing includes it, so import/module/export and module names get no scope on their own. This injection grammar covers only the module syntax the built-in grammar misses; its capture-group scopes match mcpp-community.mcpp-vscode's own module grammar exactly, so installing both extensions is harmless. Anchored at the start of the line (leading whitespace allowed) so 'x = import;' and 'obj.import(1)' are never matched -- only a line that itself begins with (an optional 'export' then) 'module' or 'import', or with the 'export' of an export declaration, is. Each pattern accepts an incomplete, still-being-typed name (for example 'import hello.' colors 'import' and 'hello', leaving the trailing dot unscoped) because nothing here requires a trailing ';' or an end of line. It also colors the C++26 contract keywords the built-in grammar does not know yet (contract_assert, and pre and post as contract specifiers).", "scopeName": "source.cpp.mcppls-modules", "injectionSelector": "L:source.cpp", "patterns": [ { "include": "#module-declaration" }, - { "include": "#import-declaration" } + { "include": "#import-declaration" }, + { "include": "#export-declaration" }, + { "include": "#contract-assert" }, + { "include": "#contract-specifier" } ], "repository": { "module-declaration": { @@ -34,6 +37,27 @@ "6": { "name": "string.quoted.other.header.cpp" }, "7": { "name": "string.quoted.double.header.cpp" } } + }, + "contract-assert": { + "comment": "C++26 contracts (P2900): contract_assert is a keyword, which VS Code's built-in grammar does not know yet. Colored like static_assert. The leading blanks are part of the match: the built-in grammar's function-call rule starts at them, and a match that starts where it does wins, being injected on the left.", + "match": "\\s*\\b(contract_assert)\\b", + "captures": { + "1": { "name": "keyword.other.contract_assert.cpp" } + } + }, + "contract-specifier": { + "comment": "C++26 contracts (P2900): pre and post are keywords only as a function's contract specifiers -- after its declarator (a closing parenthesis, a cv, ref or noexcept specifier, override or final) and before their own parenthesis. Anywhere else they are ordinary names (`int pre(int)` declares a function called pre), and after a trailing return type they are left uncolored rather than guessed.", + "match": "(?<=\\)|\\bconst|\\bvolatile|\\bnoexcept|\\boverride|\\bfinal|&)\\s+(pre|post)(?=\\s*\\()", + "captures": { + "1": { "name": "keyword.other.contract.cpp" } + } + }, + "export-declaration": { + "comment": "export namespace | export { | export int f() ... -- the export of an export declaration, colored like export module's (fix plan F8). Anchored at the start of the line like the rules above, and never before module or import, which those color.", + "match": "^\\s*(export)\\b(?!\\s*(?:module|import)\\b)(?=\\s*(?:\\{|[A-Za-z_]))", + "captures": { + "1": { "name": "keyword.control.export.cpp" } + } } } } diff --git a/editors/vscode/test/suite/grammar.test.ts b/editors/vscode/test/suite/grammar.test.ts index bca6407..52679c7 100644 --- a/editors/vscode/test/suite/grammar.test.ts +++ b/editors/vscode/test/suite/grammar.test.ts @@ -81,9 +81,31 @@ suite('module-syntax highlighting: the injected grammar (WA-VSCODE-001)', functi 'x = import;', 'obj.import(1);', 'int module = 5;', + // Fix plan F8: the export of an export declaration. + 'export namespace ns {', + ' export int inner();', + '}', + 'export {', + '}', + 'export template T g(T);', + 'exports = 1;', + // C++26 contracts (P2900). + 'int f(int x) pre(x > 0) post(r: r > 0);', + 'void h() const noexcept pre(ok());', + 'void k() { contract_assert(ready); }', + 'int pre(int post);', ]); }); + test('the export of an export declaration is colored like export module\'s (F8)', () => { + for (const index of [12, 13, 15, 17]) { + const token = findToken(lines[index], 'export'); + assert.ok(token?.scopes.includes('keyword.control.export.cpp'), JSON.stringify(lines[index])); + } + const identifier = lines[18].find((token) => token.text.startsWith('exports')); + assert.ok(!identifier?.scopes.includes('keyword.control.export.cpp'), JSON.stringify(lines[18])); + }); + test('bare "module;" colors the keyword', () => { const token = findToken(lines[0], 'module'); assert.ok(token?.scopes.includes('keyword.control.module.cpp'), JSON.stringify(lines[0])); @@ -153,4 +175,17 @@ suite('module-syntax highlighting: the injected grammar (WA-VSCODE-001)', functi assert.ok(token, JSON.stringify(lines[11])); assert.ok(!token.scopes.includes('keyword.control.module.cpp'), JSON.stringify(lines[11])); }); + + test('the C++26 contract keywords are colored, and pre and post only as contract specifiers', () => { + for (const [index, word] of [[19, 'pre'], [19, 'post'], [20, 'pre']] as const) { + const token = findToken(lines[index], word); + assert.ok(token?.scopes.includes('keyword.other.contract.cpp'), `${word}: ${JSON.stringify(lines[index])}`); + } + const assertion = findToken(lines[21], 'contract_assert'); + assert.ok(assertion?.scopes.includes('keyword.other.contract_assert.cpp'), JSON.stringify(lines[21])); + for (const word of ['pre', 'post']) { + const name = lines[22].find((token) => token.text.includes(word)); + assert.ok(!name?.scopes.includes('keyword.other.contract.cpp'), `${word} as a name: ${JSON.stringify(lines[22])}`); + } + }); }); diff --git a/editors/vscode/test/unit/completionGate.test.ts b/editors/vscode/test/unit/completionGate.test.ts new file mode 100644 index 0000000..e519e08 --- /dev/null +++ b/editors/vscode/test/unit/completionGate.test.ts @@ -0,0 +1,27 @@ +// The editor side of the space trigger (fix plan 2026-09-26 F9, D4 layer 1), in plain Node: no VS +// Code, same reason test/unit/serverLog.test.ts is. The cases are the server's own +// (tests/test_completion.cpp), so the two gates cannot drift apart unnoticed. +import * as assert from 'assert'; +import { isImportLinePrefix, sendTriggeredCompletion } from '../../src/completionGate'; + +suite('space-triggered completion', () => { + test('an import directive keyword and one blank passes', () => { + for (const prefix of ['import ', 'export import ', ' import ', '\timport\t', 'export import ', '\texport\timport ']) { + assert.strictEqual(isImportLinePrefix(prefix), true, JSON.stringify(prefix)); + } + }); + + test('anything else does not', () => { + for (const prefix of ['import ', 'import', 'import a', 'import a ', 'int x = ', '', ' ', 'exportimport ', 'export ', + 'export module ', 'module ', 'importer ', '// import ', 'x import ']) { + assert.strictEqual(isImportLinePrefix(prefix), false, JSON.stringify(prefix)); + } + }); + + test('only a space is ever dropped, and only off an import line', () => { + assert.strictEqual(sendTriggeredCompletion(' ', 'int x = 1;', 8), false); + assert.strictEqual(sendTriggeredCompletion(' ', 'import hello;', 7), true, 'the line before the cursor is what counts'); + assert.strictEqual(sendTriggeredCompletion('.', 'import hello.', 13), true); + assert.strictEqual(sendTriggeredCompletion(undefined, 'int x = ', 8), true, 'typed or invoked completion always goes'); + }); +}); diff --git a/editors/vscode/test/unit/redact.test.ts b/editors/vscode/test/unit/redact.test.ts new file mode 100644 index 0000000..8fa6530 --- /dev/null +++ b/editors/vscode/test/unit/redact.test.ts @@ -0,0 +1,26 @@ +// src/redact.ts in plain Node (`npm run test:unit`): the extension's own part of Collect Report names no one. +import * as assert from 'assert'; +import { distinctiveName, redactJson, redactText } from '../../src/redact'; + +suite('the extension redacts its own part of a report', () => { + test('the home directory is ~ with either separator, and only as a whole path', () => { + const who = { home: '/home/alicewonder', user: 'alicewonder' }; + assert.strictEqual(redactText('/home/alicewonder/proj', who), '~/proj'); + assert.strictEqual(redactText('/home/alicewonderland/proj', who), '/home/alicewonderland/proj'); + assert.strictEqual(redactText('ran by alicewonder', who), 'ran by '); + }); + + test('a Windows profile is ~ in a JSON document, in either case', () => { + const who = { home: 'C:\\Users\\runneradmin', user: 'runneradmin' }; + const report = { workspaceFolders: ['c:\\users\\RunnerAdmin\\proj', 'D:\\a\\proj'], note: 'runneradmin' }; + assert.deepStrictEqual(redactJson(report, who), { workspaceFolders: ['~\\proj', 'D:\\a\\proj'], note: '' }); + }); + + test('a common user name is replaced only where it names a directory', () => { + const who = { home: '/home/runner', user: 'runner' }; + assert.strictEqual(redactText('/opt/runner/x and the test runner', who), '/opt//x and the test runner'); + assert.strictEqual(distinctiveName('runner'), false); + assert.strictEqual(distinctiveName('bob'), false); + assert.strictEqual(distinctiveName('runneradmin'), true); + }); +}); diff --git a/editors/zed/extension.toml b/editors/zed/extension.toml index 72fe2ec..299f63d 100644 --- a/editors/zed/extension.toml +++ b/editors/zed/extension.toml @@ -1,6 +1,6 @@ id = "mcppls" name = "C++ Modules Language Server" -version = "0.0.4" +version = "0.0.5" schema_version = 1 description = "C++20 named modules that work on any compiler: navigation, completion and diagnostics from mcpp-language-server" repository = "https://github.com/Sunrisepeak/mcpp-language-server" diff --git a/mcpp.toml b/mcpp.toml index 2dd7198..89a8e7f 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -34,7 +34,7 @@ libarchive = "3.8.7" [package] name = "mcpp-language-server" -version = "0.0.4" +version = "0.0.5" description = "Compiler-agnostic C++ modules language server" license = "Apache-2.0" authors = ["Sunrisepeak"] @@ -42,6 +42,9 @@ repo = "https://github.com/Sunrisepeak/mcpp-language-server" [build] sources = ["src/**/*.cppm", "src/**/*.cpp"] +# The server imports three libSystem symbols, all of them macOS 10.12+; clangd in the payload is +# built for 11.0. Honoured when cross-compiling from Linux since mcpp 2026.9.24.1 (mcpp#685). +macos_deployment_target = "11.0" # Every executable shares every module under src/. [targets.mcppls] diff --git a/modules/base/src/text.cpp b/modules/base/src/text.cpp index b25145f..3a2231b 100644 --- a/modules/base/src/text.cpp +++ b/modules/base/src/text.cpp @@ -151,6 +151,11 @@ std::optional offset_at(std::string_view text, Position position) { return offset; } +std::size_t byte_order_mark_size(std::string_view text) { + static constexpr std::string_view MARK { "\xEF\xBB\xBF" }; + return text.starts_with(MARK) ? MARK.size() : 0; +} + bool is_identifier_start(char c) { return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c == '_' || static_cast(c) >= 0x80; } diff --git a/modules/base/src/text.cppm b/modules/base/src/text.cppm index e103488..839b35d 100644 --- a/modules/base/src/text.cppm +++ b/modules/base/src/text.cppm @@ -30,6 +30,9 @@ std::string replace_all(std::string_view text, std::string_view from, std::strin std::size_t utf16_length(std::string_view utf8); Position position_at(std::string_view text, std::size_t byteOffset); std::optional offset_at(std::string_view text, Position position); +// A UTF-8 byte order mark that begins a file's text: its size, 3, or 0 when there is none. Editors do +// not show it, so the scanners start after it and give it no column (fix plan F2). +std::size_t byte_order_mark_size(std::string_view text); bool is_identifier_start(char c); bool is_identifier_char(char c); diff --git a/modules/base/src/version.cppm b/modules/base/src/version.cppm index fa142c5..f92d6db 100644 --- a/modules/base/src/version.cppm +++ b/modules/base/src/version.cppm @@ -9,7 +9,7 @@ export namespace mcppls::base { // checked against mcpp.toml (the one source) by `mcppls-devtools version --check`, not kept in step by // hand. Three constants that lived here and nothing read were removed rather than left to drift: // the S1 profile version is spec::PROFILE_VERSION, the kit manifest version is spec::KIT_VERSION. -inline constexpr std::string_view VERSION { "0.0.4" }; +inline constexpr std::string_view VERSION { "0.0.5" }; // The clangd the payload ships. Checked against packaging/payload.lock.json by the same command. inline constexpr std::string_view CLANGD_VERSION { "23.1.0" }; // The oldest mcpp that answers `mcpp emit build-database` — the `mcpp.build-database` kind, which diff --git a/modules/platform/src/process.cpp b/modules/platform/src/process.cpp index 54418fc..66b3c50 100644 --- a/modules/platform/src/process.cpp +++ b/modules/platform/src/process.cpp @@ -549,6 +549,34 @@ std::optional cpu_seconds(std::int64_t pid) { } } +std::vector thread_cpu(std::int64_t pid) { + std::vector threads; + if constexpr (mcppls::os::FAMILY == mcppls::os::Family::linux) { + if (pid <= 0) return threads; + constexpr double TICKS_PER_SECOND { 100 }; + for (const auto& task : fs::list_directory(std::format("/proc/{}/task", pid))) { + const auto stat = fs::read_file(task + "/stat"); + if (!stat) continue; + // " () ...": comm may hold spaces and parentheses, so the last ')' ends it. + const auto open = stat->find('('); + const auto close = stat->rfind(')'); + if (open == std::string::npos || close == std::string::npos || close < open || close + 2 >= stat->size()) continue; + std::vector fields; + for (auto field : base::split(std::string_view { *stat }.substr(close + 2), ' ')) { + if (!field.empty()) fields.push_back(field); + } + if (fields.size() < 13) continue; + const auto id = parse_count(std::string_view { *stat }.substr(0, open > 0 ? open - 1 : 0)); + const auto user = parse_count(fields[11]); + const auto system = parse_count(fields[12]); + if (!id || !user || !system) continue; + threads.push_back(ThreadCpu { *id, stat->substr(open + 1, close - open - 1), static_cast(*user + *system) / TICKS_PER_SECOND }); + } + } + (void)pid; + return threads; +} + std::string last_lines(std::string_view text, std::size_t lines) { if (lines == 0 || text.empty()) return std::string {}; std::size_t end { text.size() }; diff --git a/modules/platform/src/process.cppm b/modules/platform/src/process.cppm index 0b31dc6..33359a7 100644 --- a/modules/platform/src/process.cppm +++ b/modules/platform/src/process.cppm @@ -116,6 +116,16 @@ std::optional cpu_seconds(std::int64_t pid); // ps(1)'s cumulative "time" column, "[[dd-]hh:]mm:ss[.ss]", in seconds. std::optional parse_cpu_time(std::string_view text); +// One thread of a process and the CPU time it has used so far (an incident's "which thread spins"). +struct ThreadCpu { + std::int64_t id { 0 }; + std::string name; + double seconds { 0 }; +}; +// Every thread of a running process, where the platform can say: /proc//task on Linux; empty +// elsewhere, where only the whole process's time is known (cpu_seconds). +std::vector thread_cpu(std::int64_t pid); + // The openkal preopened directory that contains an absolute path, and the path // beneath it. Exposed for tests and for callers that need to explain a failure. struct PreopenMatch { diff --git a/src/bin/conformance.cpp b/src/bin/conformance.cpp index f628e1b..0a126af 100644 --- a/src/bin/conformance.cpp +++ b/src/bin/conformance.cpp @@ -26,6 +26,8 @@ import mcppls.platform.task; import mcppls.lsp.jsonrpc; import mcppls.lsp.connection; import mcppls.orchestrator.tokens; +import mcppls.bundle.redact; +import mcppls.bundle.zip; namespace base = mcppls::base; namespace fs = mcppls::platform::fs; @@ -78,6 +80,8 @@ struct Options { // under the real $HOME/%USERPROFILE%. Empty until `run()` reads the scenario; once set, every // process the runner starts for the server under test uses it as HOME (and USERPROFILE). std::string isolatedHome; + // issue #23 fix plan F18: where `bundle` checks leave a copy of the bundle they checked, for CI to keep; empty: nowhere. + std::string keepBundles; }; // Replaces HOME (POSIX) and USERPROFILE (Windows) in a spawn's environment, so @@ -614,7 +618,7 @@ bool includes(const Json& candidate, const Json& expected) { } // A fixture's expectations of a JSON result (conformance/README.md, S5 checks): each names a pointer and -// one of equals, contains, min-items, max-items, exists or absent, and holds when any value the pointer names satisfies it -- +// one of equals, contains, min-items, max-items, at-least (a number), exists or absent, and holds when any value the pointer names satisfies it -- // except each-contains, which every value the pointer names must satisfy (and holds when it names none). std::pair expectations_hold(const Json& value, const Json& expectations) { for (const auto& expectation : expectations) { @@ -643,6 +647,9 @@ std::pair expectations_hold(const Json& value, const Json& ex } else if (expectation.contains("max-items")) { const std::size_t wanted { expectation.value("max-items", std::size_t { 0 }) }; held = std::ranges::any_of(matches, [&](const Json* match) { return (match->is_array() || match->is_object()) && match->size() <= wanted; }); + } else if (expectation.contains("at-least")) { + const double wanted { expectation.value("at-least", 0.0) }; + held = std::ranges::any_of(matches, [&](const Json* match) { return match->is_number() && match->get() >= wanted; }); } if (!held) { std::string found { matches.empty() ? std::string { "nothing" } : lsp::dump(*matches.front()) }; @@ -933,6 +940,7 @@ class Scenario { std::unique_ptr mcpDaemon_; // the first mcp check "via": "daemon" std::string mcpFailure_; Json semanticTokensLegend_ = Json::object(); // initialize's capabilities.semanticTokensProvider.legend + Json capabilities_ = Json::object(); // initialize's capabilities, for "capabilities" checks McpClient* mcp_client(bool daemon) { auto& kept = daemon ? mcpDaemon_ : mcp_; @@ -956,13 +964,24 @@ class Scenario { Scenario(Client& client, const Options& options, std::vector serverArguments, std::string workspace, std::chrono::seconds timeout, std::map prepared, std::string cacheDirectory, bool expectWarm, - std::map> moduleFilesBefore, Json semanticTokensLegend = Json::object()) + std::map> moduleFilesBefore, Json semanticTokensLegend = Json::object(), + Json capabilities = Json::object()) : client_ { client }, options_ { options }, serverArguments_ { std::move(serverArguments) }, workspace_ { std::move(workspace) }, timeout_ { timeout }, prepared_ { std::move(prepared) }, cacheDirectory_ { std::move(cacheDirectory) }, expectWarm_ { expectWarm }, moduleFilesBefore_ { std::move(moduleFilesBefore) }, - semanticTokensLegend_ ( std::move(semanticTokensLegend) ) {} + semanticTokensLegend_ ( std::move(semanticTokensLegend) ), capabilities_ ( std::move(capabilities) ) {} std::string uri(std::string_view relative) const { return base::path_to_uri(base::join_path(workspace_, relative)); } + // A completion request at the check's "at"; "trigger" sends it as typing that character asked for it + // (CompletionTriggerKind.TriggerCharacter), the way an editor does for a trigger character. + Json completion_params(const Json& check, std::string_view file) const { + Json params { { "textDocument", Json { { "uri", uri(file) } } }, { "position", position(check.at("at")) } }; + if (const auto trigger = check.find("trigger"); trigger != check.end() && trigger->is_string()) { + params["context"] = Json { { "triggerKind", 2 }, { "triggerCharacter", trigger->get() } }; + } + return params; + } + std::string text_of(std::string_view relative) { if (auto it = open_.find(std::string { relative }); it != open_.end()) return it->second.first; return fs::read_file(base::join_path(workspace_, relative)).value_or(""); @@ -1428,11 +1447,23 @@ class Scenario { open(file); const std::string documentUri { uri(file) }; const std::string code { check.value("expect", std::string {}) }; + // Fix plan F11, F12: where the diagnostic is (`line`, 0-based), how severe (`severity`), and which codes + // must not be there with it (`absent`). + const std::optional line { check.contains("line") ? std::optional { check.value("line", 0) } : std::nullopt }; + const std::optional severity { check.contains("severity") ? std::optional { check.value("severity", 1) } : std::nullopt }; + const Json absent = check.value("absent", Json::array()); const bool found { client_.wait_for([&] { + bool hit { false }; for (const auto& diagnostic : client_.diagnostics[documentUri]) { - if (diagnostic.value("code", Json {}) == Json(code)) return true; + const Json& diagnosticCode { diagnostic.value("code", Json {}) }; + if (std::ranges::find(absent, diagnosticCode) != absent.end()) return false; + if (diagnosticCode != Json(code)) continue; + const Json* start { lsp::find_path(diagnostic, { "range", "start" }) }; + if (line && (start == nullptr || start->value("line", -1) != *line)) continue; + if (severity && diagnostic.value("severity", 1) != *severity) continue; + hit = true; } - return false; + return hit; }, timeout_) }; return { found, lsp::dump(client_.diagnostics[documentUri]) }; } @@ -1490,17 +1521,46 @@ class Scenario { // Touch the importing buffer so it is rebuilt against the edited module. change(file, text_of(file) + " "); } - const std::string expected { check.value("expect", std::string {}) }; - auto [ok, result] = retry("textDocument/completion", - [&] { return Json { { "textDocument", Json { { "uri", uri(file) } } }, { "position", position(check.at("at")) } }; }, + // "expect": a label prefix, or several that must all be there ("exact": whole labels); "absent": labels that must not be. + const bool exact { check.value("exact", false) }; + std::vector expected; + if (const auto wanted = check.find("expect"); wanted != check.end() && wanted->is_array()) { + for (const auto& one : *wanted) expected.push_back(one.get()); + } else { + expected.push_back(check.value("expect", std::string {})); + } + std::vector absent; + for (const auto& one : check.value("absent", Json::array())) absent.push_back(one.get()); + auto [ok, result] = retry("textDocument/completion", [&] { return completion_params(check, file); }, [&](const Json& value) { const auto labels = completion_labels(value); - return std::ranges::any_of(labels, [&](const std::string& label) { return label.starts_with(expected); }); + const auto present = [&](const std::string& prefix) { + return std::ranges::any_of(labels, [&](const std::string& label) { return exact ? label == prefix : label.starts_with(prefix); }); + }; + return std::ranges::all_of(expected, present) + && std::ranges::none_of(absent, [&](const std::string& label) { return std::ranges::find(labels, label) != labels.end(); }); }); auto labels = completion_labels(result); if (labels.size() > 12) labels.resize(12); return { ok, lsp::dump(labels) }; } + if (kind == "completion-empty") { + // F9 (fix plan 2026-09-26, D4): a completion answered with no items, within "within-ms" when given -- + // a space typed outside an import line is answered at once, without asking the core engine. + open(file); + const auto start = Clock::now(); + auto answer = client_.request("textDocument/completion", completion_params(check, file), timeout_); + const auto elapsed = std::chrono::duration_cast(Clock::now() - start).count(); + if (!answer) return { false, "no answer" }; + const bool empty { completion_labels(*answer).empty() }; + const bool inTime { !check.contains("within-ms") || elapsed <= check.value("within-ms", std::int64_t { 0 }) }; + return { empty && inTime, std::format("{} in {} ms", lsp::dump(*answer).substr(0, 120), elapsed) }; + } + if (kind == "capabilities") { + // The server capabilities initialize answered with, held to "expect" like a tool's result. + auto [held, detail] = expectations_hold(capabilities_, check.value("expect", Json::array())); + return { held, held ? lsp::dump(capabilities_.value("completionProvider", Json::object())).substr(0, 160) : detail }; + } if (kind == "references-span") { open(file); std::vector expected; @@ -1562,6 +1622,91 @@ class Scenario { } return { ok, detail.substr(0, std::min(detail.size(), 200)) }; } + if (kind == "bundle") { + // issue #23 fix plan F18: `mcppls.exportBundle` writes a zip within its size cap whose manifest is its contents, + // digest for digest, and in which no file -- nor the report cxxModules/report answers -- names the home directory + // the server runs with or the user it runs as (S3-5.5-3). The client's log it is sent carries the home too. + const std::string home { base::normalize_path(options_.isolatedHome.empty() ? mcppls::platform::dirs::home_directory() : options_.isolatedHome) }; + std::vector forbidden { home }; + forbidden.push_back(base::replace_all(home, "/", "\\")); + for (const std::string_view name : { "USER", "USERNAME", "LOGNAME" }) { + const auto user = mcppls::platform::env::get(name); + if (!user || !mcppls::bundle::distinctive_name(*user)) continue; + forbidden.push_back(*user); + // Its 8.3 form (RUNNER~1 for runneradmin), which a Windows temporary directory is spelled with. + if (user->size() > 8) forbidden.push_back(base::to_lower_ascii(user->substr(0, 6)) + "~"); + } + // The report is redacted but keeps the project's own paths; only a bundle can be asked to hide them. + const std::size_t forbiddenInReport { forbidden.size() }; + if (check.value("forbid-workspace", false)) forbidden.push_back(base::normalize_path(workspace_)); + const auto named = [&](std::string_view text, std::size_t count) -> std::string { + const std::string lower { base::to_lower_ascii(text) }; + for (const auto& needle : std::span { forbidden }.first(count)) { + if (!needle.empty() && lower.contains(base::to_lower_ascii(needle))) return needle == home ? std::string { "the home directory" } : std::format("'{}'", needle); + } + return {}; + }; + std::string path; + if (check.value("via", std::string { "command" }) == "cli") { + // `mcppls report --bundle`, the way CI and a person without an editor export one. + path = base::join_path(cacheDirectory_, std::format("{}.zip", check.value("id", std::string { "bundle" }))); + mcppls::platform::SpawnOptions spawn; + spawn.program = options_.server; + spawn.arguments = { "report", "--root", workspace_, "--settle", "30", "--bundle", path }; + for (const auto& argument : check.value("args", Json::array())) spawn.arguments.push_back(argument.get()); + if (!options_.payload.empty()) spawn.arguments.insert(spawn.arguments.end(), { "--payload", options_.payload }); + if (!options_.clangd.empty()) spawn.arguments.insert(spawn.arguments.end(), { "--clangd", options_.clangd }); + if (!options_.kit.empty()) spawn.arguments.insert(spawn.arguments.end(), { "--kit", options_.kit }); + spawn.arguments.insert(spawn.arguments.end(), serverArguments_.begin(), serverArguments_.end()); + spawn.workDirectory = workspace_; + auto environment = mcppls::platform::env::variables(); + environment.push_back("MCPPLS_CACHE_DIR=" + cacheDirectory_); + apply_isolated_home(environment, options_); + spawn.environment = std::move(environment); + auto running = std::async(std::launch::async, [spawn, timeout = timeout_]() mutable { return mcppls::platform::run(std::move(spawn), timeout); }); + while (running.wait_for(std::chrono::milliseconds { 200 }) != std::future_status::ready) client_.drain(std::chrono::milliseconds { 0 }); + auto result = running.get(); + if (!result) return { false, result.error().message }; + if (result->timedOut || result->exitCode != 0) return { false, std::format("mcppls report --bundle: exit {}: {}", result->exitCode, (result->output + result->error).substr(0, 300)) }; + } else { + Json arguments = check.value("arguments", Json::object()); + arguments["client"] = Json { { "name", "mcppls-conformance" }, { "log", std::format("started in {}\nworkspace {}\n", home, workspace_) } }; + const auto answer = client_.request("workspace/executeCommand", Json { { "command", "mcppls.exportBundle" }, { "arguments", Json::array({ arguments }) } }, + timeout_); + if (!answer || !answer->is_object() || !answer->contains("path")) return { false, "mcppls.exportBundle: no answer, or an error" }; + path = answer->value("path", std::string {}); + } + auto archive = fs::read_file(path); + if (archive && !options_.keepBundles.empty()) { + (void)fs::create_directories(options_.keepBundles); + (void)fs::write_file(base::join_path(options_.keepBundles, std::format("{}-{}.zip", base::file_name(options_.fixture), check.value("id", std::string { "bundle" }))), *archive); + } + fs::remove_all(path); + if (!archive) return { false, std::format("no bundle at {}", path) }; + if (archive->size() > 25 * 1024 * 1024) return { false, std::format("the bundle is {} bytes, over its 25 MB cap", archive->size()) }; + auto files = mcppls::bundle::read_archive(*archive); + if (!files) return { false, files.error() }; + if (!files->contains("manifest.json")) return { false, "no manifest.json" }; + const Json manifest = Json::parse(files->at("manifest.json"), nullptr, false); + if (!manifest.is_object() || !manifest.contains("files")) return { false, "manifest.json is not a manifest" }; + if (manifest["files"].size() + 1 != files->size()) return { false, std::format("the manifest lists {} files, the bundle has {}", manifest["files"].size(), files->size() - 1) }; + for (const auto& file : manifest["files"]) { + const std::string name { file.value("path", std::string {}) }; + const auto found = files->find(name); + if (found == files->end()) return { false, std::format("{} is in the manifest, not in the bundle", name) }; + if (base::sha256_hex(found->second) != file.value("sha256", std::string {})) return { false, std::format("{} is not what the manifest's digest says", name) }; + } + for (const auto& expected : check.value("expect-files", Json::array())) { + if (!files->contains(expected.get())) return { false, std::format("no {} in the bundle", expected.get()) }; + } + for (const auto& [name, content] : *files) { + if (const auto what = named(content, forbidden.size()); !what.empty()) return { false, std::format("{} names {}", name, what) }; + } + const auto report = client_.request("cxxModules/report", Json::object(), timeout_); + if (!report) return { false, "cxxModules/report: no answer" }; + if (const auto what = named(lsp::dump(*report), forbiddenInReport); !what.empty()) return { false, std::format("cxxModules/report names {}", what) }; + return { true, std::format("{} files, {} bytes, redactions {}", files->size(), archive->size(), lsp::dump(manifest["redaction"]["rules"])) }; + } if (kind == "report") { // robustness design O3: cxxModules/report, held to "expect" like a tool's result, retried within the check's time // (a plan or an engine may still be on its way). @@ -1880,6 +2025,9 @@ int run(Options options) { Json initializeParams { { "processId", nullptr }, { "rootUri", base::path_to_uri(workspace) }, { "workspaceFolders", workspaceFolders }, { "capabilities", capabilities } }; if (!initializationOptions.empty()) initializeParams["initializationOptions"] = initializationOptions; + // A scenario's own "client-info": the client this runner says it is (fix plan 2026-09-26 F9: what a + // server tells VS Code differs from what it tells any other client). + if (const auto info = scenario.find("client-info"); info != scenario.end() && info->is_object()) initializeParams["clientInfo"] = *info; auto initialized = client.request("initialize", std::move(initializeParams), std::chrono::seconds { 120 }); if (!initialized || !initialized->is_object()) { say("FAIL initialize: no result"); @@ -1902,7 +2050,7 @@ int run(Options options) { ? (*initialized)["capabilities"]["semanticTokensProvider"]["legend"] : Json::object(); Scenario runner { client, options, serverArguments, workspace, options.timeout, std::move(prepared), cacheDirectory, options.expectWarm, - std::move(moduleFilesBefore), semanticTokensLegend }; + std::move(moduleFilesBefore), semanticTokensLegend, initialized->value("capabilities", Json::object()) }; int failures { advertised ? 0 : 1 }; // "initialize-within": seconds. The handshake is answered at all, and in time (0.0.3 plan B1). if (const auto within = scenario.find("initialize-within"); within != scenario.end() && within->is_number()) { @@ -2220,6 +2368,80 @@ int prepare_generated_module_compdb(const std::string& compiler) { return 0; } +// compdb-lto-msvc (issue #23, fix plan F1): the compile_commands.json of a project built with LTO for the +// MSVC ABI, as CMake writes it for `-flto` with clang++ on Windows, with `compiler` (or clang++ on PATH) +// as the driver. Nothing is compiled: the fixture is about the commands the server gives clangd, whose +// module scan failed on `LTO requires -fuse-ld=lld` when they carried `-flto` and no `-c`. The driver +// raises that for the windows-msvc target on any host, so the fixture runs on Linux. `--no-default-config` +// makes the driver the one of the LLVM Windows installer, with no configuration file: an LLVM that +// carries one choosing lld (as mcpp's does) would not plan the link that fails. +int prepare_compdb_lto_msvc(const std::string& compiler) { + const std::string root { fs::current_directory() }; + auto clangxx = on_path(compiler.empty() ? std::string { "clang++" } : compiler); + if (!clangxx) { + say("compdb-lto-msvc: {} is not on PATH", compiler); + return 1; + } + Json database = Json::array(); + for (const std::string_view relative : { "src/answer.cppm", "src/main.cpp" }) { + const std::string source { native(base::join_path(root, relative)) }; + database.push_back(Json { { "directory", native(root) }, { "file", source }, + { "arguments", Json::array({ *clangxx, "--no-default-config", "--target=x86_64-pc-windows-msvc", "-std=c++23", + "-flto", "-O2", "-c", source, "-o", source + ".obj" }) } }); + } + if (auto written = fs::write_file(base::join_path(root, "compile_commands.json"), database.dump(2)); !written) { + say("compdb-lto-msvc: {}", written.error().message); + return 1; + } + return 0; +} + +// Fix plan F6: a compile_commands.json whose commands carry an option value the compiler rejects, the kind of +// command #23's LTO one was: clangd's module scan fails on every unit, and the status says the command was rejected, +// with the driver's own words, instead of leaving the user to find "Scanning modules dependencies ... failed" in a log. +int prepare_compdb_rejected_command(const std::string& compiler) { + const std::string root { fs::current_directory() }; + auto clangxx = on_path(compiler.empty() ? std::string { "clang++" } : compiler); + if (!clangxx) { + say("compdb-rejected-command: {} is not on PATH", compiler); + return 1; + } + Json database = Json::array(); + for (const std::string_view relative : { "src/answer.cppm", "src/main.cpp" }) { + const std::string source { native(base::join_path(root, relative)) }; + database.push_back(Json { { "directory", native(root) }, { "file", source }, + { "arguments", Json::array({ *clangxx, "-std=c++99999", "-c", source, "-o", source + ".o" }) } }); + } + if (auto written = fs::write_file(base::join_path(root, "compile_commands.json"), database.dump(2)); !written) { + say("compdb-rejected-command: {}", written.error().message); + return 1; + } + return 0; +} + +// C++26 alignment (fix plan 2026-09-26 §9): a compile_commands.json whose units name two standards -- a module and an +// importer of std at C++23, an application at C++26 importing both. One std BMI cannot serve both standards. +int prepare_compdb_mixed_standards(const std::string& compiler) { + const std::string root { fs::current_directory() }; + auto clangxx = on_path(compiler.empty() ? std::string { "clang++" } : compiler); + if (!clangxx) { + say("compdb-mixed-standards: {} is not on PATH", compiler); + return 1; + } + Json database = Json::array(); + for (const auto& [relative, standard] : { std::pair { "src/core.cppm", "-std=c++23" }, std::pair { "src/legacy.cpp", "-std=c++23" }, + std::pair { "src/app.cpp", "-std=c++26" } }) { + const std::string source { native(base::join_path(root, relative)) }; + database.push_back(Json { { "directory", native(root) }, { "file", source }, + { "arguments", Json::array({ *clangxx, "-stdlib=libc++", standard, "-c", source, "-o", source + ".o" }) } }); + } + if (auto written = fs::write_file(base::join_path(root, "compile_commands.json"), database.dump(2)); !written) { + say("compdb-mixed-standards: {}", written.error().message); + return 1; + } + return 0; +} + // real-project plan RP2.1: a second, newer mock mcpp under a fixture's isolated HOME // (`"isolate-home": true`), at the path producer negotiation searches // (`mcppls::project::other_mcpp_executables`, `xim-x-mcpp//bin/mcpp`), so a fixture whose @@ -2395,6 +2617,56 @@ int prepare_clangd_cannot_load() { return 0; } +// Fix plan F3: a clangd that crashes the way clangd 23.1 did on Windows in issue #23 -- its crash context on standard +// error, naming a file that is not the one being edited, and then gone -- once, 25 seconds after it starts; every +// later start is the payload's real clangd. POSIX only: the stand-in is a shell script around the real one. +int prepare_clangd_crash_context(const std::string& payload) { + if constexpr (mcppls::os::FAMILY == mcppls::os::Family::windows) { + say("clangd-crash-context: POSIX only"); + return 2; + } + const std::string workspace { fs::current_directory() }; + const std::string real { base::join_path(payload, "clangd/bin/clangd") }; + if (!fs::exists(real)) { + say("clangd-crash-context: no clangd at {}", real); + return 1; + } + const std::string directory { base::join_path(workspace, "stand-in") }; + (void)fs::create_directories(directory); + const std::string crashed { base::join_path(directory, "crashed-once") }; + const std::string crasher { base::join_path(workspace, "src/crasher.cpp") }; + const std::string script { std::format( + "#!/bin/sh\n" + "real='{}'\n" + "case \"$1\" in --version|--help) exec \"$real\" \"$@\" ;; esac\n" + "[ -e '{}' ] && exec \"$real\" \"$@\"\n" + ": > '{}'\n" + // A background job of sh reads /dev/null: the editor's input goes to clangd through a descriptor kept first. + "exec 3<&0\n" + "\"$real\" \"$@\" <&3 3<&- &\n" + "child=$!\n" + "sleep 25\n" + "echo 'PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.' >&2\n" + "echo 'Signalled during AST worker action: Build AST' >&2\n" + "echo ' Filename: {}' >&2\n" + "echo ' Directory: {}' >&2\n" + "echo ' Command Line: clang++ -std=c++23 -c -- {}' >&2\n" + "echo ' Version: 1' >&2\n" + "kill -KILL $child\n" + "exit 139\n", + real, crashed, crashed, crasher, workspace, crasher) }; + const std::string clangd { base::join_path(directory, "clangd") }; + if (auto written = fs::write_file(clangd, script); !written) { + say("clangd-crash-context: {}", written.error().message); + return 1; + } + if (auto marked = fs::make_executable(std::vector { clangd }); !marked) { + say("clangd-crash-context: {}", marked.error().message); + return 1; + } + return 0; +} + int prepare(const std::string& kind, const std::string& argument) { if (kind == "s1-two-sets") return prepare_s1_two_sets(argument); if (kind == "payload-corrupt") return prepare_payload_corrupt(argument); @@ -2404,7 +2676,11 @@ int prepare(const std::string& kind, const std::string& argument) { if (kind == "compdb-clangxx-msvc-std") return prepare_compdb_msvc_std(false); if (kind == "generated-module-old-mcpp") return prepare_generated_module_compdb(argument); if (kind == "clangd-cannot-load") return prepare_clangd_cannot_load(); - say("prepare: unknown fixture kind {} (s1-two-sets, payload-corrupt, producer-candidate, failure-at-base, compdb-clang-cl-std, compdb-clangxx-msvc-std, generated-module-old-mcpp, clangd-cannot-load)", kind); + if (kind == "clangd-crash-context") return prepare_clangd_crash_context(argument); + if (kind == "compdb-rejected-command") return prepare_compdb_rejected_command(argument); + if (kind == "compdb-mixed-standards") return prepare_compdb_mixed_standards(argument); + if (kind == "compdb-lto-msvc") return prepare_compdb_lto_msvc(argument); + say("prepare: unknown fixture kind {} (s1-two-sets, payload-corrupt, producer-candidate, failure-at-base, compdb-clang-cl-std, compdb-clangxx-msvc-std, generated-module-old-mcpp, clangd-cannot-load, compdb-lto-msvc, clangd-crash-context, compdb-rejected-command, compdb-mixed-standards)", kind); return 2; } @@ -2437,6 +2713,7 @@ int main(int argc, char* argv[]) { (void)runCommand.option("plain-client").help("Alias for --client plain"); (void)runCommand.option("client").takes_value().help("The capabilities a real editor sends: vscode, neovim, zed or plain (default: this runner's own, the full experimental.cxxModules block)"); (void)runCommand.option("stress-seed").takes_value().help("Overrides every stress check's own \"seed\" (mcppls-devtools stress --seed)"); + (void)runCommand.option("keep-bundles").takes_value().help("Directory a copy of every diagnostic bundle a bundle check exported is left in"); (void)runCommand.action([&](const cmdline::ParsedArgs& args) { Options options; options.server = absolute(args.value("server").value_or("")); @@ -2454,6 +2731,7 @@ int main(int argc, char* argv[]) { options.expectWarm = args.is_flag_set("expect-warm"); options.noDynamicWatch = args.is_flag_set("no-dynamic-watch"); options.plainClient = args.is_flag_set("plain-client"); + options.keepBundles = args.value("keep-bundles") ? absolute(*args.value("keep-bundles")) : std::string {}; if (auto clientName = args.value("client")) { if (*clientName == "vscode") options.client = Options::ClientProfile::vscode; else if (*clientName == "neovim") options.client = Options::ClientProfile::neovim; diff --git a/src/bin/mockmcpp.cpp b/src/bin/mockmcpp.cpp index 7040bd9..94e810f 100644 --- a/src/bin/mockmcpp.cpp +++ b/src/bin/mockmcpp.cpp @@ -9,7 +9,8 @@ // // In mcpp-mock.json every string may use ${root} (the directory the command runs // in) and ${env:NAME} or ${env:NAME|fallback}. {"database": , "watch": [...]} -// is answered as an envelope; {"diagnostics": [...]} as a failure with exit 1; {"unavailable": "..."} +// is answered as an envelope; {"diagnostics": [...]} as a failure with exit 1, and both together as a partial +// answer (S2 0.3.0) with exit 1; {"unavailable": "..."} // as xlings answers for an mcpp a project pins but that is not installed. {"hang": {...}} is the // failure the runner exists for: a producer that does not return, and that leaves something behind // holding the caller's pipe, which is what one hung index refresh did to an editor for twelve @@ -189,8 +190,12 @@ int emit_build_database(std::span arguments) { data["database"] = recorded["database"]; data["watch"] = recorded.value("watch", Json::array({ "mcpp.toml", "mcpp.lock", "src/**/*.cppm", "src/**/*.cpp" })); data["inputs-fingerprint"] = fingerprint(root); - std::println("{}", envelope("mcpp.build-database", std::move(data), Json::array(), Json::array({ "read-project" })).dump(2)); - return 0; + // S2 0.3.0 (mcpp-community/mcpp#699): a database together with error diagnostics describes all but what they name, + // and the command exits non-zero. + const Json diagnostics = recorded.value("diagnostics", Json::array()); + const bool partial { std::ranges::any_of(diagnostics, [](const Json& diagnostic) { return diagnostic.value("severity", std::string {}) == "error"; }) }; + std::println("{}", envelope("mcpp.build-database", std::move(data), diagnostics, Json::array({ "read-project" })).dump(2)); + return partial ? 1 : 0; } } // namespace diff --git a/src/bundle/identity.cpp b/src/bundle/identity.cpp new file mode 100644 index 0000000..632ae23 --- /dev/null +++ b/src/bundle/identity.cpp @@ -0,0 +1,89 @@ +module mcppls.bundle.identity; + +import std; +import mcppls.os; +import mcppls.base.path; +import mcppls.base.text; +import mcppls.platform.dirs; +import mcppls.platform.env; +import mcppls.platform.fs; +import mcppls.bundle.redact; + +namespace mcppls::bundle { + +namespace { + +void add_unique(std::vector& list, std::string value) { + value = std::string { base::trim(value) }; + if (value.empty()) return; + if (std::ranges::any_of(list, [&](const std::string& known) { return base::iequals_ascii(known, value); })) return; + list.push_back(std::move(value)); +} + +std::string first_line(std::string_view text) { + const auto lines = base::split_lines(text); + return lines.empty() ? std::string {} : std::string { base::trim(lines.front()) }; +} + +// The value of `namevalue` in an XML property list. +std::optional plist_string(std::string_view plist, std::string_view key) { + const std::string marker { std::format("{}", key) }; + const std::size_t at { plist.find(marker) }; + if (at == std::string_view::npos) return std::nullopt; + const std::size_t open { plist.find("", at + marker.size()) }; + if (open == std::string_view::npos || open - (at + marker.size()) > 64) return std::nullopt; + const std::size_t close { plist.find("", open) }; + if (close == std::string_view::npos) return std::nullopt; + return std::string { plist.substr(open + 8, close - open - 8) }; +} + +std::vector host_names() { + std::vector hosts; + for (const std::string_view name : { "COMPUTERNAME", "HOSTNAME", "HOST" }) { + if (auto value = platform::env::get(name)) add_unique(hosts, *value); + } + if constexpr (mcppls::os::FAMILY == mcppls::os::Family::linux) { + for (const std::string_view file : { "/proc/sys/kernel/hostname", "/etc/hostname" }) { + if (auto text = platform::fs::read_file(file)) add_unique(hosts, first_line(*text)); + } + } else if constexpr (mcppls::os::FAMILY == mcppls::os::Family::macos) { + // What `scutil --get LocalHostName` and `ComputerName` answer, without starting scutil. A + // binary property list is not read; the environment and the bundle's own check remain. + if (auto plist = platform::fs::read_file("/Library/Preferences/SystemConfiguration/preferences.plist"); plist && !plist->starts_with("bplist")) { + for (const std::string_view key : { "LocalHostName", "ComputerName", "HostName" }) { + if (auto value = plist_string(*plist, key)) add_unique(hosts, *value); + } + } + } + return hosts; +} + +} // namespace + +Identity current_identity() { + Identity identity; + const std::string home { platform::dirs::home_directory() }; + add_unique(identity.homes, base::normalize_path(home)); + add_unique(identity.homes, base::normalize_path(platform::fs::canonical_path(home))); + if constexpr (mcppls::os::FAMILY == mcppls::os::Family::windows) { + // HOMEDRIVE and HOMEPATH can name another directory than USERPROFILE (a redirected home). + auto drive = platform::env::get("HOMEDRIVE"); + auto path = platform::env::get("HOMEPATH"); + if (drive && path && !path->empty() && *path != "\\") add_unique(identity.homes, base::normalize_path(*drive + *path)); + } + for (const std::string_view name : { "USER", "LOGNAME", "USERNAME" }) { + if (auto value = platform::env::get(name)) add_unique(identity.users, *value); + } + for (const auto& spelling : identity.homes) add_unique(identity.users, std::string { base::file_name(spelling) }); + identity.hosts = host_names(); + return identity; +} + +Identity hiding_workspaces(Identity identity, std::vector roots) { + for (auto& root : roots) { + if (!root.empty()) identity.workspaces.push_back(base::normalize_path(root)); + } + return identity; +} + +} // namespace mcppls::bundle diff --git a/src/bundle/identity.cppm b/src/bundle/identity.cppm new file mode 100644 index 0000000..01449cc --- /dev/null +++ b/src/bundle/identity.cppm @@ -0,0 +1,18 @@ +// Who this process runs as, for what a report or a bundle must not name (issue #23 fix plan F18): +// read from the environment and a few system files, never by starting a program, so the event loop +// may ask for it. +export module mcppls.bundle.identity; + +import std; +import mcppls.bundle.redact; + +export namespace mcppls::bundle { + +// The home directory as set and as the file system names it (a symbolic link, an 8.3 alias +// resolved), the login name and the home directory's last component, and the machine's names. +Identity current_identity(); + +// A copy of `identity` that also hides `roots` as , , ... +Identity hiding_workspaces(Identity identity, std::vector roots); + +} // namespace mcppls::bundle diff --git a/src/bundle/redact.cpp b/src/bundle/redact.cpp new file mode 100644 index 0000000..3fb7c1c --- /dev/null +++ b/src/bundle/redact.cpp @@ -0,0 +1,874 @@ +module mcppls.bundle.redact; + +import std; +import nlohmann.json; + +namespace mcppls::bundle { + +namespace { + +using Json = nlohmann::json; + +bool is_alpha(unsigned char c) { return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z'); } +bool is_digit(unsigned char c) { return c >= '0' && c <= '9'; } +bool is_alnum(unsigned char c) { return is_alpha(c) || is_digit(c); } +// A letter of any script (a UTF-8 byte of a multi-byte character) or a digit: what a word is made of. +bool is_word(unsigned char c) { return is_alnum(c) || c >= 0x80; } +char lower(char c) { return c >= 'A' && c <= 'Z' ? static_cast(c - 'A' + 'a') : c; } +char upper(char c) { return c >= 'a' && c <= 'z' ? static_cast(c - 'a' + 'A') : c; } + +std::string lowered(std::string_view text) { + std::string out { text }; + for (char& c : out) c = lower(c); + return out; +} + +int hex_value(char c) { + if (c >= '0' && c <= '9') return c - '0'; + if (c >= 'a' && c <= 'f') return c - 'a' + 10; + if (c >= 'A' && c <= 'F') return c - 'A' + 10; + return -1; +} + +// The byte a %XX escape at `i` stands for. +std::optional escaped_at(std::string_view text, std::size_t i) { + if (i + 2 >= text.size() || text[i] != '%') return std::nullopt; + const int high { hex_value(text[i + 1]) }; + const int low { hex_value(text[i + 2]) }; + if (high < 0 || low < 0) return std::nullopt; + return static_cast(high * 16 + low); +} + +// ---- paths in any spelling --------------------------------------------------------------------- +// +// One path, written the ways it reaches a log or a report: '/' or '\' separators, a backslash +// doubled (JSON) or doubled again (JSON inside JSON), "\/" (a JSON writer escaping '/'), any byte +// percent-encoded (a file:// URI: "c%3A/Users/John%20Doe"), the drive letter in either case, a +// Windows drive seen from WSL (/mnt/c/...), and a Windows profile under its 8.3 name (RUNNER~1). +// Names compare case-insensitively: Windows and macOS file systems do, and nothing is lost on Linux +// by also catching "/HOME/SPEAK". + +enum class PieceKind { drive, separator, literal, short_name }; + +struct Piece { + PieceKind kind { PieceKind::literal }; + std::string text; // drive: the letter; literal: the name; short_name: the 8.3 prefix +}; + +struct PathPattern { + std::vector pieces; + std::string rule; + std::string replacement; +}; + +// The number of bytes a separator takes at `i`, 0 when there is none. +std::size_t match_separator(std::string_view text, std::size_t i) { + if (i >= text.size()) return 0; + if (text[i] == '/') return 1; + if (text[i] == '\\') { + std::size_t j { i }; + while (j < text.size() && text[j] == '\\') ++j; + if (j < text.size() && text[j] == '/') return j + 1 - i; // "\/", a JSON writer escaping '/' + return j - i; + } + if (const auto escaped = escaped_at(text, i); escaped && (*escaped == '/' || *escaped == '\\')) return 3; + return 0; +} + +// The number of bytes `literal` takes at `i`, each of its bytes as itself or percent-encoded; 0 when it is not there. +std::size_t match_literal(std::string_view text, std::size_t i, std::string_view literal) { + std::size_t j { i }; + for (const char expected : literal) { + if (j < text.size() && lower(text[j]) == lower(expected)) { + ++j; + continue; + } + if (const auto escaped = escaped_at(text, j); escaped && lower(*escaped) == lower(expected)) { + j += 3; + continue; + } + return 0; + } + return j - i; +} + +std::size_t match_short_name(std::string_view text, std::size_t i, std::string_view prefix) { + const std::size_t head { match_literal(text, i, prefix) }; + if (head == 0) return 0; + std::size_t j { i + head }; + if (j >= text.size() || text[j] != '~') return 0; + ++j; + const std::size_t digits { j }; + while (j < text.size() && is_digit(static_cast(text[j]))) ++j; + return j == digits ? 0 : j - i; +} + +std::size_t match_drive(std::string_view text, std::size_t i, char letter) { + if (i >= text.size() || lower(text[i]) != letter) return 0; + if (i + 1 < text.size() && text[i + 1] == ':') return 2; + if (const auto escaped = escaped_at(text, i + 1); escaped && *escaped == ':') return 4; + return 0; +} + +// Whether a name goes on at `j`: "/home/speak" is not in "/home/speaker" nor in "/home/speak.old", +// but it is in "/home/speak." at the end of a sentence and in "/home/speak/src". +bool name_continues(std::string_view text, std::size_t j) { + if (j >= text.size()) return false; + const auto c = static_cast(text[j]); + if (is_word(c) || c == '_') return true; + if (c == '.' || c == '-' || c == '+' || c == '~') { + if (j + 1 >= text.size()) return false; + const auto next = static_cast(text[j + 1]); + return is_word(next) || next == '_'; + } + if (const auto escaped = escaped_at(text, j)) return is_alnum(static_cast(*escaped)); + return false; +} + +// The end of `pattern` matched at `i`, when it is there as a whole path of its own. +// Whether a path may start at `i`: not the tail of a longer name ("/data/home/speak" does not +// contain the home "/home/speak"), but it may be glued to a compiler option of one or two letters +// ("-I/home/speak/include", "-LC:/Users/x/lib", MSVC's "/IC:\Users\x"). `drive`: the path starts +// with a drive letter, which a '/' option may come before. +bool path_starts_at(std::string_view text, std::size_t i, bool drive) { + if (i == 0) return true; + const auto nameChar = [](unsigned char c) { return is_word(c) || c == '_' || c == '.' || c == '-'; }; + if (!nameChar(static_cast(text[i - 1]))) return true; + std::size_t run { i }; + while (run > 0 && is_alpha(static_cast(text[run - 1]))) --run; + const std::size_t length { i - run }; + if (length == 0 || length > 2 || run == 0) return false; + return text[run - 1] == '-' || (drive && text[run - 1] == '/'); +} + +std::optional match_path(std::string_view text, std::size_t i, const PathPattern& pattern) { + if (pattern.pieces.empty()) return std::nullopt; + if (!path_starts_at(text, i, pattern.pieces.front().kind == PieceKind::drive)) return std::nullopt; + std::size_t j { i }; + for (const auto& piece : pattern.pieces) { + std::size_t taken { 0 }; + switch (piece.kind) { + case PieceKind::drive: taken = match_drive(text, j, piece.text.front()); break; + case PieceKind::separator: taken = match_separator(text, j); break; + case PieceKind::literal: taken = match_literal(text, j, piece.text); break; + case PieceKind::short_name: taken = match_short_name(text, j, piece.text); break; + } + if (taken == 0) return std::nullopt; + j += taken; + } + if (name_continues(text, j)) return std::nullopt; + return j; +} + +// Whether a pattern could start at `i`, cheaply, before match_path looks. +bool could_start(std::string_view text, std::size_t i, const PathPattern& pattern) { + const Piece& first { pattern.pieces.front() }; + const char c { text[i] }; + if (first.kind == PieceKind::drive) return lower(c) == first.text.front(); + return c == '/' || c == '\\' || c == '%'; +} + +// The segments of a '/'-separated path, empty ones dropped. +std::vector segments_of(std::string_view path) { + std::vector segments; + std::string current; + for (const char c : path) { + if (c == '/' || c == '\\') { + if (!current.empty()) segments.push_back(std::move(current)); + current.clear(); + } else { + current += c; + } + } + if (!current.empty()) segments.push_back(std::move(current)); + return segments; +} + +bool is_windows_path(std::string_view path) { + return path.size() >= 2 && is_alpha(static_cast(path[0])) && path[1] == ':'; +} + +PathPattern pattern_of(std::string_view path, std::string rule, std::string replacement) { + PathPattern pattern { {}, std::move(rule), std::move(replacement) }; + std::vector segments; + if (is_windows_path(path)) { + pattern.pieces.push_back(Piece { PieceKind::drive, std::string(1, lower(path[0])) }); + segments = segments_of(path.substr(2)); + } else { + segments = segments_of(path); + } + for (auto& segment : segments) { + pattern.pieces.push_back(Piece { PieceKind::separator, {} }); + pattern.pieces.push_back(Piece { PieceKind::literal, std::move(segment) }); + } + return pattern; +} + +// The 8.3 name Windows gives a directory whose own name is not one (longer than eight bytes, or +// with a space or a second dot): its first six valid characters, upper case, then ~N. +std::optional short_name_prefix(std::string_view name) { + std::string valid; + for (const char c : name) { + if (c == ' ' || c == '.' || std::string_view { "\"*+,/:;<=>?[\\]|" }.contains(c)) continue; + valid += upper(c); + } + const bool needsOne { name.size() > 8 || name.contains(' ') || std::ranges::count(name, '.') > 1 || valid.size() != name.size() }; + if (!needsOne || valid.size() < 2) return std::nullopt; + return valid.substr(0, 6); +} + +// Every spelling of a path worth looking for: itself, and for a Windows path the WSL view of it +// and, for its last component, the 8.3 name. +std::vector path_patterns(std::string_view path, std::string_view rule, std::string_view replacement) { + std::vector patterns; + patterns.push_back(pattern_of(path, std::string { rule }, std::string { replacement })); + if (is_windows_path(path)) { + const std::string wsl { std::format("/mnt/{}{}", lower(path[0]), path.substr(2)) }; + patterns.push_back(pattern_of(wsl, std::string { rule }, std::string { replacement })); + auto shortened = patterns.front(); + auto& last = shortened.pieces.back(); + if (last.kind == PieceKind::literal) { + if (auto prefix = short_name_prefix(last.text)) { + last = Piece { PieceKind::short_name, std::move(*prefix) }; + patterns.push_back(std::move(shortened)); + } + } + } + return patterns; +} + +// Replaces every match of `pattern`; returns how many there were. +std::size_t replace_path(std::string& text, const PathPattern& pattern) { + if (pattern.pieces.empty() || text.empty()) return 0; + std::string out; + std::size_t count { 0 }; + std::size_t copied { 0 }; + for (std::size_t i { 0 }; i < text.size();) { + if (could_start(text, i, pattern)) { + if (const auto end = match_path(text, i, pattern)) { + if (out.empty()) out.reserve(text.size()); + out.append(text, copied, i - copied); + out += pattern.replacement; + copied = i = *end; + ++count; + continue; + } + } + ++i; + } + if (count == 0) return 0; + out.append(text, copied); + text = std::move(out); + return count; +} + +std::optional find_path(std::string_view text, const PathPattern& pattern) { + if (pattern.pieces.empty()) return std::nullopt; + for (std::size_t i { 0 }; i < text.size(); ++i) { + if (could_start(text, i, pattern) && match_path(text, i, pattern)) return i; + } + return std::nullopt; +} + +// ---- words ------------------------------------------------------------------------------------- + +// Case-insensitive search for `needle` from `from`. +std::size_t ifind(std::string_view haystack, std::string_view needle, std::size_t from = 0) { + if (needle.empty() || haystack.size() < needle.size()) return std::string_view::npos; + const char first { lower(needle.front()) }; + for (std::size_t i { from }; i + needle.size() <= haystack.size(); ++i) { + if (lower(haystack[i]) != first) continue; + bool same { true }; + for (std::size_t k { 1 }; k < needle.size() && same; ++k) same = lower(haystack[i + k]) == lower(needle[k]); + if (same) return i; + } + return std::string_view::npos; +} + +// A whole word: neither neighbour a letter or digit. '_', '-' and '.' separate words here, so +// "speak_dev" and "speak.log" name the user "speak", "speaker" does not. +bool whole_word_at(std::string_view text, std::size_t at, std::size_t length) { + if (at > 0 && is_word(static_cast(text[at - 1]))) return false; + const std::size_t end { at + length }; + return end >= text.size() || !is_word(static_cast(text[end])); +} + +// A name as a word, but not as a JSON object's key: a key is this program's own vocabulary ("plan", +// "server"), and a user of that name leaks nothing through it. +bool name_word_at(std::string_view text, std::size_t at, std::size_t length) { + if (!whole_word_at(text, at, length)) return false; + const std::size_t end { at + length }; + if (at == 0 || text[at - 1] != '"' || end >= text.size() || text[end] != '"') return true; + std::size_t next { end + 1 }; + while (next < text.size() && (text[next] == ' ' || text[next] == '\t')) ++next; + return next >= text.size() || text[next] != ':'; +} + +// A directory name: right after a separator, and not going on as a longer name. +bool directory_name_at(std::string_view text, std::size_t at, std::size_t length) { + if (at == 0 || (text[at - 1] != '/' && text[at - 1] != '\\')) return false; + return !name_continues(text, at + length); +} + +template +std::size_t replace_words(std::string& text, std::string_view word, std::string_view replacement, Accept accept) { + if (word.empty()) return 0; + std::string out; + std::size_t count { 0 }; + std::size_t copied { 0 }; + for (std::size_t at { ifind(text, word) }; at != std::string::npos; at = ifind(text, word, at + 1)) { + if (at < copied || !accept(std::string_view { text }, at, word.size())) continue; + out.append(text, copied, at - copied); + out += replacement; + copied = at + word.size(); + ++count; + } + if (count == 0) return 0; + out.append(text, copied); + text = std::move(out); + return count; +} + +// ---- secrets ----------------------------------------------------------------------------------- + +// The words of an identifier: "apiKey", "API_KEY", "x-api-key" and "ApiKey" are all {api, key}. +std::vector words_of(std::string_view name) { + std::vector words; + std::string current; + const auto flush = [&] { + if (!current.empty()) words.push_back(lowered(current)); + current.clear(); + }; + for (std::size_t i { 0 }; i < name.size(); ++i) { + const auto c = static_cast(name[i]); + if (!is_alnum(c)) { + flush(); + continue; + } + if (!current.empty()) { + const auto previous = static_cast(name[i - 1]); + const bool lowerToUpper { (previous >= 'a' && previous <= 'z') && (c >= 'A' && c <= 'Z') }; + // "APIKey": the K starts a word because a lower-case letter follows it. + const bool upperRunEnds { (previous >= 'A' && previous <= 'Z') && (c >= 'A' && c <= 'Z') && i + 1 < name.size() + && name[i + 1] >= 'a' && name[i + 1] <= 'z' }; + if (lowerToUpper || upperRunEnds) flush(); + } + current += static_cast(c); + } + flush(); + return words; +} + +constexpr std::array SECRET_WORDS { + "token", "secret", "secrets", "password", "passwords", "passwd", "pwd", "passphrase", "credential", "credentials", + "authorization", "auth", "cookie", "apikey", "privatekey", "accesskey", "secretkey", "bearer", +}; + +constexpr std::array, 4> SECRET_PAIRS { + std::pair { "api", "key" }, std::pair { "private", "key" }, std::pair { "access", "key" }, std::pair { "session", "key" }, +}; + +// Values that are not secrets whatever their key says. +bool harmless_value(std::string_view value) { + const std::string lowerValue { lowered(value) }; + return value.empty() || value.starts_with('<') || lowerValue == "null" || lowerValue == "true" || lowerValue == "false" + || lowerValue == "none" || lowerValue == "***"; +} + +// Whether the `"` at `at` is escaped by the backslashes before it. +bool escaped_quote(std::string_view text, std::size_t at) { + std::size_t backslashes { 0 }; + while (at > backslashes && text[at - 1 - backslashes] == '\\') ++backslashes; + return backslashes % 2 == 1; +} + +struct Span { + std::size_t begin { 0 }; + std::size_t end { 0 }; +}; + +// Spans are collected over the original text, then replaced in one pass. +std::string replace_spans(std::string_view text, std::vector spans, std::string_view replacement) { + std::ranges::sort(spans, {}, &Span::begin); + std::string out; + out.reserve(text.size()); + std::size_t copied { 0 }; + for (const auto& span : spans) { + if (span.begin < copied) continue; // overlaps one already replaced + out.append(text, copied, span.begin - copied); + out += replacement; + copied = span.end; + } + out.append(text, copied); + return out; +} + +// The end of a value that follows `=` or `: `: at white space, a quote that is not escaped, `&`, +// or the end of the line. +std::size_t value_end(std::string_view text, std::size_t begin, bool toEndOfLine = false) { + std::size_t j { begin }; + while (j < text.size()) { + const char c { text[j] }; + if (c == '\n' || c == '\r') break; + if (!toEndOfLine && (c == ' ' || c == '\t' || c == '\'' || c == '&')) break; + if (c == '"' && !escaped_quote(text, j)) break; + ++j; + } + return j; +} + +bool token_char(unsigned char c) { return is_alnum(c) || c == '_' || c == '-'; } + +constexpr std::array TOKEN_PREFIXES { + "ghp_", "gho_", "ghu_", "ghs_", "ghr_", "github_pat_", "glpat-", "xoxb-", "xoxp-", "xoxa-", "xoxr-", "sk-", "AKIA", +}; + +// Tokens by their well-known prefix: GitHub, GitLab, Slack, OpenAI and Anthropic keys, AWS access keys. +void find_prefixed_tokens(std::string_view text, std::vector& spans) { + for (const auto prefix : TOKEN_PREFIXES) { + const bool aws { prefix == "AKIA" }; + for (std::size_t at { text.find(prefix) }; at != std::string_view::npos; at = text.find(prefix, at + 1)) { + if (at > 0 && (is_alnum(static_cast(text[at - 1])) || text[at - 1] == '_')) continue; + std::size_t end { at + prefix.size() }; + while (end < text.size() && token_char(static_cast(text[end]))) ++end; + const std::size_t body { end - at - prefix.size() }; + if (aws ? body != 16 : body < 16) continue; + spans.push_back(Span { at, end }); + } + } +} + +// "scheme://user:password@host": the credentials. +void find_url_credentials(std::string_view text, std::vector& spans) { + for (std::size_t at { text.find("://") }; at != std::string_view::npos; at = text.find("://", at + 3)) { + const std::size_t begin { at + 3 }; + std::size_t j { begin }; + std::optional colon; + std::optional atSign; + while (j < text.size()) { + const char c { text[j] }; + if (c == '/' || c == ' ' || c == '"' || c == '\'' || c == '\n' || c == '\\' || c == '?' || c == '#') break; + if (c == ':' && !colon) colon = j; + if (c == '@') atSign = j; + ++j; + } + if (atSign && colon && *colon < *atSign && *atSign > begin) spans.push_back(Span { begin, *atSign }); + } +} + +// "Bearer ", "Basic ". +void find_authorization_schemes(std::string_view text, std::vector& spans) { + for (const std::string_view scheme : { "Bearer ", "bearer ", "Basic ", "basic " }) { + for (std::size_t at { text.find(scheme) }; at != std::string_view::npos; at = text.find(scheme, at + 1)) { + if (at > 0 && is_alnum(static_cast(text[at - 1]))) continue; + const std::size_t begin { at + scheme.size() }; + std::size_t end { begin }; + while (end < text.size() && (token_char(static_cast(text[end])) || std::string_view { "._~+/=" }.contains(text[end]))) ++end; + if (end - begin >= 8 && !harmless_value(text.substr(begin, end - begin))) spans.push_back(Span { begin, end }); + } + } +} + +// "key": "value", where the key names a secret: the value. +void find_json_secrets(std::string_view text, std::vector& spans) { + for (std::size_t open { text.find('"') }; open != std::string_view::npos;) { + std::size_t close { open + 1 }; + while (close < text.size() && close - open <= 64 && text[close] != '"' && text[close] != '\n') { + close += text[close] == '\\' ? 2 : 1; + } + if (close >= text.size() || text[close] != '"' || close - open > 64) { + open = text.find('"', open + 1); + continue; + } + std::size_t j { close + 1 }; + while (j < text.size() && (text[j] == ' ' || text[j] == '\t')) ++j; + if (j >= text.size() || text[j] != ':') { + open = text.find('"', close); + continue; + } + ++j; + while (j < text.size() && (text[j] == ' ' || text[j] == '\t')) ++j; + if (j < text.size() && text[j] == '"' && secret_name(text.substr(open + 1, close - open - 1))) { + std::size_t end { j + 1 }; + while (end < text.size() && !(text[end] == '"' && !escaped_quote(text, end)) && text[end] != '\n') ++end; + if (end < text.size() && text[end] == '"' && !harmless_value(text.substr(j + 1, end - j - 1))) spans.push_back(Span { j + 1, end }); + open = end < text.size() ? text.find('"', end + 1) : std::string_view::npos; + continue; + } + open = text.find('"', close + 1); + } +} + +// NAME=value (an environment variable, a --flag=value, a -DNAME=value define) whose name names a secret. +void find_assigned_secrets(std::string_view text, std::vector& spans) { + for (std::size_t equals { text.find('=') }; equals != std::string_view::npos; equals = text.find('=', equals + 1)) { + std::size_t start { equals }; + while (start > 0 && (is_alnum(static_cast(text[start - 1])) || text[start - 1] == '_' || text[start - 1] == '-' + || text[start - 1] == '.')) { + --start; + } + while (start < equals && text[start] == '-') ++start; // --flag + if (start == equals) continue; + std::string_view name { text.substr(start, equals - start) }; + bool secret { secret_name(name) }; + // -DAPI_KEY=...: the define's name follows the D. + if (!secret && start >= 1 && text[start - 1] == '-' && name.size() > 1 && name.front() == 'D') secret = secret_name(name.substr(1)); + if (!secret) continue; + const std::size_t end { value_end(text, equals + 1) }; + if (end > equals + 1 && !harmless_value(text.substr(equals + 1, end - equals - 1))) spans.push_back(Span { equals + 1, end }); + } +} + +// "Authorization: ...", "password: ..." in plain text: the rest of the line. Only names that are +// never anything else; a bare "token:" is too common in a log to mean a secret. +void find_header_secrets(std::string_view text, std::vector& spans) { + static constexpr std::array HEADERS { "authorization:", "proxy-authorization:", "password:", "passwd:", "x-api-key:", + "api-key:", "api_key:", "private-token:", "client_secret:", "secret:" }; + for (const auto header : HEADERS) { + for (std::size_t at { ifind(text, header) }; at != std::string_view::npos; at = ifind(text, header, at + 1)) { + if (at > 0 && (is_alnum(static_cast(text[at - 1])) || text[at - 1] == '_' || text[at - 1] == '-' || text[at - 1] == '"')) continue; + std::size_t begin { at + header.size() }; + while (begin < text.size() && (text[begin] == ' ' || text[begin] == '\t')) ++begin; + const std::size_t end { value_end(text, begin, true) }; + if (end > begin && !harmless_value(text.substr(begin, end - begin))) spans.push_back(Span { begin, end }); + } + } +} + +bool email_local_char(unsigned char c) { return is_alnum(c) || std::string_view { "._%+-" }.contains(static_cast(c)); } + +// local@domain.tld, the domain with at least two labels and an alphabetic top-level label. +void find_emails(std::string_view text, std::vector& spans) { + for (std::size_t at { text.find('@') }; at != std::string_view::npos; at = text.find('@', at + 1)) { + std::size_t begin { at }; + while (begin > 0 && at - begin < 64 && email_local_char(static_cast(text[begin - 1]))) --begin; + if (begin == at || text[begin] == '.' || text[at - 1] == '.') continue; + if (lowered(text.substr(begin, at - begin)) == "git") continue; // git@github.com:owner/repo is an address, not a person + std::size_t end { at + 1 }; + std::size_t labels { 0 }; + bool alphabeticTop { false }; + while (end < text.size()) { + const std::size_t labelBegin { end }; + while (end < text.size() && (is_alnum(static_cast(text[end])) || text[end] == '-')) ++end; + if (end == labelBegin) break; + ++labels; + alphabeticTop = end - labelBegin >= 2 && std::ranges::all_of(text.substr(labelBegin, end - labelBegin), [](char c) { return is_alpha(static_cast(c)); }); + if (end + 1 < text.size() && text[end] == '.' && is_alnum(static_cast(text[end + 1]))) { + ++end; + continue; + } + break; + } + if (labels >= 2 && alphabeticTop) spans.push_back(Span { begin, end }); + } +} + +constexpr std::array COMMON_NAMES { + "admin", "administrator", "user", "users", "guest", "root", "test", "tests", "tester", "runner", "build", "builder", + "developer", "home", "public", "default", "shared", "ubuntu", "debian", "fedora", "centos", "docker", "vagrant", "ec2-user", + "macos", "windows", "linux", "local", "localhost", "owner", "work", "workspace", "code", "mcpp", "clang", "clangd", "server", + "client", "data", "temp", "system", "service", "demo", +}; + +// Directories under a profile root that are no person's. +constexpr std::array NOBODY { "public", "default", "default user", "all users", "shared", "linuxbrew", "guest", "defaultapppool" }; + +} // namespace + +bool distinctive_name(std::string_view name) { + if (name.size() < 4) return false; + const std::string lowerName { lowered(name) }; + return std::ranges::find(COMMON_NAMES, std::string_view { lowerName }) == COMMON_NAMES.end(); +} + +bool secret_name(std::string_view name) { + const auto words = words_of(name); + for (std::size_t i { 0 }; i < words.size(); ++i) { + if (std::ranges::find(SECRET_WORDS, std::string_view { words[i] }) != SECRET_WORDS.end()) return true; + if (i + 1 < words.size()) { + for (const auto& [first, second] : SECRET_PAIRS) { + if (words[i] == first && words[i + 1] == second) return true; + } + } + } + return false; +} + +// A path and the other spelling macOS gives the same directory: /var, /tmp and /etc are links into /private, so a +// workspace the server knows as /private/var/folders/.../T/x is /var/folders/.../T/x to the editor (CI, 2026-09-26). +std::vector spellings_of(const std::string& path) { + std::vector spellings { path }; + for (const std::string_view top : { std::string_view { "/var" }, std::string_view { "/tmp" }, std::string_view { "/etc" } }) { + const std::string privateTop { std::format("/private{}", top) }; + const auto under = [&](std::string_view root) { return path == root || (path.starts_with(root) && path[root.size()] == '/'); }; + if (under(privateTop)) spellings.push_back(path.substr(8)); + else if (under(top)) spellings.push_back("/private" + path); + } + return spellings; +} + +struct Redactor::Impl { + Identity identity; + std::vector workspacePatterns; // longest first + std::vector homePatterns; // longest first + std::vector homeTexts; // each home in its '/' and '\' spellings, for the residue check + std::vector shortNames; // 8.3 prefixes of the user names ("RUNNER" of runneradmin) + std::vector users; + std::vector hosts; + std::map> userPlaceholders; // lower-case name -> , , ... + std::map> hits; + + explicit Impl(Identity who) : identity { std::move(who) } { + const auto byLength = [](const PathPattern& a, const PathPattern& b) { return a.pieces.size() > b.pieces.size(); }; + for (std::size_t i { 0 }; i < identity.workspaces.size(); ++i) { + if (identity.workspaces[i].empty()) continue; + const std::string placeholder { i == 0 ? std::string { "" } : std::format("", i + 1) }; + for (const auto& spelling : spellings_of(identity.workspaces[i])) { + for (auto& pattern : path_patterns(spelling, RULE_WORKSPACE, placeholder)) workspacePatterns.push_back(std::move(pattern)); + } + } + std::ranges::stable_sort(workspacePatterns, byLength); + std::vector homes; + for (const auto& home : identity.homes) { + for (auto& spelling : spellings_of(home)) { + if (std::ranges::find(homes, spelling) == homes.end()) homes.push_back(std::move(spelling)); + } + } + for (const auto& home : homes) { + if (segments_of(home).empty()) continue; // "/" or "C:/": nothing personal to hide + for (auto& pattern : path_patterns(home, RULE_HOME, "~")) homePatterns.push_back(std::move(pattern)); + // Only a home named for a distinctive user is looked for as a plain string too: "/root" is in + // "/usr/lib/root/x" and "/home/runner" in "/opt/home/runner", which are nobody's home. + if (!distinctive_name(segments_of(home).back())) continue; + std::string backslashed { home }; + std::ranges::replace(backslashed, '/', '\\'); + homeTexts.push_back(home); + homeTexts.push_back(std::move(backslashed)); + } + std::ranges::stable_sort(homePatterns, byLength); + for (const auto& user : identity.users) { + if (user.empty() || std::ranges::find_if(users, [&](const std::string& known) { return lowered(known) == lowered(user); }) != users.end()) continue; + users.push_back(user); + userPlaceholders.emplace(lowered(user), ""); + if (auto prefix = short_name_prefix(user)) shortNames.push_back(std::move(*prefix)); + } + for (const auto& host : identity.hosts) { + if (host.empty()) continue; + hosts.push_back(host); + // "mac-mini.local": the name before the domain is the machine too. + if (const auto dot = host.find('.'); dot != std::string::npos && dot > 0) hosts.push_back(host.substr(0, dot)); + } + std::ranges::sort(hosts, [](const std::string& a, const std::string& b) { return a.size() > b.size(); }); + } + + void count(std::string_view rule, std::size_t n) { + if (n == 0) return; + auto it = hits.find(rule); + if (it == hits.end()) it = hits.emplace(std::string { rule }, 0).first; + it->second += n; + } + + std::string placeholder_for_user(std::string_view name) { + const std::string key { lowered(name) }; + if (const auto known = userPlaceholders.find(key); known != userPlaceholders.end()) return known->second; + // is the user's own; everybody else counts from 2, in the order they are met. + const auto others = std::ranges::count_if(userPlaceholders, [](const auto& item) { return item.second != ""; }); + std::string placeholder { std::format("", others + 2) }; + userPlaceholders.emplace(key, placeholder); + return placeholder; + } + + // Profile directories of other people: /home/, /Users/, C:\Users\, /mnt/c/Users/. + // What follows the profile root is the person's name, whoever it is. + std::size_t replace_profile_directories(std::string& text) { + return replace_profile_directories(text, "home") + replace_profile_directories(text, "users"); + } + + std::size_t replace_profile_directories(std::string& text, std::string_view root) { + std::size_t count { 0 }; + std::string out; + std::size_t copied { 0 }; + for (std::size_t at { ifind(text, root) }; at != std::string::npos; at = ifind(text, root, at + 1)) { + if (at < copied || at == 0) continue; + // The root segment follows a separator run that starts the path, follows a drive, or follows /mnt/. + std::size_t runStart { at }; + while (runStart > 0 && (text[runStart - 1] == '/' || text[runStart - 1] == '\\')) --runStart; + if (runStart == at) continue; + const auto before = [&](std::size_t back) { return static_cast(text[runStart - back]); }; + const bool atRoot { path_starts_at(text, runStart, false) }; + const bool afterDrive { runStart >= 2 && before(1) == ':' && is_alpha(before(2)) && path_starts_at(text, runStart - 2, true) }; + const bool afterMount { runStart >= 6 && is_alpha(before(1)) && lowered(std::string_view { text }.substr(runStart - 6, 5)) == "/mnt/" }; + if (!atRoot && !afterDrive && !afterMount) continue; + const std::size_t separator { match_separator(text, at + root.size()) }; + if (separator == 0) continue; + const std::size_t nameBegin { at + root.size() + separator }; + // The name ends at the next separator; a space belongs to it only when a separator follows the name. + static constexpr std::string_view ENDS_NAME { "/\\\"'\n\r\t:<>|*?,;()[]{}" }; + std::size_t nameEnd { nameBegin }; + std::optional firstSpace; + while (nameEnd < text.size() && !ENDS_NAME.contains(text[nameEnd])) { + if (text[nameEnd] == ' ' && !firstSpace) firstSpace = nameEnd; + ++nameEnd; + } + const bool separatorFollows { nameEnd < text.size() && (text[nameEnd] == '/' || text[nameEnd] == '\\') }; + if (firstSpace && !separatorFollows) nameEnd = *firstSpace; + while (nameEnd > nameBegin && text[nameEnd - 1] == '.') --nameEnd; // the end of a sentence, not of the name + if (nameEnd == nameBegin) continue; + const std::string_view name { std::string_view { text }.substr(nameBegin, nameEnd - nameBegin) }; + if (name.starts_with('<') || name.starts_with('~') || name.starts_with('%') || name.starts_with('$') + || std::ranges::find(NOBODY, std::string_view { lowered(name) }) != NOBODY.end()) continue; + out.append(text, copied, nameBegin - copied); + out += placeholder_for_user(name); + copied = nameEnd; + ++count; + } + if (count == 0) return 0; + out.append(text, copied); + text = std::move(out); + return count; + } +}; + +Redactor::Redactor(Identity identity) : impl_ { std::make_unique(std::move(identity)) } {} +Redactor::~Redactor() = default; +Redactor::Redactor(Redactor&&) noexcept = default; +Redactor& Redactor::operator=(Redactor&&) noexcept = default; + +const std::map>& Redactor::hits() const { return impl_->hits; } + +std::string Redactor::redact(std::string_view input) { + Impl& impl { *impl_ }; + // Secrets first, on the text as it came: a token in a URL or a define is found by its shape + // before any path around it is rewritten. + std::vector secrets; + find_prefixed_tokens(input, secrets); + find_url_credentials(input, secrets); + find_authorization_schemes(input, secrets); + find_json_secrets(input, secrets); + find_assigned_secrets(input, secrets); + find_header_secrets(input, secrets); + std::string text; + if (secrets.empty()) { + text = std::string { input }; + } else { + impl.count(RULE_SECRET, secrets.size()); + text = replace_spans(input, std::move(secrets), ""); + } + std::vector emails; + find_emails(text, emails); + if (!emails.empty()) { + impl.count(RULE_EMAIL, emails.size()); + text = replace_spans(text, std::move(emails), ""); + } + // The project's own paths before the home they are under, the longest spelling first. + for (const auto& pattern : impl.workspacePatterns) impl.count(RULE_WORKSPACE, replace_path(text, pattern)); + for (const auto& pattern : impl.homePatterns) impl.count(RULE_HOME, replace_path(text, pattern)); + // A profile under its 8.3 name without the drive before it (RUNNER~1). + for (const auto& prefix : impl.shortNames) { + std::size_t n { 0 }; + for (std::size_t at { ifind(text, prefix) }; at != std::string::npos; at = ifind(text, prefix, at + 1)) { + if (at > 0 && is_word(static_cast(text[at - 1]))) continue; + const std::size_t length { match_short_name(text, at, prefix) }; + if (length == 0 || (at + length < text.size() && is_word(static_cast(text[at + length])))) continue; + text.replace(at, length, ""); + ++n; + } + impl.count(RULE_SHORT_NAME, n); + } + impl.count(RULE_USER, impl.replace_profile_directories(text)); + for (const auto& user : impl.users) { + impl.count(RULE_USER, distinctive_name(user) ? replace_words(text, user, "", name_word_at) + : replace_words(text, user, "", directory_name_at)); + } + for (const auto& host : impl.hosts) { + // A host name that is a common word ("ubuntu", "localhost") identifies nobody and stays. + if (distinctive_name(host)) impl.count(RULE_HOST, replace_words(text, host, "", name_word_at)); + } + return text; +} + +Json Redactor::redact_json(const Json& value) { + const std::string redacted { redact(value.dump(-1, ' ', false, Json::error_handler_t::replace)) }; + Json parsed = Json::parse(redacted, nullptr, false); + // Never expected: no placeholder has a quote or a backslash, and no rule ends inside an escape. + // Should it happen, the text is kept rather than the original. + return parsed.is_discarded() ? Json(redacted) : parsed; +} + +std::vector Redactor::residue(std::string_view text, std::size_t limit) const { + const Impl& impl { *impl_ }; + std::vector found; + const auto add = [&](std::string_view rule, std::size_t offset) { + if (found.size() < limit) found.push_back(Residue { std::string { rule }, offset }); + }; + for (const auto& pattern : impl.homePatterns) { + if (const auto at = find_path(text, pattern)) add(RULE_HOME, *at); + } + // The home as a plain string, whatever comes before it: stricter than the rule that replaced it, so + // a spelling that rule does not know is found here rather than shipped. Not "/home/speaker" for + // "/home/speak", which is another directory. + for (const auto& home : impl.homeTexts) { + for (std::size_t at { ifind(text, home) }; at != std::string_view::npos; at = ifind(text, home, at + 1)) { + const std::size_t end { at + home.size() }; + if (end < text.size() && (is_word(static_cast(text[end])) || text[end] == '_')) continue; + add(RULE_HOME, at); + break; + } + } + for (const auto& pattern : impl.workspacePatterns) { + if (const auto at = find_path(text, pattern)) add(RULE_WORKSPACE, *at); + } + for (const auto& prefix : impl.shortNames) { + for (std::size_t at { ifind(text, prefix) }; at != std::string_view::npos; at = ifind(text, prefix, at + 1)) { + if (match_short_name(text, at, prefix) > 0) { + add(RULE_SHORT_NAME, at); + break; + } + } + } + for (const auto& user : impl.users) { + const bool distinctive { distinctive_name(user) }; + for (std::size_t at { ifind(text, user) }; at != std::string_view::npos; at = ifind(text, user, at + 1)) { + if (distinctive ? name_word_at(text, at, user.size()) : directory_name_at(text, at, user.size())) { + add(RULE_USER, at); + break; + } + } + } + for (const auto& host : impl.hosts) { + if (!distinctive_name(host)) continue; + for (std::size_t at { ifind(text, host) }; at != std::string_view::npos; at = ifind(text, host, at + 1)) { + if (name_word_at(text, at, host.size())) { + add(RULE_HOST, at); + break; + } + } + } + // Every detector redact() runs, again, on what it produced: the check is the last word on what leaves the + // machine, not the rules' own opinion of themselves. A match that is only a placeholder is what redact() left there. + const auto placeholder_only = [&](const Span& span) { + std::string_view matched { text.substr(span.begin, span.end - span.begin) }; + while (!matched.empty() && (matched.front() == ' ' || matched.front() == '"' || matched.front() == '\'')) matched.remove_prefix(1); + while (!matched.empty() && (matched.back() == ' ' || matched.back() == '"' || matched.back() == '\'')) matched.remove_suffix(1); + return matched.empty() || matched == ""; + }; + std::vector secrets; + find_prefixed_tokens(text, secrets); + find_url_credentials(text, secrets); + find_authorization_schemes(text, secrets); + find_json_secrets(text, secrets); + find_assigned_secrets(text, secrets); + find_header_secrets(text, secrets); + for (const auto& secret : secrets) { + if (!placeholder_only(secret)) add(RULE_SECRET, secret.begin); + } + std::vector emails; + find_emails(text, emails); + for (const auto& email : emails) { + if (!placeholder_only(email)) add(RULE_EMAIL, email.begin); + } + std::ranges::sort(found, {}, &Residue::offset); + return found; +} + +} // namespace mcppls::bundle diff --git a/src/bundle/redact.cppm b/src/bundle/redact.cppm new file mode 100644 index 0000000..4d29a95 --- /dev/null +++ b/src/bundle/redact.cppm @@ -0,0 +1,75 @@ +// What leaves this machine in a report or a diagnostic bundle (issue #23 fix plan F18): the user's +// home directory, the user's and the machine's names and anything that looks like a secret are +// replaced by placeholders, the same original always by the same placeholder so that paths stay +// comparable, and a check afterwards finds whatever is left. The mapping from placeholder back to +// original lives only in the Redactor and is never written anywhere. +// +// Pure text in, text out: no file, no environment. Who the user is comes in as an Identity, which +// mcppls.bundle.bundle reads from the machine. +export module mcppls.bundle.redact; + +import std; +import nlohmann.json; + +export namespace mcppls::bundle { + +// Who a bundle must not name. +struct Identity { + std::vector homes; // the home directory, '/'-separated, in each spelling known (as set, canonical) + std::vector users; // the login name and the home directory's last component + std::vector hosts; // the machine's names + std::vector workspaces; // roots replaced by , , ...; empty: roots are kept +}; + +// The rules, by the id the manifest counts them under. +inline constexpr std::string_view RULE_HOME { "home-directory" }; +inline constexpr std::string_view RULE_WORKSPACE { "workspace-root" }; +inline constexpr std::string_view RULE_USER { "user-name" }; +inline constexpr std::string_view RULE_SHORT_NAME { "short-name" }; +inline constexpr std::string_view RULE_HOST { "host-name" }; +inline constexpr std::string_view RULE_SECRET { "secret" }; +inline constexpr std::string_view RULE_EMAIL { "email" }; + +// Whether a user or host name is distinctive enough to be replaced wherever it stands as a word. +// A short name ("a", "bob") or a common word or generic account ("admin", "runner", "ubuntu") is +// replaced only where it names a directory: replacing it everywhere would rewrite ordinary words, +// and a check for it everywhere would fail every bundle of that user for ever. +bool distinctive_name(std::string_view name); + +// Whether a key or variable name (camelCase, snake_case, kebab-case, UPPER_CASE) names a secret: +// token, secret, password, api key, authorization, ... +bool secret_name(std::string_view name); + +// A place where something the Identity names was found after redaction. +struct Residue { + std::string rule; + std::size_t offset { 0 }; +}; + +class Redactor { +public: + explicit Redactor(Identity identity); + ~Redactor(); + Redactor(Redactor&&) noexcept; + Redactor& operator=(Redactor&&) noexcept; + + // The text with every rule applied, counting what each replaced. + std::string redact(std::string_view text); + // A JSON value redacted through its text, parsed back; its structure is unchanged, since no + // placeholder carries a quote or a backslash. + nlohmann::json redact_json(const nlohmann::json& value); + + // What the Identity names that is still in `text`: the home directory in any spelling, the user + // and host names under the same rules as redact(), and anything that looks like a known token. + // At most `limit` places. + std::vector residue(std::string_view text, std::size_t limit = 8) const; + + // Replacements made so far, by rule id. + const std::map>& hits() const; + +private: + struct Impl; + std::unique_ptr impl_; +}; + +} // namespace mcppls::bundle diff --git a/src/bundle/writer.cpp b/src/bundle/writer.cpp new file mode 100644 index 0000000..b838831 --- /dev/null +++ b/src/bundle/writer.cpp @@ -0,0 +1,536 @@ +module mcppls.bundle.writer; + +import std; +import nlohmann.json; +import mcppls.os; +import mcppls.base.log; +import mcppls.base.path; +import mcppls.base.sha256; +import mcppls.base.text; +import mcppls.base.version; +import mcppls.platform.dirs; +import mcppls.platform.env; +import mcppls.platform.fs; +import mcppls.platform.process; +import mcppls.platform.toolrun; +import mcppls.bundle.identity; +import mcppls.bundle.redact; +import mcppls.bundle.zip; + +namespace mcppls::bundle { + +namespace { + +using Json = nlohmann::json; +namespace log = base::log; + +// What goes first when the bundle would be larger than its cap: the report and the environment +// always, then the incidents, the engine, the client's log, the server's logs newest first, dumps last. +enum class Kind { report, environment, incident, engine, client_log, log, dump }; + +struct Candidate { + std::string name; // in the archive + std::string content; + Kind kind { Kind::report }; + bool text { true }; // redacted and checked; a dump is neither + bool truncated { false }; + std::string note; +}; + +constexpr std::size_t KIB { 1024 }; +constexpr std::size_t MIB { 1024 * KIB }; +constexpr std::size_t LOG_FILE_CAP { 2 * MIB }; +constexpr std::size_t LOG_HEAD { 256 * KIB }; +constexpr std::size_t INCIDENT_FILE_CAP { 4 * MIB }; +constexpr std::size_t DATABASE_CAP { 8 * MIB }; +constexpr std::size_t PROBES_CAP { 1 * MIB }; +constexpr std::size_t CLIENT_LOG_CAP { 2 * MIB }; +constexpr std::uint64_t MANIFEST_RESERVE { 256 * KIB }; +constexpr std::size_t LOG_SESSIONS { 3 }; // the last sessions, whenever they were +constexpr std::size_t LOG_SESSIONS_MAX { 6 }; // and any other of the last day, up to this many in all + +std::string dump(const Json& value) { return value.dump(2, ' ', false, Json::error_handler_t::replace); } + +std::string utc_now(std::string_view format) { + const auto now = std::chrono::floor(std::chrono::system_clock::now()); + return std::vformat(format, std::make_format_args(now)); +} + +bool binary(std::string_view content) { return content.substr(0, std::min(content.size(), 8 * KIB)).contains('\0'); } + +// The first and the last of a long text, with a line that says how much was left out. +std::string head_and_tail(std::string content, std::size_t cap, bool& truncated) { + if (content.size() <= cap) return content; + truncated = true; + const std::size_t tail { cap - LOG_HEAD }; + std::string kept { content.substr(0, LOG_HEAD) }; + kept += std::format("\n[... mcppls left out {} bytes here to keep the bundle small ...]\n", content.size() - LOG_HEAD - tail); + kept += content.substr(content.size() - tail); + return kept; +} + +std::optional read_text(std::string_view path) { + auto content = platform::fs::read_file(path); + if (!content) return std::nullopt; + return std::move(*content); +} + +// A system's own answer about itself (a version, a code page), bounded and in this process's own +// environment: never the login shell, never the network. +std::optional system_answer(std::string_view program, std::vector arguments) { + const auto executable = platform::env::find_executable(program); + if (!executable) return std::nullopt; + auto result = platform::toolrun::run(platform::toolrun::Request { + .program = *executable, + .arguments = std::move(arguments), + .workDirectory = platform::dirs::temp_directory(), + .purpose = "bundle", + .bounds = platform::RunBounds { .hard = std::chrono::seconds { 5 } }, + .environment = platform::env::variables(), + }); + if (!result || result->exitCode != 0 || result->timedOut) return std::nullopt; + return std::move(result->output); +} + +// "Key:\tvalue" lines, the way sw_vers answers. +std::string field(std::string_view text, std::string_view key) { + for (const auto line : base::split_lines(text)) { + const auto trimmed = base::trim(line); + if (trimmed.starts_with(key)) { + auto rest = trimmed.substr(key.size()); + if (!rest.empty() && rest.front() == ':') rest.remove_prefix(1); + return std::string { base::trim(rest) }; + } + } + return {}; +} + +std::string unquoted(std::string_view value) { + value = base::trim(value); + if (value.size() >= 2 && (value.front() == '"' || value.front() == '\'') && value.back() == value.front()) value = value.substr(1, value.size() - 2); + return std::string { value }; +} + +Json operating_system() { + Json os { { "platform", std::string { mcppls::os::PLATFORM } }, { "family", std::string { mcppls::os::FAMILY_NAME } } }; + if constexpr (mcppls::os::FAMILY == mcppls::os::Family::linux) { + if (auto release = read_text("/proc/sys/kernel/osrelease")) os["kernel"] = std::string { base::trim(*release) }; + if (auto release = read_text("/etc/os-release")) { + for (const auto line : base::split_lines(*release)) { + if (line.starts_with("PRETTY_NAME=")) os["distribution"] = unquoted(line.substr(12)); + } + } + } else if constexpr (mcppls::os::FAMILY == mcppls::os::Family::macos) { + if (auto versions = system_answer("sw_vers", {})) { + os["name"] = field(*versions, "ProductName"); + os["version"] = field(*versions, "ProductVersion"); + os["build"] = field(*versions, "BuildVersion"); + } + } else { + if (auto version = system_answer("cmd", { "/d", "/c", "ver" })) os["version"] = std::string { base::trim(*version) }; + } + return os; +} + +Json memory() { + if constexpr (mcppls::os::FAMILY == mcppls::os::Family::linux) { + if (auto info = read_text("/proc/meminfo")) { + for (const auto line : base::split_lines(*info)) { + if (!line.starts_with("MemTotal:")) continue; + std::uint64_t kib { 0 }; + const auto digits = base::trim(line.substr(9)); + (void)std::from_chars(digits.data(), digits.data() + digits.size(), kib); + return Json { { "totalBytes", kib * 1024 } }; + } + } + } else if constexpr (mcppls::os::FAMILY == mcppls::os::Family::macos) { + if (auto bytes = system_answer("sysctl", { "-n", "hw.memsize" })) { + std::uint64_t total { 0 }; + const auto digits = base::trim(*bytes); + (void)std::from_chars(digits.data(), digits.data() + digits.size(), total); + return Json { { "totalBytes", total } }; + } + } + return nullptr; +} + +Json locale() { + Json locale = Json::object(); + for (const std::string_view name : { "LANG", "LC_ALL", "LC_CTYPE" }) { + if (auto value = platform::env::get(name)) locale[std::string { name }] = *value; + } + if constexpr (mcppls::os::FAMILY == mcppls::os::Family::windows) { + // The ANSI code page is what a narrow string from the system is in (issue #23 had a GBK machine). + if (auto answer = system_answer("reg", { "query", "HKLM\\SYSTEM\\CurrentControlSet\\Control\\Nls\\CodePage", "/v", "ACP" })) { + for (const auto line : base::split_lines(*answer)) { + const auto words = base::split(base::trim(line), ' '); + std::vector parts; + for (const auto word : words) { + if (!base::trim(word).empty()) parts.push_back(base::trim(word)); + } + if (parts.size() >= 3 && parts.front() == "ACP") locale["ansiCodePage"] = std::string { parts.back() }; + } + } + } + return locale; +} + +// Only these: a variable can carry a credential, and nothing else here needs one. +bool whitelisted_variable(std::string_view name) { + const std::string upper { [&] { + std::string out { name }; + for (char& c : out) c = static_cast(std::toupper(static_cast(c))); + return out; + }() }; + return upper.starts_with("MCPP_") || upper.starts_with("XLINGS_") || upper.starts_with("LC_") || upper == "LANG" || upper == "PATH"; +} + +Json environment_variables() { + Json variables = Json::object(); + for (const auto& entry : platform::env::variables()) { + const std::size_t equals { entry.find('=', 1) }; + if (equals == std::string::npos) continue; + const std::string name { entry.substr(0, equals) }; + if (whitelisted_variable(name)) variables[name] = entry.substr(equals + 1); + } + return variables; +} + +Json environment(const BundleInput& input) { + const Json& report { input.report }; + Json versions { { "mcppls", std::string { base::VERSION } } }; + if (const auto payload = report.find("payload"); payload != report.end() && payload->is_object()) { + versions["clangd"] = payload->value("clangdVersion", std::string {}); + } + Json producers = Json::array(); + if (const auto roots = report.find("roots"); roots != report.end() && roots->is_array()) { + for (const auto& root : *roots) { + const Json* project { root.contains("project") && root["project"].is_object() ? &root["project"] : nullptr }; + if (project == nullptr) continue; + producers.push_back(Json { { "producer", project->value("producer", std::string {}) }, { "version", project->value("producerVersion", std::string {}) } }); + } + } + versions["producers"] = std::move(producers); + Json payload = nullptr; + const std::string payloadDirectory { report.contains("payload") && report["payload"].is_object() ? report["payload"].value("directory", std::string {}) : std::string {} }; + if (!payloadDirectory.empty()) { + if (auto text = read_text(base::join_path(payloadDirectory, "payload.json"))) { + payload = Json::parse(*text, nullptr, false); + if (payload.is_discarded()) payload = nullptr; + } + } + Json probes = nullptr; + if (auto text = read_text(base::join_path(platform::dirs::cache_directory(), "toolchains/probe.json")); text && text->size() <= PROBES_CAP) { + probes = Json::parse(*text, nullptr, false); + if (probes.is_discarded()) probes = nullptr; + } + return Json { + { "generatedAt", utc_now("{:%FT%TZ}") }, + { "os", operating_system() }, + { "cpu", Json { { "logicalProcessors", std::thread::hardware_concurrency() } } }, + { "memory", memory() }, + { "locale", locale() }, + { "editor", report.value("client", Json(nullptr)) }, + { "client", input.client }, + { "initializationOptions", input.initializationOptions }, + { "server", report.value("server", Json(nullptr)) }, + { "payload", payload }, + { "versions", std::move(versions) }, + { "toolchainProbes", std::move(probes) }, + { "environmentVariables", environment_variables() }, + }; +} + +// server---.log and its rotations (.log.1, .log.2): the time it started. +std::optional session_start(std::string_view name) { + static constexpr std::string_view PREFIX { "server-" }; + if (!name.starts_with(PREFIX) || name.size() < PREFIX.size() + 15) return std::nullopt; + const std::string_view stamp { name.substr(PREFIX.size(), 15) }; + int year { 0 }, month { 0 }, day { 0 }, hour { 0 }, minute { 0 }, second { 0 }; + const auto number = [&](std::size_t at, std::size_t length, int& out) { + return std::from_chars(stamp.data() + at, stamp.data() + at + length, out).ec == std::errc {}; + }; + if (stamp[8] != '-' || !number(0, 4, year) || !number(4, 2, month) || !number(6, 2, day) || !number(9, 2, hour) || !number(11, 2, minute) + || !number(13, 2, second)) { + return std::nullopt; + } + const std::chrono::year_month_day date { std::chrono::year { year }, std::chrono::month { static_cast(month) }, + std::chrono::day { static_cast(day) } }; + if (!date.ok()) return std::nullopt; + return std::chrono::sys_days { date } + std::chrono::hours { hour } + std::chrono::minutes { minute } + std::chrono::seconds { second }; +} + +// The server's logs worth reading: the last LOG_SESSIONS sessions, and any other of the last day, +// up to LOG_SESSIONS_MAX in all, each with its rotations, newest first. +std::vector recent_logs(std::string_view directory, std::string_view currentLog) { + std::map, std::greater<>> sessions; // "server-