Skip to content

Commit 41610aa

Browse files
hopelyndhopelynd
committed
test: isolate HOME/config and host agent detection in unit and e2e tests
1 parent 0a700ff commit 41610aa

38 files changed

Lines changed: 1284 additions & 251 deletions

‎AGENTS.md‎

Lines changed: 21 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -57,26 +57,27 @@ Skill / 命令手册随 `skills/bailian-*/` 经 `bl skill init` 安装（装齐
5757

5858
按当前任务从下表挑一条进入对应文档:
5959

60-
| 场景 | 何时进入 | 详见 |
61-
| ----------------- | ------------------------------------------------ | ---------------------------------------------------------------------------- |
62-
| 命令增删改 | 增加 / 删除 / 重命名 `bl xxx` 或入口命令路径 | [docs/agents/command-add-remove.md](docs/agents/command-add-remove.md) |
63-
| E2E 测试维护 | 新增/改命令或 e2e 用例、补 help/缺参/dry-run | [docs/agents/cli-e2e-tests.md](docs/agents/cli-e2e-tests.md) |
64-
| 批量压测 | 改/跑多能力并发压测、`test:stress`、fixtures | [docs/agents/stress-batch-tests.md](docs/agents/stress-batch-tests.md) |
65-
| 选项变更 | 给已有命令加 `--flag` 或改默认值 | [docs/agents/command-flag-change.md](docs/agents/command-flag-change.md) |
66-
| 模型上下架 | 增加新模型 / 改默认模型 / 废弃旧模型 | [docs/agents/model-add-remove.md](docs/agents/model-add-remove.md) |
67-
| Skill 文案 / 路由 | 改 SKILL 路由、安装约定、hand-off、hub/领域边界 | [docs/agents/skill-change.md](docs/agents/skill-change.md) |
68-
| 错误文案变更 | 改 `BailianError` 的 message 或 hint | [docs/agents/error-hint-change.md](docs/agents/error-hint-change.md) |
69-
| URL / 渠道变更 | 控制台域名 / 文档站 / 追踪参数 | [docs/agents/url-change.md](docs/agents/url-change.md) |
70-
| 埋点变更 | 改 AEM 命令事件、后端渠道 header、User-Agent | [docs/agents/telemetry-change.md](docs/agents/telemetry-change.md) |
71-
| 鉴权扩展 | 加 OAuth / SSO / 换 token 来源 | [docs/agents/auth-change.md](docs/agents/auth-change.md) |
72-
| 配置项扩展 | 新 env var 或 `~/.bailian/config.json` 字段 | [docs/agents/config-add.md](docs/agents/config-add.md) |
73-
| Profile / 激活 | 改命名 Profile、预设或 `active_config` | [docs/agents/config-profile-change.md](docs/agents/config-profile-change.md) |
74-
| 安装文档 | 改安装、鉴权、验证流程或线上 install 页面 | [docs/agents/install-doc-change.md](docs/agents/install-doc-change.md) |
75-
| 发布 | channel / stable 发布到 npm（CI 驱动） | [docs/agents/publish.md](docs/agents/publish.md) |
76-
| Change Log | 发版说明 / 历史版本说明 | [docs/agents/changelog-write.md](docs/agents/changelog-write.md) |
77-
| 工具链调整 | lint 规则 / 构建配置 / 依赖升级 | [docs/agents/lint-toolchain.md](docs/agents/lint-toolchain.md) |
78-
| Command Pack | 扩展包 / 白名单 / plugin 管理命令 | [docs/agents/command-pack.md](docs/agents/command-pack.md) |
79-
| dsh 插件 | 改 `packages/bailian-kb-dsh`、dsh 依赖、插件发布 | [docs/agents/dsh-plugin.md](docs/agents/dsh-plugin.md) |
60+
| 场景 | 何时进入 | 详见 |
61+
| ----------------- | ------------------------------------------------------ | ---------------------------------------------------------------------------- |
62+
| 命令增删改 | 增加 / 删除 / 重命名 `bl xxx` 或入口命令路径 | [docs/agents/command-add-remove.md](docs/agents/command-add-remove.md) |
63+
| E2E 测试维护 | 新增/改命令或 e2e 用例、补 help/缺参/dry-run | [docs/agents/cli-e2e-tests.md](docs/agents/cli-e2e-tests.md) |
64+
| 测试隔离 | 单测/E2E 受本地 HOME、语言、系统目录干扰；统一隔离基建 | [docs/agents/test-isolation.md](docs/agents/test-isolation.md) |
65+
| 批量压测 | 改/跑多能力并发压测、`test:stress`、fixtures | [docs/agents/stress-batch-tests.md](docs/agents/stress-batch-tests.md) |
66+
| 选项变更 | 给已有命令加 `--flag` 或改默认值 | [docs/agents/command-flag-change.md](docs/agents/command-flag-change.md) |
67+
| 模型上下架 | 增加新模型 / 改默认模型 / 废弃旧模型 | [docs/agents/model-add-remove.md](docs/agents/model-add-remove.md) |
68+
| Skill 文案 / 路由 | 改 SKILL 路由、安装约定、hand-off、hub/领域边界 | [docs/agents/skill-change.md](docs/agents/skill-change.md) |
69+
| 错误文案变更 | 改 `BailianError` 的 message 或 hint | [docs/agents/error-hint-change.md](docs/agents/error-hint-change.md) |
70+
| URL / 渠道变更 | 控制台域名 / 文档站 / 追踪参数 | [docs/agents/url-change.md](docs/agents/url-change.md) |
71+
| 埋点变更 | 改 AEM 命令事件、后端渠道 header、User-Agent | [docs/agents/telemetry-change.md](docs/agents/telemetry-change.md) |
72+
| 鉴权扩展 | 加 OAuth / SSO / 换 token 来源 | [docs/agents/auth-change.md](docs/agents/auth-change.md) |
73+
| 配置项扩展 | 新 env var 或 `~/.bailian/config.json` 字段 | [docs/agents/config-add.md](docs/agents/config-add.md) |
74+
| Profile / 激活 | 改命名 Profile、预设或 `active_config` | [docs/agents/config-profile-change.md](docs/agents/config-profile-change.md) |
75+
| 安装文档 | 改安装、鉴权、验证流程或线上 install 页面 | [docs/agents/install-doc-change.md](docs/agents/install-doc-change.md) |
76+
| 发布 | channel / stable 发布到 npm（CI 驱动） | [docs/agents/publish.md](docs/agents/publish.md) |
77+
| Change Log | 发版说明 / 历史版本说明 | [docs/agents/changelog-write.md](docs/agents/changelog-write.md) |
78+
| 工具链调整 | lint 规则 / 构建配置 / 依赖升级 | [docs/agents/lint-toolchain.md](docs/agents/lint-toolchain.md) |
79+
| Command Pack | 扩展包 / 白名单 / plugin 管理命令 | [docs/agents/command-pack.md](docs/agents/command-pack.md) |
80+
| dsh 插件 | 改 `packages/bailian-kb-dsh`、dsh 依赖、插件发布 | [docs/agents/dsh-plugin.md](docs/agents/dsh-plugin.md) |
8081

8182
如果当前任务无法对应任何场景,先按经验完成,然后**回来评估这是不是一类新场景** —— 是就新增 `docs/agents/<scenario>.md`,把清单沉淀下来。
8283

‎docs/agents/cli-e2e-tests.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ describe.skipIf(<ready>)("e2e: <topic>（DashScope …）", () => {
7171

7272
| 场景 | 条件 |
7373
| ----------------------- | ---------------------------------------------------------------------------------------------------------- |
74-
| 文本/搜索/记忆/配置 | `isDashScopeE2EReady()` |
74+
| 文本/搜索/记忆/配置 | `isDashScopeE2EReady()`（`.env` 中的 `DASHSCOPE_API_KEY`） |
7575
| 图像/语音 | `isBailianE2EMediaEnabled() && isDashScopeE2EReady()` |
7676
| 视频 | `isBailianE2EVideoEnabled() && isDashScopeE2EReady()` |
7777
| OpenAPI AK/SK | `isOpenApiE2EReady()`（`.env` 中必须同时提供完整 AK/SK） |
@@ -109,6 +109,7 @@ describe.skipIf(<ready>)("e2e: <topic>（DashScope …）", () => {
109109

110110
- **禁止破坏真实用户配置**：`auth logout` 和 `config set` 默认只用 `--dry-run`；只有验证持久化契约时，才允许通过
111111
`BAILIAN_CONFIG_DIR` 指向每个用例独占的临时目录实际落盘，并必须在 `finally` 中清理；禁止写入或复用真实 `~/.bailian`
112+
- **默认测试隔离**：`runNodeMain` / `runCommandE2e` / `runCli` / `runKscli` 默认走 `e2e/isolation`（HOME 与 `XDG_CONFIG_HOME` 隔离、强制 `language: en-US`、关闭 HOME 外的系统/项目目录探测、清空 Agent 覆盖 env）。DashScope / OpenAPI 凭证继续走环境变量；Console live 用例用 `runConsoleCommandE2e` / `runConsoleCli` 带上 `access_token`。进程内单测用 `withIsolatedHome` / `enterIsolatedHome`，**禁止**各文件自写 `inFakeHome`。完整约定见 [test-isolation.md](test-isolation.md)。
112113
- **不加 dry-run**：`dryRun` 在 `resolveFileUrl` / `resolveCredential` / 上传**之后**的命令（如 `image edit`、`speech recognize` 带 `--url`）
113114
- **`--list-voices` 等旁路**：先于 `--text` 校验的 flag，缺参用例勿带该 flag
114115
- 新增 required option → 至少一条缺参用例；改 dry-run 输出 → 更新对应断言

‎docs/agents/test-isolation.md‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# 测试隔离
2+
3+
## 触发条件
4+
5+
- 新增/修改会读 `~/.bailian`、`$HOME`、Agent 配置目录或 help 文案语言的单测 / E2E
6+
- 修「本机绿、CI 红」或反过来、且怀疑是本地配置/语言/系统目录泄漏
7+
- 调整 `packages/e2e` runner、skills/agent 探测、`BAILIAN_CONFIG_DIR` 相关测试基建
8+
9+
## 概念图
10+
11+
```
12+
宿主机器
13+
~/.bailian (language/output/凭证)
14+
/etc/codex、/Applications/…（系统级 Agent 探测）
15+
$CODEX_HOME 等覆盖变量
16+
│
17+
▼
18+
e2e/isolation（统一入口）
19+
├─ 进程内：withIsolatedHome / enterIsolatedHome
20+
└─ 子进程：runNodeMain → buildIsolatedCliEnv
21+
│
22+
├─ 调用方 HOME，否则临时 HOME；XDG 始终改写到该 HOME
23+
├─ 未传 BAILIAN_CONFIG_DIR：空 HOME 写 $HOME/.bailian；已有 config 则用自有临时目录（不覆盖）
24+
├─ overrides 的 undefined / 隔离关键键空串忽略，不能拆掉隔离
25+
├─ DashScope / OpenAPI 凭证继续走环境变量
26+
├─ Console access_token 仅 seedCredentials: true 时拷贝
27+
├─ 清空 AGENT_ENV_KEYS（含 OPENCLAW / ZCODE / LOCALAPPDATA）
28+
├─ BL_SKIP_HOST_AGENTS=1
29+
└─ runNodeMain 结束时删除自己创建的临时目录
30+
31+
Session 父目录（仅扫尾归堆，不承载隔离语义）：
32+
globalSetup createIsolationSessionRoot → provide → setupFiles 写 BL_ISO_ROOT
33+
→ 临时子目录挂其下；无 BL_ISO_ROOT 时落 os.tmpdir()
34+
→ teardown 只删闭包里的 root（无进程 ownership / 信号清理）
35+
```
36+
37+
## 必查清单
38+
39+
### A. 选用哪一层隔离
40+
41+
- [ ] **core 进程内单测**（skills / advisor）：`import { withIsolatedHome } from "./helpers/isolation.ts"`（core 不依赖 e2e，避免环）
42+
- [ ] **commands / runtime 进程内单测**（agent writer 等）：`import { withIsolatedHome, enterIsolatedHome } from "e2e/isolation"`
43+
- [ ] 禁止本地再写 `inFakeHome` / 直接 `process.env.HOME =`
44+
- [ ] **CLI 子进程 E2E**：走 `runCommandE2e` / `runCli` / `runKscli` → 底层 `runNodeMain` **默认已隔离**；用例只需在需要时传自己的 `BAILIAN_CONFIG_DIR` / `HOME`
45+
- [ ] 只传 `HOME` 且该 HOME 下**已有** `config.json` 时，基建**不会覆盖**，改用自有临时 `BAILIAN_CONFIG_DIR`；空 HOME 才写 `$HOME/.bailian`
46+
- [ ] 隔离关键键的 `undefined` / 空串会被忽略，不能拆掉隔离；其它键传 `undefined` 会删除继承凭证（如 `DASHSCOPE_API_KEY: undefined`）
47+
- [ ] **必须继承真实用户配置**的极少数用例：`runNodeMain(..., { isolate: false })`，并在注释里写明原因
48+
49+
### B. 系统目录与 Agent 探测
50+
51+
- [ ] 依赖 `detectInstalledAgents` 的断言必须在隔离环境下跑（`BL_SKIP_HOST_AGENTS=1`）
52+
- [ ] 新增落在 HOME / `XDG_CONFIG_HOME` 之外的绝对 detectDirs（含 `/etc`、`/Applications` 和 cwd 项目目录）时，确认 `isHostSystemDetectDir` 仍判为主机路径；并补一条「隔离开关打开时不被探测」的测试。不要把判断收窄成只认 `/etc` 和 `/Applications`
53+
- [ ] 新增官方路径覆盖变量时，同步加入 `AGENT_ENV_KEYS`（含 MCP 的 `OPENCLAW_CONFIG_PATH` / `ZCODE_HOME`）
54+
- [ ] inventory / mcp-agent-config 等「显式收 home、但仍读 process.env」的单测，用 `enterIsolatedHome` 或清空 `AGENT_ENV_KEYS`
55+
56+
### C. 语言 / 输出格式断言
57+
58+
- [ ] help / 文案断言默认按 **en-US** 写（隔离 config 已强制）；若测中文，显式构造带 `language: "zh-CN"` 的临时 config
59+
- [ ] 解析 stdout 的 dry-run 用例：要么显式 `--output json`，要么依赖命令默认 JSON，**不要**依赖用户 `config.output`
60+
- [ ] 禁止用本机 `~/.bailian` 的 language/output 作为隐式测试输入
61+
62+
### D. 文档
63+
64+
- [ ] 改隔离约定时同步更新本文与 [cli-e2e-tests.md](cli-e2e-tests.md) 的安全/隔离段落
65+
- [ ] 复现本机污染：保留 `/etc/codex` 或 `language: zh-CN` 配置，确认相关用例仍绿
66+
67+
## 完成后自查
68+
69+
```sh
70+
pnpm --filter e2e exec vp test tests/isolation.test.ts
71+
pnpm --filter bailian-cli-core exec vp test tests/skills-agents.test.ts tests/advisor-sync.test.ts
72+
pnpm --filter bailian-cli-commands exec vp test tests/e2e/managed-agent.e2e.test.ts tests/e2e/permission.e2e.test.ts
73+
```
74+
75+
## 常见漏点
76+
77+
- 只改 `HOME`、忘了清 `CODEX_HOME` / `CLAUDE_CONFIG_DIR` / `OPENCLAW_CONFIG_PATH` / `ZCODE_HOME` / `XDG_CONFIG_HOME` / `LOCALAPPDATA` → 探测或写入仍指向开发机
78+
- 只清 env、忘了系统路径 `/etc/codex` 或 cwd 下的项目目录 → 仍探测到本机 Agent
79+
- 对调用方已有 `HOME/.bailian/config.json` 无条件 `writeFileSync` → 冲掉用户配置；应改用基建自有临时 config 目录
80+
- 隔离关键键 `HOME: undefined` → 须忽略，否则 spawn 丢掉键后回退真实 home
81+
- 非关键键 `DASHSCOPE_API_KEY: undefined` → 须删除该键；若 sanitize 一律丢弃 undefined，则「无需凭证」用例会带着父进程 Key 假绿
82+
- E2E 子进程继承 `process.env` + 真实 `~/.bailian` → help 变成中文、output 偏好干扰 JSON 解析
83+
- 各包各自写 `inFakeHome`，清单不一致 → 用 `e2e/isolation` 统一
84+
- live console e2e 需要 `access_token`：调用 `runConsoleCommandE2e` / `runConsoleCli`，或给 `runNodeMain` 传 `seedCredentials: true`。默认不拷贝 `api_key` 和 AK/SK；不要为了带上 token 使用 `isolate: false`

‎packages/cli/tests/e2e/helpers.ts‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,19 @@ const mainTs = join(cliPackageRoot, "src", "main.ts");
4747
export async function runCli(
4848
args: string[],
4949
envOverrides: NodeJS.ProcessEnv = {},
50+
options: { seedCredentials?: boolean } = {},
5051
): Promise<RunCliResult> {
51-
return runNodeMain(mainTs, args, { cwd: cliPackageRoot, env: envOverrides });
52+
return runNodeMain(mainTs, args, {
53+
cwd: cliPackageRoot,
54+
env: envOverrides,
55+
seedCredentials: options.seedCredentials,
56+
});
57+
}
58+
59+
/** Console live 用例：隔离环境里带上本机 access_token。 */
60+
export async function runConsoleCli(
61+
args: string[],
62+
envOverrides: NodeJS.ProcessEnv = {},
63+
): Promise<RunCliResult> {
64+
return runCli(args, envOverrides, { seedCredentials: true });
5265
}

‎packages/cli/tests/e2e/usage-summary.e2e.test.ts‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
import { describe, expect, test } from "vite-plus/test";
2-
import { isConsoleE2EReady, isConsoleAuthFailure, parseStdoutJson, runCli } from "./helpers.ts";
2+
import {
3+
isConsoleE2EReady,
4+
isConsoleAuthFailure,
5+
parseStdoutJson,
6+
runCli,
7+
runConsoleCli,
8+
} from "./helpers.ts";
39

410
describe("e2e: usage summary", () => {
511
test("usage summary --help 正常退出", async () => {
@@ -18,7 +24,7 @@ describe("e2e: usage summary", () => {
1824

1925
describe.skipIf(!isConsoleE2EReady())("e2e: usage summary（Console）", () => {
2026
test("usage summary --dry-run 输出 free-tier 计划请求", async () => {
21-
const { stdout, stderr, exitCode } = await runCli([
27+
const { stdout, stderr, exitCode } = await runConsoleCli([
2228
"usage",
2329
"summary",
2430
"--dry-run",
@@ -31,7 +37,7 @@ describe.skipIf(!isConsoleE2EReady())("e2e: usage summary（Console）", () => {
3137
});
3238

3339
test("usage summary --dry-run --workspace-id 附带用量概览计划请求", async () => {
34-
const { stdout, stderr, exitCode } = await runCli([
40+
const { stdout, stderr, exitCode } = await runConsoleCli([
3541
"usage",
3642
"summary",
3743
"--dry-run",
@@ -49,13 +55,13 @@ describe.skipIf(!isConsoleE2EReady())("e2e: usage summary（Console）", () => {
4955
});
5056

5157
test("usage summary 文本输出正常返回", async () => {
52-
const result = await runCli(["usage", "summary", "--output", "text"]);
58+
const result = await runConsoleCli(["usage", "summary", "--output", "text"]);
5359
if (isConsoleAuthFailure(result)) return;
5460
expect(result.exitCode, result.stderr).toBe(0);
5561
});
5662

5763
test("usage summary JSON 输出包含 freeTier 字段", async () => {
58-
const result = await runCli(["usage", "summary", "--output", "json"]);
64+
const result = await runConsoleCli(["usage", "summary", "--output", "json"]);
5965
if (isConsoleAuthFailure(result)) return;
6066
expect(result.exitCode, result.stderr).toBe(0);
6167
const data = parseStdoutJson<{ freeTier?: unknown; period?: unknown }>(result.stdout);

‎packages/cli/vite.config.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import { defineConfig } from "vite-plus";
33
export default defineConfig({
44
test: {
55
globalSetup: "../e2e/src/global-setup.ts",
6+
setupFiles: ["../e2e/src/iso-root-setup.ts"],
67
testTimeout: 60_000,
78
hookTimeout: 60_000,
89
},

‎packages/commands/tests/config-agent-writers.test.ts‎

Lines changed: 8 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,7 @@
1-
import {
2-
mkdtempSync,
3-
rmSync,
4-
readFileSync,
5-
writeFileSync,
6-
mkdirSync,
7-
readdirSync,
8-
existsSync,
9-
} from "fs";
10-
import { tmpdir, homedir } from "os";
1+
import { readFileSync, writeFileSync, mkdirSync, readdirSync, existsSync } from "fs";
2+
import { homedir } from "os";
113
import { join } from "path";
4+
import { enterIsolatedHome, type IsolatedHomeHandle } from "e2e/isolation";
125
import { afterEach, beforeEach, describe, expect, test } from "vite-plus/test";
136
import claudeCode from "../src/commands/config/agent/writers/claude-code.ts";
147
import qwenCode from "../src/commands/config/agent/writers/qwen-code.ts";
@@ -28,20 +21,18 @@ const OAI_URL = "https://dashscope.aliyuncs.com/compatible-mode/v1";
2821
const ANTHROPIC_URL = "https://dashscope.aliyuncs.com/apps/anthropic";
2922

3023
let home = "";
31-
let prevHome: string | undefined;
24+
let isolation: IsolatedHomeHandle | undefined;
3225

3326
beforeEach(() => {
34-
home = mkdtempSync(join(tmpdir(), "bl-agent-writer-"));
35-
prevHome = process.env.HOME;
36-
process.env.HOME = home;
27+
isolation = enterIsolatedHome();
28+
home = isolation.home;
3729
// homedir() 在 POSIX 读 $HOME；断言隔离生效。
3830
expect(homedir()).toBe(home);
3931
});
4032

4133
afterEach(() => {
42-
if (prevHome === undefined) delete process.env.HOME;
43-
else process.env.HOME = prevHome;
44-
rmSync(home, { recursive: true, force: true });
34+
isolation?.restore();
35+
isolation = undefined;
4536
});
4637

4738
function readJsonAt(...segments: string[]): Record<string, unknown> {

0 commit comments

Comments
 (0)