Conversation
kernelHost.observeRunEvent() 把 run_completed / run_failed 先压成一个布尔量,
settleEvaluationRun() 便只能落 completed / failed 两种状态。而用户主动取消恰恰
是以 run_failed + error.code = 'RUN_CANCELLED' 落地的(RunManager 的取消分支,
stream-event-adapter 再把它归一成流事件的 cancelled),于是「取消」在这条可观测性
链路上被记成了「任务失败」。
后果在统计口径上可见:packages/shared/src/evaluation/aggregate.ts 的
isSkippedRun() 以 status === 'cancelled' 判定跳过,记成 failed 之后
isQualityRun() 反而为 true ⇒ 取消的运行被计入 evaluated、skipped 恒为 0,
verdictForRun() 还会把该 case 判成 'fail'。core 的 ExperimentExecutionCounts
对 skipped 的注释写得很清楚:「Cases never run: cancelled mid-flight or skipped
after an abort.」
同层的另一个生产者 experiment-service 早已按 RUN_CANCELLED 落 'cancelled'
(:911),本改动让两个生产者对 EvaluationRunStatus 的口径一致。
修法:observeRunEvent() 直接推导 EvaluationRunStatus;settleEvaluationRun() 的形参
由 completed: boolean 换成 status: EvaluationRunStatus;EvaluationRun 字面量本就是
status 简写无需改动,exportAgentTrace({ completed }) 改写为
completed: status === 'completed',取值与修复前等价,trace 侧行为不变。预算/护栏终止
(BUDGET_EXHAUSTED 等)仍记 failed,未被一并归入 cancelled。
Closes helsome#254
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题与修复
kernelHost.observeRunEvent()把run_completed/run_failed先压成一个布尔量,settleEvaluationRun()便只能落completed/failed两种状态:而用户主动取消恰恰是以
run_failed+error.code = 'RUN_CANCELLED'落地的(packages/shared/src/kernel/run-manager.ts:419-421,stream-event-adapter.ts:90-98再把它归一成流事件的cancelled),于是「取消」在这条链路上被记成了「任务失败」。后果在统计口径上可见 ——
packages/shared/src/evaluation/aggregate.ts:182-183:记成
failed之后isSkippedRun为 false ⇒isQualityRun为 true(运行已 started)⇒ 取消的运行被计入evaluated而skipped恒为 0;aggregate.ts:69-71的verdictForRun()还会把该 case 判成'fail'。core/src/evaluation.ts:624对skipped的注释正是预期口径:「Cases never run: cancelled mid-flight or skipped after an abort.」同层另一个生产者
experiment-service.ts:911早已按RUN_CANCELLED落'cancelled',本改动让两个生产者对EvaluationRunStatus的口径一致。修法(
kernelHost.ts,+20/-4):observeRunEvent()直接推导EvaluationRunStatus:run_completed→completed;run_failed+RUN_CANCELLED→cancelled;其余run_failed→failed;settleEvaluationRun()形参completed: boolean→status: EvaluationRunStatus;EvaluationRun字面量本就是status简写无需改动,exportAgentTrace({ completed })改为completed: status === 'completed',取值与修复前等价,trace 侧行为不变。预算 / 护栏终止(
BUDGET_EXHAUSTED、RETRY_STORM、LOOP_DETECTED)仍然是failed,未被一并归入cancelled—— 本 PR 只修「用户主动取消」这一条,并有一条锁定用例守住这条边界。Closes #254可复现测试报告
环境:Windows 11 + Git Bash,bun 1.3.14;上游 main
ba5dcdfd。新增两条用例(
apps/electron/src/main/kernelHost.test.ts):通过runs.subscribe注入run_started→run_failed{RUN_CANCELLED}并断言落到EvaluationStore.addRun的状态;另一条断言普通运行时错误(PROVIDER_ERROR)仍是failed。命令:
bun test apps/electron/src/main/kernelHost.test.ts --isolate修复前的失败输出(即缺陷证据):
其余门禁:
bun test apps/electron --isolate= 32 pass / 0 fail(5 个文件);bun run typecheck= core / i18n / shared / ui / electron 五个工作区全部 exit 0;packages/shared --isolate基线为 1111 pass / 3 fail(ExperimentService.runExperiment取不到gitSha、durable research recovery的 2s 轮询预算、langfuse backend依赖本机 Langfuse 连通性),三项均在本 PR 未触碰的packages/shared上、与改动无关。未运行:完整 e2e 套件(本机沙箱无法 spawn
cmd.exe,属已知环境限制,详见此前 issue #223)。文件级重叠披露
apps/electron/src/main/kernelHost.ts是一个高争用文件,当前有 7 个 open PR 触及它,逐一核对后确认与本 PR 改动块不相交(坐标以 mainba5dcdfd为准):EvaluationRun字面量的error:字段(≈1363-1370);本 PR 改:1335-1346(状态推导)与:1400(completed实参),相隔 ≥20 行,且 #64 未触碰任何状态推导逻辑6a9a288e)最接近的 #64(issue #19 的 redaction 分支)当前
mergeable_state=clean,其 base 为陈旧的6405b7b9。若 #64 先合入,本分支需按常规 rebase;两者语义独立(一个是错误文本脱敏,一个是运行状态分类),不会互相覆盖。UI 变化
无可见 UI 变化(只改主进程写入评估存储的状态字段;不改渲染层、不改 IPC 面)。