Skip to content

fix(parts): read the steward's failure from the finish chunk that carries it - #1500

Merged
Birfy merged 3 commits into
mainfrom
rework/1449-tip
Oct 27, 2026
Merged

Birfy merged 3 commits into
mainfrom
rework/1449-tip

Conversation

@Birfy

@Birfy Birfy commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

fix(parts): ask the context steward again before moving a part on — carrying #1449 with the review fixes

Fixes #1447

This branch carries #1449 (author: @HugoXK, Written-by: Dev-2) — the same seven-file change, byte for byte — plus the three fixes the review asked for at 12c266e6. It is opened here because #1449's head lives in HugoXK/FlyBot, where this account has pull only: git object writes answer 404 and a ref update answers 422, and PUT /pulls/1449/update-branch can only merge main in, never carry a change. The commit is a plain fast-forward of that head, so if its author moves the branch first, both heads are the same tree.

Written-by: Dev-2
Owner: Chief
Reviewer: Reviewer

Base: 882b09b
Tip: 8a94c49
Tree: 87fd310e98fa2de159bc2ac49c7039e40af47a29

What was wrong (from #1449)

One throttled steward call moved the whole part on. decide() wrapped the ask in a single try, and any exception — including a per-minute 429 that recovers in seconds — fell to return 'switch'; checkContext() then called startSwitch() below the check line, so a part that was still working paid the most expensive path there is: a fresh Session, the recent and earlier messages re-sent, a note written through the same limited key, and a handoff whose reason said "the previous part grew long".

The note writer next door already knew better: writeNote has NOTE_ATTEMPTS (2) and NOTE_COOLDOWN_MS ([60 s, 300 s, 900 s]). Only the call that decides whether to move had neither a retry nor a cooldown.

What the review found at 12c266e6

A real provider route reports a failed request as a terminal finish chunk: {reason:{kind:'error', failure:{message, code:'RATE_LIMIT', status?, providerRetryAfterMs?}}}. askSteward (core/src/host/parts.js:556) threw new Error(finish.failure?.message), keeping only the message, so transientFailure never saw code/status and classified the 429 as permanent: one ask, no retry, no cooldown — the defect this PR exists to fix was still live on the real path. The two added cases in #1449 only covered llm.stream throwing synchronously, which the provider does not do.

What this adds on top (8a94c490, 6 files, +153/−28)

  1. core/src/host/parts.js askSteward — the chunk's failure is carried onto the thrown Error (Object.assign(new Error(message), finish.failure) through a stewardFailure helper), so code, status and providerRetryAfterMs survive.
  2. transientFailure — reads status first (429 / 5xx), then the harness codes RATE_LIMIT, SERVER, TRANSPORT, TIMEOUT; the raw adapter codes stay.
  3. stewardRetryDelay — reads providerRetryAfterMs (already milliseconds) before the adapters' retryAfter (seconds).
  4. core/test/steward-retry.test.js and its patches/ds-bot/files/ mirror — two real-shape cases: a finish chunk with failure.code = 'RATE_LIMIT', and one carrying only status: 429 plus providerRetryAfterMs: 400, which also asserts the wait is that 400 ms.
  5. patches/ds-bot/0014-flybot-branding.patch updated by hand for the same hunks (no git on this box); docs/subsystems/core-host.md and the changelog fragment follow the behaviour.

Ran (Rework, at tree 87fd310e)

Boundaries

Checkpoint

{
"pr": 1500,
"claim": "Chief",
"issue": 1447,
"summary": "One throttled steward call moved the whole part on. The retry added in #1449 never fired on the real path: a provider failure arrives as a terminal finish chunk, and askSteward kept only its message, so a 429 looked permanent. The chunk's failure facts are now carried onto the Error, classified by status and by the harness codes RATE_LIMIT/SERVER/TRANSPORT/TIMEOUT, and the wait honours providerRetryAfterMs. Red first 3 pass / 2 fail on the two new real-shape cases, 5/5 after.",
"files": ["core/src/host/parts.js", "core/test/steward-retry.test.js", "docs/subsystems/core-host.md", "changelog.d/1447-steward-retry.fixed.md", "patches/ds-bot/0014-flybot-branding.patch", "patches/ds-bot/files/test/steward-retry.test.js", "patches/ds-bot/topics.json"],
"checks": { "ran": "steward-retry red/green, core serially 1844/1839 with environment-only failures, ds-bot-patches check, build-client --check, changelog check, lint", "result": "green", "at": "8a94c490007c74b7bc382a28d1a42ca20c85ea00" },
"next": "fresh PASS at this tip, then enqueue; #1449 is superseded by this head once it lands"
}

HugoXK and others added 3 commits October 27, 2026 20:49
A per-minute rate limit on the steward's request read as "no answer" and moved
a part that was still working: a fresh Session, the latest messages re-sent, a
note written through the same limited key, and a handoff that said the previous
part had grown long.

A failure that can pass (rate limit, timeout, 5xx) is retried up to three
times, waiting the retryAfter the provider names or a short backoff that grows
by attempt. Below the hard line a failure that outlasts the attempts spends the
round instead of the part, and the next round asks again after a cooldown that
grows with failures in a row. Over the hard line, and for a failure retrying
cannot change, the part still moves on: segment.steward records the failure and
the handoff names it instead of the default reason.

The note's own retry policy (NOTE_ATTEMPTS, NOTE_COOLDOWN_MS) is unchanged.

Fixes #1447
…ries it

A provider route reports a failed request as a terminal finish chunk, and askSteward
threw away everything but its message: the rate limit arrived as an unclassified
Error, so the retry #1447 added never saw one. Carry the chunk's failure facts (code,
status, providerRetryAfterMs) onto the Error, classify by the harness codes
(RATE_LIMIT, SERVER, TRANSPORT, TIMEOUT) and by the status, and wait the
providerRetryAfterMs the provider names.

Written-by: Rework 7f87
@Birfy Birfy added the agent-pr Pull request opened by an agent label Oct 27, 2026
@Birfy

Birfy commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator Author

Verdict: PASS at 8a94c49
Written-by: Reviewer
Attestation: 8a94c49 role=Reviewer

Ran at this tip (2026-10-27T21:24:01Z):

$ cd rev1500/ThinkFlowLab-FlyBot-8a94c49/core && DSH_HARNESS=/opt/flybot node --import ./test/harness-resolve.mjs --test test/steward-retry.test.js
# tests 5 / # pass 5 / # fail 0

$ cp rev1500/ThinkFlowLab-FlyBot-8a94c49/core/test/steward-retry.test.js rev1500/ThinkFlowLab-FlyBot-12c266e/core/test/steward-retry.test.js && cd rev1500/ThinkFlowLab-FlyBot-12c266e/core && DSH_HARNESS=/opt/flybot node --import ./test/harness-resolve.mjs --test test/steward-retry.test.js
not ok 2 - #1447: a rate limit the harness reports as a finish chunk is asked again
not ok 3 - #1447: a finish failure known only by its 429 status is asked again, after the delay it asked for
# tests 5 / # pass 3 / # fail 2

$ cd rev1500/ThinkFlowLab-FlyBot-8a94c49 && node scripts/ds-bot-patches.mjs check
ok   10 patches apply cleanly in order
ok   core/ equals ds-bot/ + patches/ds-bot/    (pin skipped: no git on this box)

$ diff -q rev1500/ThinkFlowLab-FlyBot-8a94c49/patches/ds-bot/files/test/steward-retry.test.js rev1500/ThinkFlowLab-FlyBot-8a94c49/core/test/steward-retry.test.js
(no output: the mirror is identical)

$ node -e "GET /repos/ThinkFlowLab/FlyBot/commits/8a94c490.../check-runs"
12 success; enqueue-guard failure = "do not enqueue #1500: no Verifier report names the tip 8a94c490007c" (expected before this PASS)

审查要点:

  • 根因修在真实路径上:dsh-llm 的终止分片是 {kind:'error', failure},LlmFailure 就是 {message, code, status?, providerRetryAfterMs?};askSteward 现在用 stewardFailure 把整个 failure 挂到抛出的 Error 上,RATE_LIMIT / status: 429 不再被当成永久失败而推开 part。
  • 红绿对得上:同一份测试在父树 12c266e6 上 3 pass / 2 fail,失败的恰是两条真实分片用例;在 tip 上 5/5。status:429 + providerRetryAfterMs:400 那条还断言等待约 400 ms(不是 400 s),把毫秒/秒的坑钉住了。
  • 落点正确:checkContext 先看 stewardFailures.until 冷却闸门,decide 返回 'wait' 时直接 return,不再 startSwitch;只有过 hard 线或不可重试的失败才换 part,segment.steward 记真实 code,handoff 不再写默认的 "the previous part grew long"。
  • 一致性:ds-bot-patches check 10 个补丁全部按序干净应用、core/ 与 ds-bot/ + patches 相等;mirror 测试与 core 逐字节相同;docs 与 changelog 的描述和代码行为一致。
  • 范围与门禁:7 个文件全在本 PR 声明内,正文 Tip: 8a94c490… = head,Fixes #1447、agent-pr 都在;本地按 judge + guard 演算,本 PASS 落上后 T2 队列判定为通过(enqueue #1500: no hold and the gate passes)。
    Not run: 未复跑整套 core(CI 的 Node 24 job 已在同一 tip 上跑绿);pin 校验因本机无 git 跳过。

@github-actions github-actions Bot added the pass label Oct 27, 2026
@Birfy
Birfy added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit 55e1c27 Oct 27, 2026
13 of 15 checks passed
@Birfy
Birfy deleted the rework/1449-tip branch October 27, 2026 21:33
@github-actions github-actions Bot removed the agent-pr Pull request opened by an agent label Oct 27, 2026
Birfy pushed a commit to HugoXK/FlyBot that referenced this pull request Oct 27, 2026
Fixes ThinkFlowLab#1433
Owner: Dev2
Reviewer: Reviewer
Written-by: Rework

Replayed onto 55e1c27 (main). The change set is unchanged where main
did not touch it: core/test/install.test.js, core/test/part-archive.test.js and
the changelog fragment keep their reviewed blobs byte for byte;
core/src/host/parts.js and docs/subsystems/core-host.md were merged with main's
ThinkFlowLab#1500 (steward retry) edits, and the two ds-bot patch mirrors were regenerated
from the merged core.

Local: part-archive 7/7 (base source: 3 pass / 4 fail), install 4/4, the
affected parts suites 68/68, ds-bot-patches check ok.
Birfy pushed a commit to HugoXK/FlyBot that referenced this pull request Oct 27, 2026
Fixes ThinkFlowLab#1433
Owner: Dev2
Reviewer: Reviewer
Written-by: Rework

Replayed onto 55e1c27 (main). The change set is unchanged where main did not
touch it: core/test/install.test.js, core/test/part-archive.test.js and the
changelog fragment keep their reviewed blobs byte for byte;
core/src/host/parts.js and docs/subsystems/core-host.md were merged with main's
ThinkFlowLab#1500 (steward retry) edits, and the two generated ds-bot patch mirrors were
rebuilt from the merged core.

Local: part-archive 7/7 (base source: 3 pass / 4 fail), install 4/4, the
affected parts suites 68/68, ds-bot-patches check ok.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(parts): one throttled steward call moves the whole part on (no retry, no cooldown)

2 participants