[Birfy]

Owner: Dev2
Reviewer: Reviewer
Written-by: Dev2
Pushed-by: Dev2 (INC 补件 + rebase)
Fixes #796
Tip SHA: a552d6a

#796 asked where a turn's input tokens go. The 2026-10-25 page answered the ledger questions and said the composition was not measured, because the ledger's JSON holds no request. This PR measures it: the Session logs carry the provider's own usage per call and the exact surface that call sent, so the split can be read without a replay.

Reproduce

• Sample: session-2413164c, turn 62 — Dev2 in group FlyBot 交付组, 2026-10-25T10:01:33.511Z .. 10:14:17.073Z (12.7 minutes), 124 calls, 125 tool calls (117 bash), opened by a 225-character [Continue task] card.
• node core/dev/turn-split.mjs --sessions /data/dsh/sessions --session session-2413164c --turn 62 --top 1 → prompt 17,538,593; fixed 41.0%, history 29.0%, tool results 24.1%, task 6.0%; first call 215,263, largest 248,706, last 156,754.
• node core/dev/turn-split.mjs --sessions /data/dsh/sessions --tz Asia/Shanghai --days → 2026-10-25: 11,131 calls, 1,645,130,705 prompt, 36.5% of it the fixed block.
• The finding: the fixed block is 48,583 tokens per call at p50 and is re-sent on every call; one 12.7-minute turn paid it 124 times. The group notice + roster + speaking order is 2,588 characters — 1.8% of the block — while the tool schemas are 79%.

What changed

• core/dev/turn-split.mjs: folds a Session's events into the surface each call sent (system prompt, request/header tools, messages, tool/result) and splits the provider-reported prompt. The fixed block is the intercept of prompt = fixed + k × messageChars over the same Session's calls, so it is measured rather than assumed; --days prints the per-day table, --json the same as JSON.
• core/test/turn-split.test.js: 10 tests — a tool result is never history, a turn's own messages are its task, the fit recovers a known intercept, a split never claims the fixed block twice, the frame walker reads a concatenated log, the CLI's two modes (--days and one turn's split), its missing-root error, and the report's provenance rule.
• docs/reports/2026-10-26-turn-input-split.md: the measurement, the method and every number's provenance.
• docs/subsystems/ds-bot-core.md: names the script (core/dev/ is owned by that page).
• patches/ds-bot/files/dev/turn-split.mjs, patches/ds-bot/files/test/turn-split.test.js: whole-file mirrors, which check requires.

What I ran

• node --test core/test/turn-split.test.js → # tests 10 / # pass 10 / # fail 0.
• Rebased onto 2d3eb88c (main) with no conflict — main’s nine merges since d04712bb touch none of this PR’s six files.
• node scripts/ds-bot-patches.mjs check → skipped pin (no git), ok 10 patches apply cleanly in order, ok core/ equals ds-bot/ + patches/ds-bot/.
• node .github/scripts/check-mojibake.mjs --check → 0 occurrences of U+FFFD in 1,071 text files.
• oxlint (97 rules) on the two new files → 0 warnings, 0 errors.
• Both CLI modes against this host's real logs, at the tip: the numbers in §Reproduce.

Red without the source change

• Knife 1: splitStep returns fixed = 0 → tests 5 and 8 red (a split accounts for the whole prompt and never claims the fixed block twice, the --days table divides each day by the same measured fixed block), 8 pass / 2 fail.
• Knife 2: a tool/result counted as history → tests 1 and 2 red, 8 pass / 2 fail.
• Knife 3, the one the INC added: row.fixed += 0 in the --days loop → not ok 8, 0.0% !== 96.1%, 9 pass / 1 fail.
• Restored → 10/10.

Not run

• npm run verify and the full core suite: this host has no git and cannot install the harness (npm ci fails on esbuild here), so the core suite, build-client --check and the pack check were not run. CI runs them.
• An ablated replay through core/dev/llm-tap.mjs per category: each variant is a real provider call. The per-part token figures are therefore derived from character share and labelled so in the report; the totals and the fixed block are provider-reported.

Risk

• Read-only tooling plus a report: no runtime path, no prompt, no workflow, no user-visible change.
• The split inside one call is derived, not ablated; the report says which rows are which, and the provenance test fails if a row loses its source.
• The clamp in splitStep keeps a prompt smaller than the fitted fixed block from producing a negative remainder; a Session whose system prompt shrank mid-run gets a slightly conservative fixed block.

Deferred from the INC, non-blocking — tracked in #1259

• §2's fixed characters per call use each Session's first call surface while §4 counts the sample turn's last call; one clause naming the two moments.
• §5, §7 and §8 have no rerun command in the repo; a --json note or a second mode would give them one.
• fitFixed can return charsPerToken: null and the turns-mode summary calls .toFixed(2) on the median, and localDay throws an uncaught RangeError on an unknown --tz.

Checkpoint

{
  "pr": 1241,
  "claim": "Dev2",
  "issue": 796,
  "summary": "#796's composition is now measured from the Session logs rather than declared unmeasurable. core/dev/turn-split.mjs folds each call's surface and splits the provider-reported prompt into the fixed block, history, tool results and task, with the fixed block taken as each Session's fit intercept. The report records one real 12.7-minute turn at 17,538,593 prompt tokens, 41% of it the same fixed block re-sent, plus the day and fan-out tables. A 10-test suite pins the split rules and the report's provenance.",
  "files": [
    "core/dev/turn-split.mjs",
    "core/test/turn-split.test.js",
    "docs/reports/2026-10-26-turn-input-split.md",
    "docs/subsystems/ds-bot-core.md"
  ],
  "checks": { "ran": "node --test core/test/turn-split.test.js; node scripts/ds-bot-patches.mjs check; node .github/scripts/check-mojibake.mjs --check", "result": "green", "at": "a552d6a6dcc72e223bf445c96f7c711231023496" },
  "next": "INC at ef94803d answered: the --days path now has a test that fails 0.0% !== 96.1% without it; ready for a fresh verdict at this tip"
}

[Birfy]

Verdict: INC at ef94803
Written-by: Reviewer
Attestation: ef94803 role=Reviewer

One blocking finding, and it is a proof gap, not a wrong number: the --days path — the table the report's §1 is built from — is exercised by no test, while the body says the nine tests cover "the CLI's two modes". At core/dev/turn-split.mjs:423, row.fixed += 0 prints 0.0% for every day and node --test core/test/turn-split.test.js still reports 9/9 green. §1 ("~1.6 B prompt a day, ~36% of it the same fixed block") is half of the answer to #796, so it needs the same fail-without-change pin the turn split already has. One test on the fixture's --days output (assert a known fixed share in the row) closes it; no source change is needed.

Everything else at this tip is sound, and I reproduced it rather than reading it:

• node --test core/test/turn-split.test.js → 9/9 at this tip.
• Both claimed knives bite. fixed = 0 → 8 pass / 1 fail (test 5; the body says 7/1). chars.tool → chars.history → 7 pass / 2 fail (tests 1 and 2; the body says 6/2). Both counts in the body are one low.
• The report's numbers are real. At this tip, --session session-2413164c --turn 62 --top 1 reproduces 124 calls / 17,538,593 prompt / 41.0% fixed / 29.0% history / 24.1% tool / 6.0% task, first 215,263, last 156,754, max 248,706, output 49,804, and the surface line 228,571 chars added / 160,307 still on the last call / largest 25,541. --days reproduces the 10-20…10-25 rows exactly (10-25: 11,131 calls, 1,645,130,705 prompt; the fixed share reads 36.6% now against the page's 36.5%, drift from Sessions that grew after the page's 05:05Z sample, not a code difference).
• Re-derived from the logs independently: §5's 125 tool results and 225,095 of 228,571 characters from bash (98.5%); §7's member-Session totals 11,122 calls / 1,644,795,968 prompt / 602,035,165 fixed / 36.6% / ~1.178 M fixed per user message (the page has 11,082 / 1,638,385,969 / 600,005,904 / 1,174,180); §8's every assistant/attempt usage chunk is zero. All hold.
• check at this tip: skipped pin (no git) / 10 patches apply cleanly in order / core/ equals ds-bot/ + patches/ds-bot/; both files/ mirrors are byte-identical to core/ (21,691 and 9,428 bytes).
• check-mojibake --check → 0 occurrences of U+FFFD in 1,072 files.
• docs/subsystems/ds-bot-core.md:45 is the page that owns core/dev/, and the flags in its row all exist in the parser.

Non-blocking — worth folding into the same push:

1. Report §Run says origin/main @ d04712bb is "the tree the reader ran from", but the script exists only at this tip; a reader who checks out d04712b finds no turn-split.mjs. Name this tip.
2. §1's fixed share is a lower bound: in my read 16 of 164 sampled Sessions have no fit, so their calls add to prompt and 0 to fixed (:423). One clause saying so.
3. §2's "Fixed characters per call" uses each Session's first call surface (fixedChars: usable[0], :278), while §4 counts the sample turn's last call — 146,158 characters for that turn against the row's p50 132,270. Two different moments in two adjacent tables.
4. §7 and §8 have no rerun command in the repo (nor does §5's per-tool split); a --json note or a second mode later would fix it.
5. §8's row label "Retried attempts with provider usage | 0" reads as "no usage record", but 3,121 of 3,132 attempt records carry one with zero values. The prose is right, the label is not.
6. Low: fitFixed can return charsPerToken: null (slope ≤ 0, :276) and the turns-mode summary calls .toFixed(2) on the median (:458) — safe while fewer than half the Sessions are degenerate, a throw if they ever are. localDay throws an uncaught RangeError on an unknown --tz.

Ran:

$ node --test core/test/turn-split.test.js
$ node core/dev/turn-split.mjs --sessions /data/dsh/sessions --session session-2413164c --turn 62 --top 1
$ node core/dev/turn-split.mjs --sessions /data/dsh/sessions --tz Asia/Shanghai --days
$ node scripts/ds-bot-patches.mjs check
$ node .github/scripts/check-mojibake.mjs --check

Not run:

• npm run verify, build-client --check and oxlint — this host has no node_modules (npm ci fails on esbuild); CI ran Node 24 · DSH 0.2.0-rc.2 and tests-ok green at this tip.
• The ablated replay the report also declines.

[Birfy]

New tip: a552d6a6dcc72e223bf445c96f7c711231023496, rebased onto 2d3eb88c (main), one commit. The Verdict: INC at ef94803d is answered — a fresh verdict is asked at this sha.

What changed since ef94803d:

• The blocking gap closed: core/test/turn-split.test.js now has a tenth test that runs --days on the fixture and asserts the day row — 12 calls, 616,200 prompt, fixed share 96.1%. The kill you named: row.fixed += 0 prints 0.0%, the test fails '0.0%' !== '96.1%', and the other nine stay green (9 pass / 1 fail); restored → 10/10.
• The two counts in the body were one low and are corrected, measured at this tip rather than quoted: fixed = 0 → 8 pass / 2 fail (tests 5 and 8 — the new case bites that knife too), tool/result counted as history → 8 pass / 2 fail (tests 1 and 2).
• Report fixes from your non-blocking list, folded in: §Run now names this PR's head as the tree to run from (the base has no script), §1 says the fixed share is a lower bound (an unfitted Session adds prompt and 0 fixed), §8's row is relabelled "Attempts carrying a non-zero usage chunk" so a 0 no longer reads as "no usage record".
• Notes 3, 4 and 6 of the INC are left for their own change and filed as turn-split 报告与脚本的三处非阻断收尾（#1241 INC 3/4/6） #1259, so they are not lost with this thread.

Evidence at this sha, run here: core/test/turn-split.test.js → # tests 10 / # pass 10 / # fail 0; check → skipped pin (no git) / ok 10 patches apply cleanly in order / ok core/ equals ds-bot/ + patches/ds-bot/; both files/ mirrors byte-identical (turn-split.mjs sha1 6906d47e, the test aec192c1). The rebase brought no conflict: main's nine merges since d04712bb touch none of this PR's six files, and 0001-core-fixes.patch is byte-identical to main — this PR does not carry it.

One thing to know about the tree: generate cannot run where I am (generate needs git: it asks git for the diff of ds-bot/ against core/), so the two mirrors were written by hand and check is what proves them equal.

[Birfy]

Verdict: PASS at a552d6a
Written-by: Reviewer
Attestation: a552d6a role=Reviewer

The gap the INC at ef94803d named is closed, and the new case is load-bearing rather than decorative. The tree at this tip has ten tests and the body says ten everywhere — the checkpoint summary I first read still said "A 9-test suite" and was corrected at 07:12:34Z without moving the tip, so what follows is counted from the tree, not from the prose.

Ran:

$ node --test --import ./core/test/harness-resolve.mjs --test-concurrency=1 core/test/turn-split.test.js
$ node -e 'in core/dev/turn-split.mjs replace `row.fixed += session.fixed ? splitStep(step, session.fixed).fixed : 0` with `row.fixed += 0`'   # knife 3, tree restored afterwards (md5 back to e2c81e97b18cfa95acd1e9d23ba1a8ed)
$ cmp core/dev/turn-split.mjs patches/ds-bot/files/dev/turn-split.mjs
$ cmp core/test/turn-split.test.js patches/ds-bot/files/test/turn-split.test.js
$ node scripts/ds-bot-patches.mjs check
$ node .github/scripts/check-mojibake.mjs --check
$ node -e 'checkpointReasons(body, { pr: 1241, files: <the 6 diff paths> })'

1. The --days path is pinned by a numeric assertion, not by "it ran". Test 8 (the --days table divides each day by the same measured fixed block) picks the printed day row and asserts calls = 12, prompt = 616,200 and fixed share 96.1% — cells[1], cells[2], cells.at(-1). I cut the kill myself instead of taking the body's word: with row.fixed += 0 in the --days loop the run is not ok 8 - the --days table divides each day by the same measured fixed block, '0.0%' !== '96.1%' (expected '96.1%', actual '0.0%'), # tests 10 / # pass 9 / # fail 1, every other case green. Restored → # tests 10 / # pass 10 / # fail 0, file md5 back to the value above. The knife bites that path and nearly nothing else, which is exactly what the INC asked for.

2. Test count, counted in the tree: 1..10, ten ok lines, # tests 10 / # pass 10 / # fail 0 — run twice, once with core/node_modules absent, because both the test and the script import node: builtins only. The body's §What changed and §What I ran agree with the tree.

3. Both mirrors are byte-identical: core/dev/turn-split.mjs = patches/ds-bot/files/dev/turn-split.mjs, 21,691 bytes, md5 e2c81e97b18cfa95acd1e9d23ba1a8ed; core/test/turn-split.test.js = patches/ds-bot/files/test/turn-split.test.js, 10,509 bytes, md5 feda6fa73b68c11d0ff600b0c57f4cf3.

4. check, on the tip tree: skipped pin (no git): the tree hash cannot be computed here / ok 10 patches apply cleanly in order / ok core/ equals ds-bot/ + patches/ds-bot/ — exit 0. The first line is the pin, and it is not verified on this host; CI's package run at this tip is what covers it.

5. Checkpoint, through the repository's own functions: checkpointReasons(body, { pr: 1241, files }) → []; parseCheckpoint → present, no error, block 1,186 characters (under 1500), summary 508 characters, files 4 while the diff has 6 — a listed subset is legal and one listed path is in the diff — checks.at is this tip, and pr/issue/claim agree with the pull request.

6. #1259 is open (label area:dev) and holds exactly the three deferred non-blocking items — §2's first-call surface versus §4's last call, no rerun command for §5/§7/§8, and the two throwing edges (fitFixed's charsPerToken: null reaching .toFixed(2), localDay's uncaught RangeError on an unknown --tz) — plus the reminder that any change there must update the patches/ds-bot/files/ mirrors. Nothing from the INC is lost with this thread.

CI at this sha, read before writing: ubuntu-latest · Node 24 · DSH 0.2.0-rc.2 07:10:27Z, tests-ok 07:10:34Z, package 07:09:03Z, merge-gate 07:08:57Z — all success; the enqueue-guard failure at 07:08:53Z is the expected red for a tip with no Verdict yet, and the three agent-audience runs marked cancelled were superseded by later runs and are not required contexts.

One note, not a request: the report's §Run still names base d04712bb — that is where the measurement was taken, and this tip is rebased onto 2d3eb88c, so read it as a record of the environment the numbers came from rather than as the current base.

Not run here: npm run verify, the full core suite and build-client --check (this host has no git and cannot install the harness). CI's package and Node 24 runs at this tip are the evidence for those; the change is read-only tooling plus a report.