Research: measure and cut token spend per Bot turn (prompt overhead, history growth, tool output, fan-out)

Owner: Dev2

Why

Every FlyBot turn spends a large and unmeasured share of its input tokens on things that are not the task: the group notice, the member roster, the speaking order, the full conversation so far, and every tool result that has ever been pasted into it. Nobody has measured that split, so nobody can tell a real saving from a change that only feels cheaper. On this host the delivery group has 10 Bots; one user message in the group can start up to 10 model calls, each carrying its own copy of the same context.

This issue is a research task, not a refactor: measure first, then land only what the measurement justifies.

What to measure (per turn, on this host)

Fixed prompt overhead. The group notice, the member list, the speaking order, the tree block, and the injected memory files are re-sent on every turn of every member. Measure their size in tokens and multiply by turns per day.
Conversation growth. Tokens per turn as a function of turns already taken in that conversation, before and after a checkpoint/compaction.
Tool-result weight. The share of history that is raw tool output (GitHub JSON, test logs, file reads). Report the largest contributors and whether anything ever trims them.
Fan-out cost. Model calls per user message in a group round, and the tokens each member re-reads that the others also read.
Retry and duplicate work. Turns that were stopped, restarted, or repeated because a Bot re-read the same state — count them, because they are pure cost.

Measurement protocol

Take one ordinary working day and one group round as the sample. Record input and output tokens per turn from the provider side, not from an estimate in the prompt.
Produce a table: turn id, Bot, phase, input tokens, output tokens, and a category split for the input (fixed overhead / history / tool results / task).
State the method so a second person can reproduce the numbers on another day.

Candidate savings (each must be proven, not assumed)

Sending the group notice only when it changed, or compressing it to the rules that actually change behaviour.
Roster and speaking order trimmed to the names that matter for that turn.
Tool results summarised at the point they enter history, with the full text kept in a file.
Earlier turns condensed more aggressively, with the exact wording still reachable through search rather than carried inline.
Fewer members called per round when the work is sequential anyway.

Acceptance

A report committed to the repository with the measured split above, and the method behind it.
At least one landed change that cites a measured before/after in tokens per turn, on the same workload.
Nothing in the safety rules is traded away for the savings: the attribution rules, the verdict rules, and the per-role write limits stay, and the report says which of them cost what.

Out of scope

pricing negotiation, provider choice, and per-team budgets (that is #645); this issue is only about spending fewer tokens for the same work.

Split, and only the measurable half is ready. #796 stays the umbrella: which savings to take changes what every Bot is sent on every turn, so it needs the owner's word. The measurement does not, and it needs nothing from CI or the provider.

Split out: #799 — Token report: read the existing usage ledger into per-turn numbers and a reproducible prompt split (enhancement | area:automation | agent-ready, hanging under this epic).

Why it is not starting from zero: core/src/host/usage.js already writes a ledger with one row per billed attempt and keeps input, cacheRead, cacheWrite and output apart (tokensOf() :44-49; schema :142-152; file $DSH_HOME/usage.sqlite). core/dev/llm-tap.mjs already logs the provider's usage per call plus a prefix diff, and can dump request bodies for replay. Only the UI reads the ledger (core/src/client/usage-page.js); there is no per-turn distribution, no growth curve, no split, and no report.

One number from this host's ledger, read read-only from a copy and covering 2026-10-20T07:34Z → 2026-10-22T22:05Z (16,399 calls):

Prompt per call: p50 129,231, p90 210,614, p99 244,788.
Cache reads are 96% of all prompt tokens (2,039,296,640 of ~2,120,000,000).

That reframes the work: the biggest single block is not the group notice, it is the history that every turn re-reads, and any report that prints one "tokens" column will hide the difference between a cheap cache read and a fresh input token. #799 therefore requires the columns to stay apart and every number to carry its provenance.

Scope guard: #799 may not touch prompts, core/src/**, workflows, or start a third metrics framework — #791 defines the RSI baseline metrics and #357 is the Nightly smoothness dashboard. #645 stays out of scope, as you wrote.

feat(dev): read the usage ledger into a token report (#802)

Ask for list_children only when more than one child fits a part (Thin…

Keep a group's fixed rules in its standing section, not in every turn (…

Keep children's load out of a tree coordinator's system prompt (Think…

Ask for list_children only when more than one child fits a part (Thin…

Written-by: Dev

Token audit of a Bot turn (owner request), two fixes merged:

Group turns re-send and store the group notice and fixed room rules on every turn #894 → Keep a group's fixed rules in its standing section, not in every turn (#894) #897 (9fe5c26): a group's fixed rules (members / face roster, @Name calls, notice, admin / lead notes, face routing rule, reply rule) now live in the member Session's bot-room system-prompt section, which is rendered every step and survives a checkpoint, instead of in every stored turn. Turn text: 4 Bots, 713-char notice, 80 turns went 161,750 → 46,790 chars (−71%). A face room went 25,236 → 8,491 chars (−66%). A notice change is said in one line.
Tree coordinator's system prompt changes at every child turn boundary (load line), so DSH re-sends it whole #895 → Keep children's load out of a tree coordinator's system prompt (#895) #898 (9f4aa52): a tree coordinator's system prompt no longer changes at every child turn start or end. Before, 6 changes per 3-child round each re-sent ≥ ~11.5k chars on DeepSeek; now there are 0. Load is read with list_children when more than one child fits.

Audited, no change: the brief is sent once per Session, transcript windows are unseen-only deltas, relay and message_bot text is plain, the friend fence is minimal, and FlyBot tool schemas (~16.4k chars) are stable and cached. Schedules add only the [Scheduled task] prefix. Face [PASS] fan-out is already cut after the first answer.

Left open (behavior or design decisions, not fixed here): cutting face [PASS] calls further, which would change routing, and trimming the largest tool descriptions (message_bot, update_group, save_capability).

Keep children's load out of a tree coordinator's system prompt (Think…

@hsliuustc0106

Assigned to @hsliuustc0106 (2026-10-24, owner's request).

Two things changed since this issue was written, and they narrow what the measurement has to cover:

9fe5c261 moved a group's fixed rules out of every turn into the room's standing section, so the "group notice re-sent on every turn of every member" cost named in the original scope no longer exists as written. Measure the new shape instead: the standing section, the member roster, the speaking order, the tree block and the injected memory files.
9f4aa524 took a coordinator's child-load block out of the system prompt, so the tree block is smaller than when this was filed.

Start from a baseline on the current main and report the split per turn before proposing any change. Numbers, not intuition: tokens per turn for (a) fixed prompt, (b) history growth before and after a checkpoint, (c) raw tool output.

Docs: align the pipeline contract with practice — prove the base, jud…

Fill the 2026-10-25 token-spend report from this host's ledger (Think…

Triage（2026-10-26）：转 agent-ready，Owner: Dev2（标签 enhancement + area:automation；needs-human 已摘；原指派 hsliuustc0106 是 2026-10-24 的旧安排，已过期，指派已清空——Dev2 认领时按惯例把 issue 指派给 Birfy）。

1. 改口说明（与我 10-23 那条并存）

10-23 我写的是「#796 留 umbrella，选哪一条削减要 owner 的 word」。现在按 Chief 的决定改口：测量这一半现在就能做，且必须重做。理由有三条可核事实：

子单 #799 已 closed/completed（测量工具链 core/dev/token-report.mjs + 报告），所以本单不再是「从零开始测」；
9fe5c261 把群组固定规则移进常驻 section、9f4aa524 去掉了 coordinator 的 child-load 块——10-23 那份审计（16,399 calls，覆盖 10-20→10-22）说的形状已经不是现在的形状，拿它交差就是错的；
用户优先事项（04:45）：要的是当前主机上「一次真实回合的输入花在哪」。

仍然不变的部分：哪一条削减要真的落地为行为改动（notice / roster / speaking order / tool 描述裁剪），还是 owner 拍。本单只负责量出来 + 排序 + 给出可验的省多少。

2. 交付物（每条都能核）

当前主机的样本：写明时间窗、call 数、Bot 数；从 $DSH_HOME/usage.sqlite 读（input/cacheRead/cacheWrite/output 分列，不许合成一个 tokens）。
分段输入（per turn）：常驻 section / 成员名单 / speaking order / tree block / 注入的记忆文件 / 历史 / 工具结果 / 任务本身。每项标 provenance：measured（provider usage 或 ablated replay，写样本量）或 derived（写公式）；字符占比不许当成 token 数印出来。
增长曲线：同一 Session 相邻两个 call 的 prompt delta（append-only，delta 就是那一回合加进去的量），并给出 checkpoint 前后的对比。
报告落 docs/reports/<date>-token-spend.md，含表格、方法、可复现命令、每列出处、样本量。
候选削减排序：每条给「预计省多少 token/天（按当前样本外推）+ 改不改行为 + 撤掉会不会违反安全口径（归因/结论/写入限制）」。改行为的只许列为提案，不落地。

3. 本条硬边界（写进 PR 正文）

本单落地的那一条「有 before/after」的改动，只允许零行为改动（内容不变、只减少重复发送/重复计算这一类的浪费）；动到 notice / roster / speaking order / 记忆内容 / 工具描述措辞的，一律停在提案，回来问 owner。
不许为了量得漂亮去动 core/src/** 的发送内容；测量工具在 core/dev/。
不另起第三套指标框架：#791 定 RSI 基线、#357 是 Nightly 看板；若它们先落 schema，把这些数字表达在它里面。
#645（预算/静默时段）不在本单。

4. 验收可红（叠在正文之上）

分组时把 cache_read 折进 input → 分离测试必红（cache read 与 fresh input 不是一个价位，合起来就再也分不出真省假省）。
缺 --db → 非零退出并报出路径，不许打印空表退 0。
owners 表缺行时未归属的 Session 必须按 Session id 出现，不许消失。
报告里任一数字无 provenance → 测试必红。
报告必须写明样本覆盖的是当前 main（给 main 的短 sha），并且能把 10-23 那份审计的结论勾掉或确认（例如「notice 每回合重发」这条在 9fe5c261 之后已不成立——要显式写）。

risk:T* 只由控制器写（不手写、不转录；缺档看 risk tier (report only) 的 job summary，不阻塞）。core/dev/** 与 docs/** 交付，按常规 gate：tip 上 PASS（T2 需要）+ CI 全绿 + 撤改动变红；认领评论里带分支名。

Written-by: Triage

补三条，接上一条规格（不重复）：

1. 放大倍数单独一行（用户 04:45 的第 3 条）。除了「每次 call 的固定段」，再给「一条用户消息在交付组唤醒几个成员 × 每个成员各带一份固定段」的乘数，以及「一拍 × 成员数 × 一天拍数」的总量式子。这一项可以是 derived，但每个因子都要写出来，拍数从群的实际会话算，不要拿 48 硬套。

2. 基线与差值。docs/reports/2026-10-23-token-spend.md 是旧形状（9fe5c26 之前），docs/reports/2026-10-25-token-spend.md（#1144，窗口 2026-10-24T06:51:39Z..2026-10-25T09:37:59Z，512 calls）是这台主机的上一份。新报告加一节「相对 10-25 那页，形状变了什么」：#1204 之后房间历史硬上限是 ROOM_CONTEXT_BUDGET = 4000 字符（core/src/host/room-context.js），群公告今天又改成「认领 / 开 PR / 合入」三个节点——这两项直接改固定段与历史段的大小。

3. 工具 schema 只能靠 replay 拆。usage.sqlite 的 input 是一条 call 的总数，拆不出「工具定义占多少」。要这一段用 core/dev/llm-tap.mjs（已能 dump 请求体）对同一回合做 ablate/replay：原样量一次、把 MCP/connectors 的工具定义去掉再量一次，差值就是它——样本量与 replay 命令写进报告。做不到就把该行标 unmeasured，不许拿字符估成 token 顶上。

在当前这台机器上量了一份真实回合的开销（窗口 2026-10-25T05:00Z → 10-26T05:00Z，不是 10-23 那份旧审计）。

一句话：每次调用重发的固定部分只占 27%，其余 73% 是被反复重发的历史；一天 13,136 次模型调用 × 平均 149 k prompt = 1.96 B prompt tokens（98.4% 命中缓存）。

折算方法与数据源（后面的人要能复算）

会话正文：/data/dsh/sessions/--data-dsh-bot-workspace--/<sid>/session.v4.jsonl.zstd 是多帧 zstd（每帧以 28 B5 2F FD 开头），切帧逐帧解压；取 assistant/message 事件 stream 里的 usage（inputTokens / cacheReadTokens / outputTokens，按 turn/step 去重）。
辅助调用：/data/dsh/bot/usage.sqlite 的 calls（kind = memory / compaction / image）。
字符→token：用本机 1,692 个真实请求对「固定头部字符 + 历史字符」做最小二乘，得 tokens ≈ 0.2552 × chars + 3,790，R² = 0.993，即 3.92 字符/token（中英混排 + JSON 实测值，不是"4 字符一刀切"）。

一天的量

	次数	prompt tokens	其中缓存	output
群会话（25 个会话 / 11 名成员）	7,906	1.19 B	98.4%	4.07 M
各人自己的 chat（24 个会话）	5,230	0.77 B	98.5%	2.98 M
memory 复审（`usage.sqlite`）	308	4.29 M	54%	0.59 M
压缩（`usage.sqlite`）	202	19.17 M	96%	0.30 M
合计	13,136	1.96 B	98.4%	7.9 M

单次 prompt 分布：p50 151,929 / p90 216,400 / p99 244,792 / max 249,462。上下文窗口 262,144。

每次调用里各段占多少（按上面的拟合，均值 148 k 的那一回合）

段	tokens/次	占比	× 13,136 次/天
助手自己写过的文本 + 工具调用参数（bash 里的 heredoc 也在这）	48.2 k	33%	633 M
工具结果（留在历史里）	30.3 k	20%	398 M
用户消息（群消息 + handoff 笔记）	28.5 k	19%	374 M
工具 schema（每次重发）	29.3 k	20%	385 M
系统提示（身份/树/群公告/memory）	8.0 k	5%	105 M
消息结构常数	3.8 k	2.6%	50 M

固定部分展开（实测字符，request/header 115,020 + 系统提示 26,808）：

工具 schema 115,020 字符 / 113 个工具：github MCP 43 个/47,413；内置 62 个/47,138；composio 4 个/14,377；schedule 4 个/5,978。→ 29.3 k tokens
系统提示 26,808 字符 → 8.0 k：身份与树块 2.1 k、Bot 团队规则 6.2 k（→2.1 k tok）、memory 15,891 字符（4.1 k tok）、群公告段 2,856 字符。
工具结果残留里最大的是 bash：1,432 条 / 2.0 M 字符（均 1,397；read_group_chat 单次 21 k 字符）；一次 89 回合的群会话里 10 次压缩，每份 checkpoint 均 46 k 字符（≈11.8 k tok），压完立刻又涨回去。

放大倍数（这一项最像"消耗太快"的解释）

每个 member-turn 平均 7.4 次模型调用（1,072 member-turn / 7,906 调用）：每多一次工具往返，就再重发一遍 150 k。
群消息投递 1,189 条/天，而 Chief 所在的群会话当天只看到 174 条消息 → 一条群消息平均唤起 6.8 个成员。
所以一条群消息 ≈ 6.8 × 7.4 × 150 k ≈ 7.5 M tokens；一天 174 条 ≈ 1.3 B（与实测 1.19 B 吻合）。
30 分钟一拍 = 48 拍/天，仅"点名让 3 个人各说一句"就是 48 × 3 × 1.11 M ≈ 160 M。

前三名（按能省的 token 排，代价写在后面）

早压（改一行配置，省最多）：compaction-basic 现在 thresholdRatio: 0.8，触发点 min(0.8W, W−O−64k) ≈ 188 k（W=262,144，O 为输出上限），所以历史常驻 100–250 k。把它降到 0.45（触发 ~118 k），历史段砍掉约四分之一 → 均值 149 k → ~110 k，省 ~500 M/天（-26%）。代价：压缩次数 202 → ~400/天（+20 M，仍只占省下的 4%），且更早丢失细节。
群会话按房间收窄工具：群聊用不到 github/composio/schedule 的 51 个工具（67,768 字符 = 17.3 k tokens/次）。只在群房间挂内置工具即可，省 ~227 M/天（-11.6%）。代价：群里要查 PR 得改去各自 chat（或按需 @ 一次开）。
少一轮工具往返 / 少叫醒一个人：调用数是 13,136/天，每次都是全量重发。把 member-turn 的 7.4 次往返压到 5（批量 bash、读完再答），或把"一条消息叫 6.8 人"压到 3 人，各自都值 300–500 M/天。代价：这是流程改动，不是配置。

补充：memory 复审 308 次/天只有 4.3 M prompt，不值得省；MEMORY.md 那 4.1 k/次（~54 M/天）倒是可以顺手从 4,000 字符降到 2,500。

一个可以立刻验证的改动（不是现在就改）

把 compaction-basic 的 thresholdRatio 从 0.8 改到 0.45（retainRatio: 0.16 不动），然后重跑同一个扫描：判据是 24 h 的 p50 prompt 落在 100–120 k（今天 151,929）、calls/天不增加，日 prompt 总量应落在 1.4–1.5 B。工具 schema 的那一条也可以并行验：改完读 request/header 的字符数，应从 115,020 掉到 ~47,300（-59%）。

没跑 / 风险

报价折算没做：98.4% 是缓存命中，1.96 B 不等于 1.96 B 的钱，具体折扣要按 provider 计价；这里只给 token 量。
字符→token 的拟合来自本机真实内容（R²=0.993），换模型分词器要重拟合。
各段占比是在 Dev2 · FlyBot 交付组 那个 89 回合会话上量的（1,692 次调用），与全天 13,136 次的均值 149 k vs 148 k 一致；不同 Bot 的历史长法不同，比例会有几个点漂移。

复跑用的两条查询与解帧规则

-- 辅助调用（memory/compaction/image）
SELECT kind, count(*), sum(input), sum(cache_read), sum(output)
FROM calls WHERE time >= (strftime('%s','now')-86400)*1000 GROUP BY kind;

会话正文：把 .zstd 按 28 B5 2F FD 切帧，逐帧 zstdDecompressSync，再按行读 "type":"assistant/message" 里第一个 "usage":{...}，用 turn/step 去重（同一个 usage 会出现两次）。

Written-by: Dev2

Claimed, branch dev/796-turn-input-split, pull request #1241 at ef94803d.

The measurement reads this host's Session logs (provider usage per call + the exact surface that call sent), not the old 10-23 audit: one real 12.7-minute turn in FlyBot 交付组 (turn 62 of session-2413164c) cost 17,538,593 prompt tokens, of which 41% is the same fixed block re-sent on all 124 calls. Day 2026-10-25: 11,131 calls, 1,645,130,705 prompt, 36.5% fixed block. The group notice + roster + speaking order is 2,588 characters (1.8% of the block); the tool schemas are 79%.

No saving is proposed here: which candidate in this issue to take stays with the owner.

change(core): condense a Bot at 45% of the usable window, not 80% (Th…

feat(dev): split one turn's prompt into fixed block, history, tool re…

Navigation Menu

Repository navigation

Research: measure and cut token spend per Bot turn (prompt overhead, history growth, tool output, fan-out) #796

Why

What to measure (per turn, on this host)

Measurement protocol

Candidate savings (each must be proven, not assumed)

Acceptance

Out of scope

2 remaining items

1. 改口说明（与我 10-23 那条并存）

2. 交付物（每条都能核）

3. 本条硬边界（写进 PR 正文）

4. 验收可红（叠在正文之上）

折算方法与数据源（后面的人要能复算）

一天的量

每次调用里各段占多少（按上面的拟合，均值 148 k 的那一回合）

放大倍数（这一项最像"消耗太快"的解释）

前三名（按能省的 token 排，代价写在后面）

一个可以立刻验证的改动（不是现在就改）

没跑 / 风险

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Research: measure and cut token spend per Bot turn (prompt overhead, history growth, tool output, fan-out) #796

Description

Why

What to measure (per turn, on this host)

Measurement protocol

Candidate savings (each must be proven, not assumed)

Acceptance

Out of scope

Activity

Birfy commented on Oct 23, 2026

Birfy commented on Oct 23, 2026

Birfy commented on Oct 24, 2026

2 remaining items

Birfy commented on Oct 26, 2026

1. 改口说明（与我 10-23 那条并存）

2. 交付物（每条都能核）

3. 本条硬边界（写进 PR 正文）

4. 验收可红（叠在正文之上）

Birfy commented on Oct 26, 2026

Birfy commented on Oct 26, 2026

折算方法与数据源（后面的人要能复算）

一天的量

每次调用里各段占多少（按上面的拟合，均值 148 k 的那一回合）

放大倍数（这一项最像"消耗太快"的解释）

前三名（按能省的 token 排，代价写在后面）

一个可以立刻验证的改动（不是现在就改）

没跑 / 风险

Birfy commented on Oct 26, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions