perf(authority): copy journal JSON without repeated primitive allocation

[LIHUA919]

Goal, gap and delivered result

Refs #4224 and RFC 7.2; follows merged #4931.

A source-stable matched-1m --cli baseline at 3009cdfbe had 13 passed / 1 failed / 10 missing: 100k scan-100 p95 was 251.909 ms against 250 ms. This PR removes repeated projection materialization through the existing strict-JSON owner. Historical failures and qualification holds remain recorded.

• Share an order-preserving JSON copy/validation walker beside canonical encoding.
• Use it for replay snapshots and independent public journal pages; SQLite reads historical projections from verified replay snapshots.
• Preserve unknown JSON metadata, key order, sparse arrays, negative zero, mutable-result isolation, cursor/lookahead rules and persisted proof bytes.

Owner: existing coordination codec, replay and journal boundaries. No new provider, capability, configuration, format, default or budget. Existing frontend/Lark JSON contracts remain unchanged; real CLI readback/update/failure tests cover the affected entrypoint.

Runtime acceptance

At aabf55979: 692 File/SQLite/archive/migration tests; 303 tests on a disposable real PostgreSQL 16.15 server, zero skipped; 5 real Python-to-TypeScript SQLite CLI tests; TypeScript typecheck passed. A historical-projection mutation fails the independent real-SQLite oracle. Matched 128-commit/1 MiB diagnostic scan p95 improved 194.902 -> 67.940 ms; this is not formal qualification.

The full source-stable formal report at aabf55979 was independently verified: 14 passed / 0 failed / 10 missing. Scan p95 was 137.646 ms at 10k and 77.918 ms at 100k; receipt p95 at 100k was 31.756 ms. Workload, sample counts, 20 cold CLI samples per command/axis, WAL/FULL, source fingerprint and cleanup were checked. Original failed reports remain intact.

Integration and CI repairs

All commits are signed and appended without rewriting history.

Commit	Repair and evidence
64a75ca94	Refresh stale census locator, reproduced on base/head with Python 3.11/3.12. Nine architecture tests pass; exact-head CI had 28 successful / 4 conditional skips / zero failures.
748ace442	Merge upstream 738115bde, preserving commits and resolving the census conflict to line1063. Architecture, real CLI and typecheck pass; four Python-child Node PATH failures pass with qualified Node22.22.3 after retaining the initial failures.
1df14ecc6	Repair six stale user-Todo/scheduler-ACK fixture assumptions reproduced on unchanged main. Related Python27, typed negative10 and final ACK2 pass; runtime rules remain unchanged.
53fab7f09	Replace a 0.4-second timing proxy with the real lifetime-lock invariant. Related suite30 passes; unrelated0.65-second delay distinguishes the old/new oracle, and a held-real-guard mutation is rejected. All four remote Python shards pass.

At 53fab7f09, the remaining Windows lifecycle failure was an ambiguous coordination.runtime_shadow.bootstrap response at its 10-second request deadline: 1 failed / 151 passed / 5 skipped. merge-gate failed downstream. The original failed result remains recorded. Exact-head workflow.

Latest fixture repair: 8e5c3dca3

The outbox fixture now handles the typed ambiguous-response exception by reading the exact completed bootstrap result under the existing maintenance lock. It verifies operation/request identity, immutable manifest, source path and capture binding. It never resends bootstrap. Missing, pending, wrong-operation and damaged evidence still fail; semantic rejection is not converted to success.

The RPC request budget remains 10 seconds and the existing maintenance-lock wait remains 5 seconds. The additional bounded readback checks an already-dispatched operation; it does not raise either deadline or change product runtime behavior.

A private real TCP relay withheld the actual bootstrap response beyond the unchanged 10-second deadline. Unchanged main (b6c3d3d4b) fails with a complete durable receipt; the candidate reads that receipt and continues, with one bootstrap dispatch in each arm. The complete outbox suite has 17 passed; the related outbox/management/Effect integration set has 88 passed. These controls prove response-loss recovery, not the cause of Windows runner latency. At 8e5c3dca3, Windows passed; two Python failures shared the stale CLI registry locator846/835 in CI merge 31f166846 (PR8e into mainb6). Unchanged main reproduces both failures (2 failed / 7 passed). This failed run remains retained.

The future-facing pass keeps recovery beside this fixture and reuses existing typed exception, lock and journal readers; a wider fixture framework is unnecessary. Public/private scans and whitespace checks pass; private probes, raw reports, databases and local paths are excluded.

Current integration/census repair: 0097931ea

Signed merge aa790fd8f appends current main at ee1ea64b0 without rewriting commits. The single overlapping Todo fixture conflict adopts upstream's valid user_action/goal_bound representation and retains the existing task-class readback assertion.

The actual merged source reproduces the two census failures (2 failed / 7 passed). A separate signed commit changes only the loopx/cli.py locator from846 to835. Site, column, API, classification and source policy remain identical; the direct complete inventory comparison is clean. Existing architecture inventories, resolved real File/SQLite fixture and real SQLite CLI entrypoints: 21 passed. Control-plane typecheck, whitespace and public/private checks passed. At exact head 0097931ea, full CI completed: 28 successful / 4 expected conditional skips / 0 failed or pending. All five workflows passed, including four Python shards, Windows, the real PostgreSQL server, release build and merge-gate. CI tested merge b36c96e60 (parents009/main ee1); its complete Git tree 77069228d1141494bda8070a6db13aee8ba0b539 equals the PR head tree. Accepted workflow.

The bounded future-facing pass keeps the existing AST census/generator owner and strict metadata equality. No classifier, protocol or additional test framework is needed. Upstream's fixture repair is reused; there is no duplicate fixture implementation. The PR base is main and its merge base is ee1ea64b0, so upstream history is excluded from the contribution diff.

The accepted009 source is now frozen in a separate qualification checkout. Its source-stable rehearsal completed100/1000 commits with real cold CLI3 samples per command/axis and verified cleanup on Node22.22.3/SQLite3.51.3. This is execution preflight, not formal10k/100k qualification; the full run and independent report acceptance remain pending.

Current main integration: 6902346f1

Two signed merge commits append current canonical main at 649826221 without rewriting the original PR history. The first resolves scheduler ACK fixture assumptions: an ACK for the current settled Turn succeeds; after a newer heartbeat, the old ACK is rejected as stale, while the new ACK and replay preserve scheduler bytes and Goal accounting. The second incorporates upstream GoalRef scope into the existing outbox ambiguous-response fixture without resending bootstrap. The checked-in registry census matches the merged source; the final contribution diff against main contains 12 focused files and excludes upstream history.

Local validation on the merged source: 102 architecture/quota tests, 25 outbox/registry tests, 61 real File/SQLite TypeScript snapshot/coordination tests, control-plane typecheck, public/private scan of 13 own files and whitespace checks passed. These are local acceptance checks; fresh exact-head CI for 6902346f1 is pending. Older 0097931ea CI and qualification evidence remain attributed only to their old source. The formal 10k/100k D2 run is held until this integrated head is accepted and frozen.

Exact-head CI fixture repair: 106b923a5

The 6902346f1 Python CI run completed with two independent stale test assumptions: the prompt-upgrade read test omitted the newly projected turn_start_capability_hook_dispatch field, and the dual-runtime-twin test still pinned one verified generated twin after main added the generated content-digest pair. Python shard 1 had 1 failed / 3387 passed / 18 skipped; shard 2 had 2 failed / 3384 passed / 20 skipped. pytest and merge-gate failed downstream. The failing tests and their owning hook/semantic-smoke sources had identical Git blobs on canonical main and the PR head before repair.

This signed test-only commit asserts the exact hook/read projection and two verified generated twins while retaining the independent-twin budget mutation. The complete related hook, Turn contract and semantic-binding suites pass locally: 71 passed. Runtime/API behavior and frozen formal evidence are unchanged. Public boundary and whitespace checks pass; the new exact-head CI remains pending.

Latest Host-process test repair: e4e754982

Exact-head 106b923a5 CI finished with one independent failure in TypeScript core shard 2: host_process.test.ts read the descendant counter as 27 after managed return and as an empty string 100 ms later. The test's in-place writeFileSync marker can be truncated if termination lands during a write, so an empty read cannot distinguish a partial write from continued descendant work. checks, pytest and merge-gate failed downstream. This test and the Host-process implementation were unchanged relative to main before the repair.

The fixture now writes the next count to a sibling file and atomically renames it into the observed marker. A surviving child still publishes changing counts; interruption during a write cannot expose an empty marker. Five complete local runs of the nine Host-process tests passed (45/45), as did control-plane typecheck, whitespace and public boundary checks. Runtime behavior is unchanged.

Signed merge d38a830aa incorporates canonical main b76ca16ce with no overlapping PR paths; the PR contribution merge base is that main commit. Signed test repair e4e754982 was pushed without rewriting history. That commit is historical; current-head CI and the formal D2 result are recorded below.

Current source and CI: d767b06f1

Signed merge d767b06f1 incorporated main 07311771d with no overlapping PR-owned paths. On that source, 48 architecture/GoalRef/hook tests, 60 real CLI/Turn-journal/shadow tests, 37 File/SQLite snapshot and Host-process Node tests, and control-plane typecheck passed locally. All five exact-head workflows then completed successfully: Python Tests, DCO, Dependency Review, real PostgreSQL Integration, and Release Artifacts.

Canonical main later advanced to 7e60e6999; its 82 changed paths since the integrated base have no overlap with the 15 PR-owned paths, and Git reports a clean merge tree. The frozen d767 report remains attributable only to d767 and does not qualify the later merged source automatically. This draft remains for maintainer review and merge.

Formal matched-1m result and remaining acceptance

A single supervised, source-stable d767 run completed both 1 MiB axes at 10k and 100k commits with Node 22.22.3, SQLite 3.51.3, WAL/FULL, 20 cold CLI samples per command and axis, and verified axis cleanup. Independent checks confirmed the source fingerprint, sample counts, exit and fixed-budget arithmetic. The measured profile failed: 8 passed / 6 failed / 10 missing. At 100k, receipt p95 was 65.172 ms against 50 ms; commit/head/receipt history-growth ratios were 2.634/3.086/2.170 against 2; cold CLI status p95 was 2594.854 ms against 2000 ms; and the cold CLI mutation increment was 1623.362 ms against 200 ms. The scan-100 p95 was 177.430 ms against 250 ms. No budget has been raised and no D2 pass is claimed. Diagnosis is open; previous aabf and 3009 reports retain their own source and run context.

The older frozen 64a75ca94 attempt remains interrupted without a complete report. The ten missing D2 conditions, elapsed/HALT recovery and promotion are separate holds. This PR does not resume that attempt, migrate active Goals or grant promotion. Maintainer review and merge remain required.

中文摘要

本 PR 在既有 strict-JSON 边界修复重复投影复制，保留字段顺序、未知元数据、可变对象隔离及持久证明。真实 File/SQLite 回归692项、真实 PostgreSQL16.15回归303项（零跳过）、真实CLI5项与类型检查通过；诊断扫描 p95 为194.902 -> 67.940ms，不替代正式资格。

冻结版本 aabf55979 的正式报告已独立核验：14通过、零失败、10 missing；10万笔扫描 p95 为77.918ms、收据为31.756ms。原失败报告保留。追加提交依次修复旧清单定位、上游冲突、六项过期 Todo/ACK 夹具与 claim 等待的计时代理；53fab7f09 的四个远端 Python 分片均通过。

该版本的剩余 Windows 失败发生在 bootstrap 的10秒请求期限，写操作可能已提交，不能自动重发。最新 8e5c3dca3 只修复 outbox 夹具：在现有维护锁下核验同一操作的持久完成结果、请求、不可变清单、源路径与绑定；缺失、未完成、错操作、损坏回执和语义拒绝仍失败。请求10秒和既有锁等待5秒均未增加，运行时规则不变。

真实 TCP 响应延迟对照中，未修改 main 有完整回执但旧夹具失败，候选读回执后继续，两组均只发送一次 bootstrap。outbox 套件17项、相关管理/Effect集成88项通过；它不证明 Windows 延迟根因，8e5c3dca3 的 Windows 检查通过；两个 Python 失败均来自 CI 合并基线的 census846/835 漂移，未修改 mainb6 同样复现。追加签署合并 aa790fd8f 接入 main ee1ea64b0，唯一夹具冲突采用上游合法定义并保留 task-class 读回断言。最新 0097931ea 只更新 CLI 定位846→835，site/列/API/分类/policy 不变。架构 inventory、真实 File/SQLite 夹具和 CLI 共21项、类型及边界检查通过；最新 0097931ea 的全部 CI 已结束：28通过、4项预期条件跳过、零失败或待处理；四个 Python 分片、Windows、真实 PostgreSQL、发布构建和 merge-gate 均通过。CI 合并版本b36的完整 Git tree 与009相同。随后 main 前进，原009的28/4/0仅属于旧合并树。现已追加两个签署合并提交，将规范 main 649826221 纳入 6902346f1；保留旧 ACK 失效、新 ACK 幂等和 Goal 状态隔离，也保留模糊 bootstrap 回执核验与上游 GoalRef 范围。合并后的本地架构/配额102项、outbox/清单25项、真实 File/SQLite TypeScript61项、类型和公开边界检查通过；head690 的远端 CI 已完成并暴露两处上游测试前提漂移：提示词升级 hook 新增的必读派发字段未纳入旧断言，生成的 content-digest 配对也未纳入旧的单配对计数。签署测试修复 106b923a5 保留派发内容和独立配对预算的负例，相关套件本地71项通过；新 head 的 CI 待验收，正式长跑继续等待该版本冻结。复用现有类型、锁与读回接口，不新增通用框架。

64a75ca94 的独立复测已中断，缺少退出回执与完整报告；其部分证据保留原来源，不能作为资格通过。现有 d767 正式复测已完整结束，但性能预算失败；十项 D2 缺口、自然时间/HALT 恢复和晋升仍未闭合。PR 保持草稿，交由维护者评审和合并。

进一步复测中，head106 的 TypeScript core 分片出现一项独立失败：子进程计数标记先读到27、后读到空串。原夹具在位写入，进程终止落在写入期间可能留下截断文件，不能据此断言子进程仍在继续工作。签署提交 e4e754982 改为同目录原子替换标记；存活的子进程仍会持续改变计数。Host 进程测试本地完整复跑5次共45项通过，类型与公开边界检查通过。另以签署合并 d38a830aa 接入最新 main b76ca16ce；该旧版本的远端 CI 状态由下文当前版本结果替代，正式 D2 资格仍未确认。

签署合并 d767b06f1 接入 main 07311771d，与 PR 自有路径无交集；相关本地 Python 测试48项、真实 CLI/Turn journal/shadow 测试60项、File/SQLite 快照及 Host 进程 Node 测试37项和类型检查通过。d767 的五项精确版本 CI 均已成功，包括真实 PostgreSQL。规范 main 此后前进至 7e60e6999，新增82个路径仍与本 PR 的15个路径无交集，但 d767 的报告不能自动证明后续合并源码。

一次受监督、源码稳定的 d767 正式 matched-1m 测试已完成 1 MiB、1万/10万笔两轴，Node22.22.3/SQLite3.51.3、WAL/FULL、每个命令每轴20个冷 CLI 样本与轴清理均核验。独立验证结果为 8通过、6失败、10缺失。十万笔时收据 p95 为65.172ms（预算50ms），提交/头/收据历史增长比为2.634/3.086/2.170（预算2），冷 CLI 状态 p95 为2594.854ms（预算2000ms），变更命令增量为1623.362ms（预算200ms）；scan-100 p95 为177.430ms（预算250ms）。固定阈值未上调，P1 已转为查明原因；不声明 D2 通过或推进晋升。

[mergify]

This pull request has merge conflicts with main and cannot be merged
until they are resolved. Please rebase or merge the base branch, @LIHUA919.

Choose the remote for the base repository, not an out-of-date fork.
For a fork clone, first inspect git remote -v; upstream must point
to https://github.com/loopx-project/loopx.git. If it is absent, add it
with git remote add upstream https://github.com/loopx-project/loopx.git.
Then run:

git fetch upstream
git rebase upstream/main
# Resolve each conflict, git add the resolved files, then git rebase --continue.
git push --force-with-lease origin HEAD

For a same-repository clone whose origin points to
https://github.com/loopx-project/loopx.git, use origin instead of
upstream for fetch/rebase. If you prefer merging the base, use
git merge <base-remote>/main and push normally.

Keep the DCO Signed-off-by trailer on every commit when you rebase.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts with main and cannot be merged
until they are resolved. Please rebase or merge the base branch, @LIHUA919.

Choose the remote for the base repository, not an out-of-date fork.
For a fork clone, first inspect git remote -v; upstream must point
to https://github.com/loopx-project/loopx.git. If it is absent, add it
with git remote add upstream https://github.com/loopx-project/loopx.git.
Then run:

git fetch upstream
git rebase upstream/main
# Resolve each conflict, git add the resolved files, then git rebase --continue.
git push --force-with-lease origin HEAD

For a same-repository clone whose origin points to
https://github.com/loopx-project/loopx.git, use origin instead of
upstream for fetch/rebase. If you prefer merging the base, use
git merge <base-remote>/main and push normally.

Keep the DCO Signed-off-by trailer on every commit when you rebase.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[Duang777]

Commit 106b923a57c60d411dba9276c37d1f927e0023bc also fixes three failures currently reproducible on main@996bcc027 and blocking #5340. I applied only that signed commit to an isolated current-main worktree; the affected tests passed (3 passed in 6.74s).

Could you split this test-only commit into a small standalone PR? That would let the baseline repair land without waiting for this draft performance change and its main synchronization.

[Duang777]

I extracted the test-only repair into #5344 to unblock the current main baseline while preserving Lihua as the commit author. No performance changes from this draft were copied.

[mergify]

This pull request has merge conflicts with main and cannot be merged
until they are resolved. Please rebase or merge the base branch, @LIHUA919.

Choose the remote for the base repository, not an out-of-date fork.
For a fork clone, first inspect git remote -v; upstream must point
to https://github.com/loopx-project/loopx.git. If it is absent, add it
with git remote add upstream https://github.com/loopx-project/loopx.git.
Then run:

git fetch upstream
git rebase upstream/main
# Resolve each conflict, git add the resolved files, then git rebase --continue.
git push --force-with-lease origin HEAD

For a same-repository clone whose origin points to
https://github.com/loopx-project/loopx.git, use origin instead of
upstream for fetch/rebase. If you prefer merging the base, use
git merge <base-remote>/main and push normally.

Keep the DCO Signed-off-by trailer on every commit when you rebase.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork