Repository navigation
Performance: coalesce redundant transcript paints during output bursts #505
Description
Problem
The frontend synchronously builds a complete frame for every PTY output chunk, repeatedly wrapping the entire growing transcript. A deterministic production-frame fixture produces 200 paints for 200 chunks. CPU profiling attributes substantial work to transcript wrapping and garbage collection.
On macOS 27 / Apple Silicon / 8 GB / Node 26.12.0, unchanged dev be3f6ab, a real zsh command emitting 1,000 Unicode lines has a warm median of 349 ms (Native prompt) / 348 ms (Prompt None). Disposable HOME, configuration and service; built CLI; 100×30 PTY. An output-only frame coalescing prototype measures 127/128 ms with identical settings; post-output frontend RSS also decreases. These are PTY observer timings, not physical display measurements.
Scope and acceptance
- Coalesce output-triggered paints only; parse every chunk immediately and preserve all stored output and exact copy semantics.
- Keep lifecycle/input ownership, queue approvals, paste review, terminal queries and fullscreen forwarding synchronous and unchanged.
- Keyboard, prompt and resize frames remain immediate and consume redundant pending output work.
- Cancel pending timers at shutdown; no idle polling or transcript cache.
- Deterministic regression coverage plus existing canonical verification and independent review.
- Save repeatable benchmark evidence and report timing/memory regressions honestly.
- Physical Ghostty/Zed output, typing during output, Ctrl+C/Ctrl+Z, resize and fullscreen restoration remain NOT RUN until a person performs them. Keep this issue open through that gate.
Refs #20. Performance/stabilization only; feature freeze remains. PR targets dev and must not be auto-merged.
Activity
raiseCatError commentedon Oct 27, 2026
Implementation merged into
devvia #506 asfe40fd52fb68f0db81b3d35011d15e10e49119c4after the approved sequential preflight. Exact resulting dev push CI 40481488888 passed all macOS/Ubuntu shards, Quality, Node22, Fedora and both timing-smoke jobs. #511's fullscreen correctness fix was merged first. Original source branches/commits are preserved.Physical Ghostty/Zed QA is NOT RUN. This issue stays OPEN, with
needs-human-test. Output batching is merged. Test long numbered Unicode output while typing, scrolling and selecting/copying; resize, cancel, and detach/reattach less/vim at changed and identical pane sizes. Expect exact output/copy/order, correct input ownership and a genuine final-size program repaint.Integration housekeeping remains blocked on PR #512, head
48d2530d0ca3c394693b06cd567bd5568f83f1c8: full local canonical verification passed and all six hosted macOS/Ubuntu shards passed, but CI 40504558737 failed its required gate because Node22 exceeded the existing five-minute job limit. Shell installation consumed about 4m37s; the test step was cancelled after about 21s. Its emitted test summary has no assertion failures, but cancellation is not a passing job. No automatic retry, timeout change or further push/merge followed the stop.The focused physical QA guide and updated report are available in that unmerged PR. Keep existing service sessions alive; use separate QA sessions/runtime instead of terminating them to load new code.
Earlier matched measurements retain the broad gains and the observed +6.7% history-typing p95 / +2.3% deletion p95 differences without causal attribution. Fresh final-dev benchmark repetition and post-test-PR final push verification have NOT RUN because the CI gate stopped integration. The runtime content on current dev matches the prior validated combined runtime; no new feature/refactor was introduced.
Project-board updates were unavailable because the current token lacks
read:project; the existing label and this handoff provide the manual-validation gate. Exact merge/CI metadata, local logs and cancellation annotations are retained in/Volumes/T7Tmp/ai-dev/nmsh-merge-20261027/.raiseCatError commentedon Oct 27, 2026
Final integration gate update: #512 merged as
c690de055daeb7b33ed8b5e302ef01133b3e55b4after the single authorized unchanged PR rerun passed, including Node22 and aggregate verification. Approved head remained48d2530d0ca3c394693b06cd567bd5568f83f1c8; merge tree/parents matched fresh local simulation.The resulting exact dev push CI40512364215 FAILED macOS2/3: existing
hostProfiles.test.tstestcase “reattaching across profiles updates input modes while keeping the real shell environment” timed out waiting for the completed journal forecho SIZE-$(stty size)at line116 / liveFrontend.ts:229. Zero cancellations; process leak check passed. The test/helper are unchanged from originaldev. Root cause is unconfirmed; this is a real failed gate, separate from the earlier Node22 setup-budget timeout. No automatic rerun, fix push or rollback followed it.All other five platform shards, Node22, Fedora, Quality and both timing jobs passed. Fresh merged-source local fullscreen/interaction tests also passed, but do not override the failed gate. Prior full canonical verification passed on the identical PR512 tree.
Final merged-source benchmarks are NOT RUN because the required push gate failed first. Earlier matched measurements remain in the performance report, including +6.7% history-typing p95 / +2.3% deletion p95 fixed-versus-pre-fix differences without causal attribution. No new final performance result is claimed.
Physical Ghostty/Zed QA remains NOT RUN; keep this work open / Needs Human Test. Use the merged physical QA guide. Existing user services/sessions remain alive. Next step: investigate the preserved macOS cross-profile attach/journal failure before resuming final validation. Evidence and regression sources remain under
/Volumes/T7Tmp/ai-dev/nmsh-merge-20261027; no master/release/tag/deployment/Homebrew action occurred.raiseCatError commentedon Oct 27, 2026
The focused host-profile correction remains OPEN / UNMERGED in PR513, exact head
a57568be5e8b11cf927a00371c1b4783cadde53d. Normal exact-head PR CI40521828253 passed. Complete local canonical verification, real fullscreen/resize/combined interaction tests, deterministic original-dev red/corrected-source green, bounded concurrent stress and independent review passed. No production/shared-helper change was made.The complete reliability gate remains FAILED: comprehensive40521955602 failed the existing macOS3/3
inputOwnershipLive.test.tszsh waiting-read testcase, timing out forANSWER=hellowith the read running and a line held for approval. Other platform shards, Node22, Fedora, Quality and both timing jobs passed. No retry of either validation run was requested.The single permitted hosted diagnostic40549379269 passed all six bounded two-worker read cases, the full original shard and process-leak checks. Its89 captured OS samples all contained foreground processes and modes;41 on-demand look results were visible/hidden. These are passing controls, not the missing trace from the failing run, and do not establish a fix or historical cause. Local isolated plus six concurrent read controls also passed. The cause of this additional #473-family failure remains unresolved; no speculative runtime correction or deadline/coverage relaxation is justified by the captured evidence.
Diagnostic source/artifacts remain preserved at branch
diag/input-ownership-20261027, head901fe7465372216b5721291f59ccaa656d85805e, and T7 evidence root/Volumes/T7Tmp/ai-dev/nmsh-stabilization-20261027. Never merge that diagnostic branch or its workflow replacement. The focused PR contains only three test files and two documentation files. Original dev remainsc690de055daeb7b33ed8b5e302ef01133b3e55b4; no implementation PR integration was repeated.Final median/p95/RSS/CPU/startup/idle benchmark timings remain NOT RUN, because the comprehensive gate is not satisfactorily resolved. Built actual merged-dev functional stress did pass exact Unicode output/raw bytes, draft/input order, scrolling/copy, fast resize and Ctrl+C followed by another command. The original matched measurements and +6.7% history-typing p95 / +2.3% deletion p95 differences remain retained without causal attribution. Measurement workloads, CPU/idle extensions, provenance and plan are preserved for resumption; no fresh performance gain or unmeasured startup/idle result is claimed.
Physical Ghostty/Zed QA remains NOT RUN. Keep #473 and performance issues open. Existing sessions/stashes/unique work are preserved; only clean task-owned disposable worktrees/builds are removed after evidence preservation. No master/release/tag/deploy/Homebrew action occurred. Next step requires capturing the failing read/ownership decision (exec state, OS probe and routing) in its failing environment, not another uninstrumented lucky retry.
raiseCatError commentedon Oct 27, 2026
Follow-up evidence, separate from PR validation: remote dev remains
c690de055daeb7b33ed8b5e302ef01133b3e55b4; PR #513 remains OPEN/UNMERGED at unchangeda57568be5e8b11cf927a00371c1b4783cadde53d. Normal exact-head PR CI passed; comprehensive40521955602 remains FAILED. No validation retry, production change, timeout increase, skip, merge or release action. Physical Ghostty/Zed QA remains NOT RUN.Final merged-source benchmarks are now COMPLETE independently of the input gate. Exact original
be3f6abevs actual mergedc690de05, built CLI, identical dependencies/configuration, Native/zsh100×30, private HOME/service, Node26.12.0/macOS, no simultaneous local tests/builds/profiles. Warm median / nearest-rankp95:Observed limits/regressions: idle CPU3.35%→3.75% of one core (+0.41 percentage point), idle-window frontend RSS129.28→152.78MiB, output-finish service RSS20.38→70.62MiB. Highly variable n3; service output snapshot is at different elapsed lifetimes (38–49s baseline vs0.27–0.30s merged), so no leak/heap-growth/causal claim. Idle service RSS49.86→60.53MiB median; no sustained-idle claim. Five-second idle CPU and memory snapshots cover owned frontend/service/shell, not entire descendants or energy. Output CPU total48780→520ms median across these roles.
Runtime n3 warm sessions, history n4, long editing n7; key samples correlated. Runtime/history p95 is the observed warm maximum. No OS-cache purge/true-cold or physical-pixel claim. First-observed samples retained separately. Every10k output row/order and archive record/order validated. Raw samples, ranges, standard deviations, exact harnesses and provenance are retained at
/Volumes/T7Tmp/ai-dev/nmsh-stabilization-followup-20261027/(measurements.json,FINAL_REPORT.md, runtime/history/input JSON). Prior +6.7% history-typingp95 and +2.3% deletionp95 fixed-vs-pre-geometry differences remain retained without causal attribution; no pooling with this experiment.Benchmarks do not imply CI is green or authorize a release. Existing sessions, package edits, stashes and prior QA evidence preserved.