fix(stream): rate-limit live-stream rotations to prevent disk thrash - #16
Merged
Conversation
Post-deploy verification showed the engine keeps the full stream in memory and re-materializes it into the fresh file on the first save after a rotation; with the in-memory mirror still over the cap, every subsequent publish re-rotated ~130MB (rename + full rewrite per observation). Rotations are now rate-limited to one per 10-minute window so disk stays bounded without the storm. Also records two deployment-proven corrections: npm ci needs --legacy-peer-deps on this lockfile (upstream transitive drift), and the CHANGELOG gains the cooldown note.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second follow-up to #14, found while verifying #15 on the live deployment.
Problem
The engine keeps the full viewer stream in memory and re-materializes it into the fresh file on its first save after a rotation. With the in-memory mirror still over the cap (130 MB history vs 32 MB cap), EVERY subsequent publish rotated again — rename + full ~130 MB rewrite per observation. Verified empirically on the deployed daemon: two observations produced two consecutive generations.
Fix
Rotations are rate-limited to one per 10-minute window (
ROTATION_COOLDOWN_MS, injectablenowMsfor tests). Disk stays bounded without the storm; the write path remains never-blocked and best-effort.Also, both deployment-proven:
docs/upstream-sync.md: deploy builds usenpm ci --legacy-peer-deps(plainnpm cifails on upstream lock drift transitive to @anthropic-ai/claude-agent-sdk).Battery
npx vitest run: 1967 passed / 1 skipped (+2 cooldown tests); same 3 known base artifacts.tscsorted identical to baseline set..r13-receipts/1787697755266-0-c80c6888).skills:checkgreen; evidence inventory refreshed.