Skip to content

fix: rollover durability β€” no malformed archives or lost logs on abrupt app death#427

Draft
tony19 wants to merge 2 commits into
mainfrom
claude/triage-recent-issues-i5277s
Draft

fix: rollover durability β€” no malformed archives or lost logs on abrupt app death#427
tony19 wants to merge 2 commits into
mainfrom
claude/triage-recent-issues-i5277s

Conversation

@tony19

@tony19 tony19 commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Third round of issue triage, tackling the rollover durability reports.

No malformed archives or lost logs on interrupted rollover (fixes #369)

The reporter's fleet kills the app right at midnight β€” mid-rollover β€” and ends up with malformed .gz archives. Three defects in Compressor conspired:

  1. Compression streamed straight into the final .gz/.zip name. A kill mid-compression left a truncated archive that looks complete, is never repaired, and blocks any retry ("exist already. Aborting").
  2. A failed compression still deleted the source file β€” the delete ran unconditionally after the try/finally, so an exception (disk full, unwritable dir) silently destroyed that period's logs.
  3. Stranded temp files were never cleaned. With the file property set, rollover renames the active log to <name>.gz<nanotime>.tmp before compressing; a kill leaves that data invisible to maxHistory/totalSizeCap cleanup forever.

The fix:

  • Compression streams into <target>.tmp and atomically renames to the final name only after a fully successful write and close (close moved in-try, so a failed close counts as a failed compression). A malformed archive can no longer exist under the final name β€” after a kill you get the intact uncompressed source instead of a corrupt .gz.
  • The source file is deleted only after the rename succeeds. On any failure the partial temp file is removed and the source is preserved.
  • Stale *.gz.tmp / *.zip.tmp partials from previous kills are swept when a later compression runs in the same directory β€” only when untouched for 30+ minutes (an in-flight compression writes continuously, so it can't be swept), and the filter can't match the rename-stage <name>.gz<digits>.tmp files, which hold original log data.

New CompressorDurabilityTest covers: success leaves no temp file; failure preserves the source (both GZ and ZIP β€” this is the regression test for defect 2); an interrupted compression can be retried over its stale temp file; and the sweep removes old partials while sparing fresh temps and rename-stage data files.

Opt-in syncOnFlush for power-off durability (refs #371)

The #371 reporter loses the tail of the log when the device is powered off. That's physics, not a bug: an ordinary flush hands bytes to the OS page cache, which survives a process kill but not a power-off β€” the kernel may not have written the tail back to flash yet. The only fix is fsync, which costs real latency, so it's opt-in:

<appender class="ch.qos.logback.core.rolling.RollingFileAppender">
  <file>...</file>
  <syncOnFlush>true</syncOnFlush>
  ...
</appender>

When enabled, every flush also syncs the file descriptor to the storage device, so with the default immediateFlush each event is durable the moment the append returns. The stream is also flushed (and synced) on close. Added to FileAppender, ResilientFileOutputStream, and the XML schema; a sync failure flows into the existing resilient-recovery path. Documented with the suggestion to pair it with AsyncAppender when write latency matters.

Verification

Compiled all main sources and ran the tests standalone against Robolectric's android-all jar (Gradle/AGP can't run in this sandbox β€” dl.google.com blocked): the 6 new durability tests, the pre-existing CompressTest (3), the end-to-end TimeBasedRollingTest (10, exercises real GZ/ZIP rollovers through the new code), and the new syncOnFlush test all pass; the updated schema still validates configs using syncOnFlush. CI runs the full suite.

πŸ€– Generated with Claude Code

https://claude.ai/code/session_01JD28c26fMeRWhJNWyNWk5Q


Generated by Claude Code

claude added 2 commits July 10, 2026 04:53
An app killed during rollover compression left a truncated archive at
the final .gz/.zip name: Compressor streamed straight into the target
file, so the malformed archive looked complete, was never repaired
(retries abort with 'exist already'), and with the file property set
the original data was stranded in a nanotime-suffixed .tmp file that
nothing ever cleaned. Worse, a compression that failed with an
exception still deleted the source file, silently destroying that
period's logs.

Compression now streams into '<target>.tmp' and atomically renames to
the final name only after a fully successful write and close, so a
malformed archive can never appear under the final name. The source
file is deleted only after the rename succeeds; on any failure the
partial temp file is removed and the source is kept. Stale '*.gz.tmp'
/ '*.zip.tmp' partials from previous kills are swept when a later
compression runs in the same directory β€” only when old enough that
they cannot belong to an in-flight compression, and never matching the
rename-stage temp files that hold original log data.

Fixes #369

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JD28c26fMeRWhJNWyNWk5Q
An ordinary flush hands bytes to the OS page cache, which survives a
process kill but not an abrupt device power-off β€” the kernel may not
have written the tail of the log back to flash yet, which is why users
lose the last stretch of logs when a device is powered off.

Add an opt-in syncOnFlush property to FileAppender (and the schema):
when enabled, every flush also syncs the file descriptor to the
storage device, so with the default immediateFlush each event is
durable the moment the append returns, at the cost of a slower write
per event. The stream is also flushed (and synced) before close.

Refs #371

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JD28c26fMeRWhJNWyNWk5Q
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

App death during rollover causes losing logs and malformed gz files

2 participants