Skip to content

fix(storage): truncate local files on overwrite - #3649

Open
tttboy123 wants to merge 2 commits into
e2b-dev:mainfrom
tttboy123:fix/fs-overwrite-truncate-3642
Open

tttboy123 wants to merge 2 commits into
e2b-dev:mainfrom
tttboy123:fix/fs-overwrite-truncate-3642

Conversation

@tttboy123

@tttboy123 tttboy123 commented Sep 18, 2026

Copy link
Copy Markdown

Summary

  • truncate Local FS objects before complete replacement writes
  • clear stale .uncompressed-size sidecars after successful uncompressed replacements
  • cover Put and uncompressed StoreFile across shorter, longer, equal-length, and empty replacements
  • cover compressed-to-uncompressed replacement through both public write paths

Why

The Local FS backend opened existing files without O_TRUNC. Writing a shorter replacement started at offset zero but retained the previous suffix, unlike object-storage put semantics.

A compressed object also leaves an .uncompressed-size sidecar that Size() prefers. If the same object is later replaced through an uncompressed path, that sidecar must be removed or Size() reports the old value even when the replacement bytes are correct.

The truncation flag remains limited to getHandle(false), whose only callers are complete-object replacement paths. Sidecar cleanup happens only after a successful uncompressed copy. Read/range-read and compressed StoreFile behavior remain unchanged.

Validation

  • replacement matrix for Put and uncompressed StoreFile: shorter, longer, equal-length, and empty
  • compressed-to-uncompressed sidecar regression for both write paths
  • focused Local FS suite
  • race-enabled Local FS suite
  • all storage tests not requiring Docker/testcontainers
  • package compile-only test
  • golangci-lint run --new-from-rev=upstream/main ./... from packages/shared (0 issues)

The complete storage suite was also attempted; its Docker-backed Azure, GCS, and S3 integration tests cannot start locally because rootless Docker is unavailable.

Closes #3642.

@tttboy123

Copy link
Copy Markdown
Author

Follow-up scenario verification found one additional Local FS replacement edge case and it is now included in b30f27fb9.

A compressed write creates an .uncompressed-size sidecar, and Size() prefers that value. Before this follow-up, replacing the same object through Put or uncompressed StoreFile fixed the bytes but left the old sidecar, so a 5-byte replacement could still report size 63. The new regression reproduced that on both public paths before the fix.

The uncompressed paths now remove the stale sidecar only after a successful copy. I also expanded both replacement regressions to cover shorter, longer, equal-length, and empty payloads.

Fresh verification after the update:

  • focused Local FS smoke suite: pass;
  • focused Local FS race suite: pass;
  • all non-Docker storage tests: pass;
  • package compile check: pass;
  • golangci-lint: 0 issues;
  • git diff check: pass.

The Docker-backed integration tests remain unavailable locally because rootless Docker is not installed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Local filesystem writes can retain stale tail data when overwriting shorter objects

1 participant