Repository navigation
s3proxy: cut every build tool off MinIO, and store compressed blobs once - #60
Conversation
A non-empty disconnect set used to skip only the named tools, so the next tool read MinIO until somebody appended it. Naming any tool now skips every build-tool read and every tool's LRU artifacts. Named tools keep their existing reason labels; the rest count as tool_disconnect. A request with no tool identity still reads the backend. Co-authored-by: Cursor <cursoragent@cursor.com>
Identity uploads of an already-framed gzip or zstd body were zstd-encoded into the casblob. The digest is of those bytes, so the second pass only spent CPU. Those blobs are now stored as identity casblobs, and a read returns the original bytes. Co-authored-by: Cursor <cursoragent@cursor.com>
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c653b7a. Configure here.
| if err = binary.Write(f, binary.LittleEndian, h.chunkOffsets); err != nil { | ||
| return -1, fmt.Errorf("failed to write chunk offsets: %w", err) | ||
| } | ||
|
|
There was a problem hiding this comment.
Identity writes skip durability sync
Low Severity
The new Identity store path rewrites the casblob offset table after the body, then closes without an f.Sync. The zstd branch in this function and WriteZstdFrameAndClose both sync after that same seek-back. A crash can persist the body with the placeholder offsets, so readHeader rejects the blob and the guest-cache object is lost.
Reviewed by Cursor Bugbot for commit c653b7a. Configure here.


Summary
BAZEL_REMOTE_BACKEND_DISCONNECT_TOOLSset now skips MinIO for every build-tool read and every tool's LRU artifacts, not only the names in the set. Named tools keep their existing reason labels (go_disconnect, and so on). Any other tool counts astool_disconnect. A request with no build-tool identity still reads the backend, which is the quarantine prefix.Test plan
go test ./cache/s3proxy/ ./cache/disk/casblob/ ./config/bazel_remote_s3_backend_lookups_skipped_total{reason="tool_disconnect"}climbs for any tool that is not one of the named labels, andbazel_remote_s3_cache_hitsstays at zero for build-tool traffic.Made with Cursor
Need help on this PR? Tag
@codesmith-botwith what you need.Need help on this PR? Tag
@codesmith-botwith what you need. (Staging)