fix(services/gcs): reject multipart writes with if_not_exists - #8046
Open
dgvj-work wants to merge 1 commit into
Open
fix(services/gcs): reject multipart writes with if_not_exists#8046dgvj-work wants to merge 1 commit into
dgvj-work wants to merge 1 commit into
Conversation
GCS XML multipart uploads cannot enforce request preconditions, so fail closed with Unsupported instead of silently overwriting. Single-shot conditional writes continue to use ifGenerationMatch=0 on the JSON API.
dosubot
Bot
added
size:S
Member
|
Returning If this is truly impossible, it's probably better to consider the nuance for "write_with_if_not_exists". e.g., either having some partial support flag or declaring false while sacrificing one-shot implementation. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #8040
Rationale for this change
GCS supports
write_with_if_not_existsfor single-shot uploads viaifGenerationMatch=0, but the multipart writer uses the XML API, which cannot carry request preconditions. After #8028/#8041 added chunked-writer coverage, that path could silently overwrite an existing object whenif_not_exists(true)was set.Moving multipart onto resumable uploads (issue option 2) is a larger change. This PR takes the fail-closed approach: refuse the unsupported combination instead of advertising success and overwriting.
What changes are included in this PR?
UnsupportedfromGcsWriter::initiate_partwhenif_not_existsis set.write_onceconditional creates working via the JSON upload API.test_writer_write_with_if_not_existsso GCS assertsUnsupportedinstead of skipping.Are there any user-facing changes?
Yes.
writer_with(...).if_not_exists(true)on GCS now returnsUnsupportedwhen the multipart path would be used, instead of overwriting.write_with(...).if_not_exists(true)is unchanged.AI Usage Statement
None.