Reject negative Postgres binary field lengths#533
Closed
marknefedov wants to merge 1 commit into
Closed
Conversation
ReadValue treated -1 as NULL but let every other negative length through to ReadString(idx_t), where -2 becomes 18446744073709551614. The bounds check there added that to buffer_ptr first, which overflows the pointer and wraps back below end, so the check passed and the length reached AddStringOrBlob unchanged. Reject any negative length other than -1, and bound-check ReadString against the bytes remaining instead of via pointer arithmetic. The JSONB branch is fixed separately: it decrements value_len after the NULL check, so a zero length underflowed to -1 and hit the same path. Fixed width types still only check their length with D_ASSERT, which is compiled out in release builds. Those go through the bounds checked ReadInteger so they cannot read out of bounds, but they do misparse silently - worth a follow up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PostgresBinaryParser::ReadValuetreats-1as NULL but lets every other negative field length through:https://github.com/duckdb/duckdb-postgres/blob/main/src/postgres_binary_parser.cpp#L154
ReadStringtakes anidx_t, so-2arrives as18446744073709551614, and the bounds check adds it tobuffer_ptrbefore comparing. That overflows the pointer and wraps back belowend, so the check passes:The length then reaches
StringVector::AddStringOrBlobunchanged. Reachable fromread_postgres_binaryon a corrupt or hand-crafted file.Fix
-1inReadValue.ReadStringagainst the bytes remaining rather than via pointer arithmetic, so a corrupt length cannot overflow past the guard.value_lenafter the NULL check, so a zero length underflowed to-1and hit the same path. Rejecting< -1alone does not cover this.This also makes the existing
value_len < sizeof(uint16_t) * 4check in the DECIMAL branch meaningful — it compares a signedvalue_lenagainst asize_t, so a negative length used to wrap to a huge unsigned value and pass.Test
Two 27 byte fixtures under
test/data/, read bytest/sql/misc/postgres_binary_read_corrupt.test:postgres_binary_negative_length.bin— field length-2, checked againstVARCHAR,BLOBandINTEGERcolumnspostgres_binary_oversized_length.bin— field length1000in a 27 byte file, locking in theReadStringbounds checkKnown gap, not addressed here
Fixed width types still validate their length only with
D_ASSERT, which is compiled out in release builds. Those all go through the bounds-checkedReadIntegerso they cannot read out of bounds, but a wrong length is silently misparsed rather than rejected. That felt like a separate change; happy to fold it in if you would prefer one pass over the whole parser.Notes
Draft — not built locally, relying on CI. The overflow characterisation above comes from a standalone harness, not from a live run.