Skip to content

Bound the orientation count of portable tex coord prediction - #1237

Open
Filyus wants to merge 1 commit into
google:mainfrom
Filyus:bound-tex-coord-orientation-count
Open

Filyus wants to merge 1 commit into
google:mainfrom
Filyus:bound-tex-coord-orientation-count

Conversation

@Filyus

@Filyus Filyus commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

MeshPredictionSchemeTexCoordsPortableDecoder::DecodePredictionData() reads the number of orientations as an int32_t from the stream and checks only that it is not negative. The count then sizes the orientation storage and drives a loop that cannot stop early, since RAnsBitDecoder::DecodeNextBit() returns false for a zero bit and for an exhausted buffer alike. A 224-byte stream declaring about two billion orientations, found by fuzzing the decoder, takes 2 s to decode with a release build of 1.5.7 and 14 s with main under AddressSanitizer, and reports success.

Each predicted entry consumes at most one orientation, so a count above the number of entries can never be used whatever the stream contains. The count is now refused when it exceeds the size of the data-to-corner map. The bound is structural rather than a guess from the remaining buffer size: the orientations are rANS-coded, and a run of equal ones takes far less than a bit each.

TestTexCoordOrientationCountIsBounded decodes that stream and expects a failure, which now comes in 1 ms.

MeshPredictionSchemeTexCoordsPortableDecoder::DecodePredictionData()
reads the number of orientations as an int32_t from the stream and
checks only that it is not negative. The count then sizes the
orientation storage and drives a loop that cannot stop early, since
RAnsBitDecoder::DecodeNextBit() returns false for a zero bit and for an
exhausted buffer alike. A 224-byte stream declaring about two billion
orientations, found by fuzzing the decoder, takes 2 s to decode with a
release build of 1.5.7 and 14 s with main under AddressSanitizer, and
reports success.

Each predicted entry consumes at most one orientation, so a count above
the number of entries can never be used whatever the stream contains.
The count is now refused when it exceeds the size of the data-to-corner
map. The bound is structural rather than a guess from the remaining
buffer size: the orientations are rANS-coded, and a run of equal ones
takes far less than a bit each.

TestTexCoordOrientationCountIsBounded decodes that stream and expects a
failure, which now comes in 1 ms.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant