Skip to content

fix(gs): decode sceGsSetDefDBuff trailing args via EABI registers, not o32 stack offsets - #186

Draft
smmathews wants to merge 1 commit into
ran-j:mainfrom
smmathews:feature/23-gssetdefdbuff-eabi-args
Draft

fix(gs): decode sceGsSetDefDBuff trailing args via EABI registers, not o32 stack offsets#186
smmathews wants to merge 1 commit into
ran-j:mainfrom
smmathews:feature/23-gssetdefdbuff-eabi-args

Conversation

@smmathews

Copy link
Copy Markdown
Contributor

fix(gs): decode sceGsSetDefDBuff's trailing args via EABI registers, not o32 stack offsets

The bug

sceGsSetDefDBuff reads ztest/zpsm/clear from stack offsets 16/20/24, but under this runtime's EABI those args are register-passed in $t0-$t2 (GPRs 8-10), so the reads pick up caller-frame bytes and seed the double buffer's Z-test/Z-format from stale stack data on every call.

The fix mirrors an existing sibling

decodeGsTrailingArgs3 (registers first, all-zero-register stack fallback) is already used by sceGsSetDefDBuffDc for the identical three args. This change makes sceGsSetDefDBuff consistent with it. The helper degrades to the exact prior stack reads only when all three registers are zero, so the sole behavioural change is for register-passed nonzero trailing args (the buggy case).

Explicit non-fix

sceGifPkRefLoadImage in the same file also uses readStackU32 (offsets 0 and 8) and is correct as written, untouched by this PR: it consumes eight register args first (GPRs 4-11), so its width/height are genuinely the 9th/10th args and belong on the stack. The distinguishing test for any readStackU32 site is how many register args precede it: under eight is a bug, eight or more is correct.

Scope

No repo-wide EABI audit of other readStackU32 sites was done (SIF.cpp and MemoryCard.cpp also call it); flagged as separate follow-up work.

Test / repro

A register-ABI regression test seeds distinct sentinels in GPRs 8-10 and at the o32 stack offsets, then asserts the written ZBUF/TEST fields reflect the register values rather than the stack bytes.

Build and run:

cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_FLAGS=-msse4.1 -DCMAKE_CXX_FLAGS="-msse4.1 -include cstdint"
cmake --build build
./build/ps2xTest/ps2x_tests

Durable mutation claim: revert the three-line helper swap in sceGsSetDefDBuff and the two new assertions report the stack sentinels instead of the register sentinels.

…t o32 stack offsets

sceGsSetDefDBuff read ztest/zpsm/clear with readStackU32 at offsets
16/20/24. Under the EABI this runtime targets, arguments 5-8 arrive in
$t0-$t3 (GPRs 8-11), so those stack offsets hold caller-frame bytes, not
the trailing args. The sibling sceGsSetDefDBuffDc already decodes the
same three args correctly with decodeGsTrailingArgs3 (registers first,
stack only as an all-zero fallback); this routes sceGsSetDefDBuff through
the same helper. Adds a register-ABI regression test that seeds distinct
register and stack sentinels and asserts the written ZBUF/TEST fields
reflect the register values.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant