fix(gs): correct depth write for all AFAIL alpha-fail modes - #188
Open
smmathews wants to merge 1 commit into
Open
fix(gs): correct depth write for all AFAIL alpha-fail modes#188smmathews wants to merge 1 commit into
smmathews wants to merge 1 commit into
Conversation
The GS documents a four-way truth table for what TEST.AFAIL does when the alpha test fails: KEEP writes neither color nor depth, FB_ONLY writes color only (no Z), ZB_ONLY writes depth only, and RGB_ONLY writes RGB while preserving destination alpha (no Z). The old rasterizer used a single writeFramebuffer flag to stand in for both the color and depth disposition, and writePixel bailed out early only when that flag was false. Every case that survived the early-out then fell through to an unconditional depth write gated only by ZMSK. That made pass, FB_ONLY, and RGB_ONLY all write depth on alpha-fail, against the table, while ZB_ONLY was folded into the same branch as KEEP and wrote neither color nor its required Z. Give AlphaTestResult an independent writeDepth field so each AFAIL case states its own color and depth disposition, and gate the two WriteVram calls in writePixel on writeFramebuffer and writeDepth separately instead of bailing out on a single flag. Each case now matches the table: FB_ONLY and RGB_ONLY stop writing Z on alpha-fail, and ZB_ONLY gains its previously-missing depth-only write. A passing fragment still writes both. The depth test itself is unchanged; a fragment that fails it still writes nothing. Add depth-behavior pins for all three affected AFAIL modes, each with ZMSK=0, ZTST=ALWAYS, and a Z page distinct from the color page so the depth path is actually exercised: ZB_ONLY asserts depth is written on alpha-fail, and FB_ONLY and RGB_ONLY siblings assert depth is left untouched. The existing FB_ONLY and RGB_ONLY framebuffer tests are unchanged.
smmathews
marked this pull request as ready for review
July 27, 2026 10:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(gs): correct depth write for all AFAIL alpha-fail modes
Corrected premise
This PR fixes a defect in the alpha-test-fail (AFAIL) truth table that sits next to the depth logic.
The bug
TEST.AFAIL documents four outcomes for a fragment that fails the alpha
test:
classifyAlphaTestinps2xRuntime/src/lib/ps2_gs_rasterizer.cppused asingle
writeFramebufferflag to stand in for both the color and depthdisposition, and
writePixelbailed out early only when that flag wasfalse. Every AFAIL case that survived the early-out then fell through to
an unconditional depth write (gated only by ZMSK). That made every
non-KEEP case except ZB_ONLY write depth on alpha-fail, against the table:
FB_ONLY and RGB_ONLY wrote Z when the table says they should not, and
ZB_ONLY (folded into the same branch as KEEP) wrote neither color nor its
required Z.
The fix
AlphaTestResultgains an independentwriteDepthfield so each AFAILcase states its own color and depth disposition, instead of a single
writeFramebufferflag standing in for both.writePixelgates the colorWriteVramcall onwriteFramebufferand the depthWriteVramcall onwriteDepthseparately. The Z test itself is unchanged: a fragment thatfails the depth test still writes nothing, matching a passing fragment's
depth-test behavior.
A normal alpha-test pass still writes both color and depth. KEEP is
unchanged (it hits the early-out with both flags false, same as before).
FB_ONLY and RGB_ONLY now stop writing depth on alpha-fail, and ZB_ONLY
gains its previously-missing depth-only write.
Test
Three regression tests sit next to the existing AFAIL FB_ONLY and RGB_ONLY
framebuffer pins, each programming ZMSK=0 with a Z page distinct from the
color page and a pre-seeded depth sentinel so the depth path is actually
exercised:
the framebuffer stays untouched while depth is overwritten with the
drawn fragment's z.
the drawn color while depth stays at its pre-seeded sentinel.
preserved while depth stays at its pre-seeded sentinel.
Repro
From the repository root: