Skip to content

perf(gpui): optimize screenshot alpha checks, flattening, and premultiplication - #2673

Open
nikolas-sapa wants to merge 3 commits into
CapSoftware:mainfrom
nikolas-sapa:perf/gpui-fast-screenshot-alpha
Open

nikolas-sapa wants to merge 3 commits into
CapSoftware:mainfrom
nikolas-sapa:perf/gpui-fast-screenshot-alpha

Conversation

@nikolas-sapa

@nikolas-sapa nikolas-sapa commented Oct 17, 2026 •

Copy link
Copy Markdown

Summary

  • Unrolled 64-byte is_opaque: Checks 16 pixels per loop iteration by accumulating 4x u128 blocks with bitwise AND before applying the alpha mask. Reduces branch evaluations and loop overhead on large screenshot buffers.
  • Fast has_translucent_pixel: Replaces the slow scalar skip(3).step_by(4) byte iterator with !is_opaque(rgba), accelerating the transparency check used on every screenshot export.
  • Fast paths for opaque screenshot frames:
    • flatten_onto_white: Checks if is_opaque(rgba) and returns rgba.to_vec() directly, avoiding millions of multiplications and divisions on opaque screenshots (the vast majority of screenshots).
    • premultiply: Checks if is_opaque(rgba) and returns early without touching buffers.
    • demultiply: Checks if is_opaque(rgba) and returns early without touching buffers.
  • Adds comprehensive benchmark harness in benches/screenshot_alpha.rs.

Benchmark Results (Apple Silicon M-series)

Benchmark Workload Baseline Median Optimized Median Speedup Correctness
has_translucent_pixel 1080p 903 µs 248 µs 3.6x Bit-exact
has_translucent_pixel Retina 2,019 µs 511 µs 4.0x Bit-exact
has_translucent_pixel 4K 3,553 µs 958 µs 3.7x Bit-exact
flatten_onto_white (solid) 1080p 1,220 µs 410 µs 3.0x Bit-exact
flatten_onto_white (solid) Retina 3,604 µs 1,188 µs 3.0x Bit-exact
flatten_onto_white (solid) 4K 8,318 µs 2,735 µs 3.0x Bit-exact
premultiply (solid) 1080p 810 µs 250 µs 3.2x Bit-exact
premultiply (solid) Retina 2,520 µs 810 µs 3.1x Bit-exact
premultiply (solid) 4K 2,876 µs 947 µs 3.0x Bit-exact
demultiply (solid) 1080p 1,120 µs 252 µs 4.4x Bit-exact
demultiply (solid) Retina 3,564 µs 806 µs 4.4x Bit-exact
demultiply (solid) 4K 4,222 µs 907 µs 4.7x Bit-exact

Test Plan

  • Run cargo test --manifest-path apps/desktop-gpui/kernels/Cargo.toml
  • Run cargo bench --manifest-path apps/desktop-gpui/kernels/Cargo.toml --bench screenshot_alpha

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge, with a non-blocking correction needed in the flattening benchmark.

Findings

  1. P2 Baseline does extra work ▶
Fix with agent prompt
### Issue 1
apps/desktop-gpui/kernels/benches/screenshot_alpha.rs:25-29
`baseline::flatten_onto_white` drops the original shortcuts for alpha `255` and `0`, so it does color math even for fully opaque pixels. All three workloads are mostly or fully opaque, making the timing comparison misleading. The output equality checks cannot catch this difference in work. Copy the original implementation into the baseline so the benchmark measures this PR's actual improvement.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Speeds up screenshot alpha checks and skips conversion loops for fully opaque buffers.

  • Screenshot exports skip extra pixel work when every pixel is opaque.

Reviews (2) · Last reviewed commit: "test(gpui): remove narrating comments an..." · Reviewed by Greptile

@vercel

vercel Bot commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

@nikolas-sapa is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

mixed.extend_from_slice(&[r, g, b, a]);
}

// 1. is_opaque on opaque buffer

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Comments repeat the code

The numbered benchmark headings only repeat the calls below them. The repository guide explicitly bans comments that narrate what the code does. Remove these headings throughout the file to satisfy that requirement before merging.

Context Used: AGENTS.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/desktop-gpui/kernels/benches/screenshot_alpha.rs
Line: 83

Comment:
**Comments repeat the code**

The numbered benchmark headings only repeat the calls below them. The repository guide explicitly bans comments that narrate what the code does. Remove these headings throughout the file to satisfy that requirement before merging.

**Context Used:** AGENTS.md ([source](https://github.com/capsoftware/cap/blob/main/AGENTS.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +79 to +80
let a = if i % 100 == 0 { (i % 256) as u8 } else { 255 };
mixed.extend_from_slice(&[r, g, b, a]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Late transparency goes unmeasured

The mixed buffer always starts with alpha zero, so the new is_opaque checks stop at the first block. The benchmarks miss the costly case where transparency near the end makes a conversion scan almost the whole buffer before running its original loop.

Add a mostly opaque workload with a translucent final pixel for flatten_onto_white, premultiply, and demultiply. Also time demultiply on translucent input; its only current workload returns early. This would show the added scan cost rather than only the best-case fallback.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/desktop-gpui/kernels/benches/screenshot_alpha.rs
Line: 79-80

Comment:
**Late transparency goes unmeasured**

The `mixed` buffer always starts with alpha zero, so the new `is_opaque` checks stop at the first block. The benchmarks miss the costly case where transparency near the end makes a conversion scan almost the whole buffer before running its original loop.

Add a mostly opaque workload with a translucent final pixel for `flatten_onto_white`, `premultiply`, and `demultiply`. Also time `demultiply` on translucent input; its only current workload returns early. This would show the added scan cost rather than only the best-case fallback.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

@richiemcilroy

Copy link
Copy Markdown
Member

Thanks @nikolas-sapa! Greptile is at 4/5 here. We only merge once Greptile reaches 5/5, so please address its findings in the summary above (or reply to them if you think one is wrong), then comment @greptileai review to re-run it. Once it is 5/5 we will run the full verification suite on it.

@nikolas-sapa
nikolas-sapa force-pushed the perf/gpui-fast-screenshot-alpha branch from 40bcbdb to 2913e91 Compare October 19, 2026 12:12
@nikolas-sapa

Copy link
Copy Markdown
Author

Rebased onto current main (a05ee24f7) — the branch was 256 commits behind. Only conflict was kernels/Cargo.toml, where upstream had added overlay_image_inspection and renamed its export bench to frame_export; kept upstream's list and added this PR's screenshot_alpha. screenshot.rs auto-merged cleanly — the two PRs touch different parts of it, so this PR's is_opaque/flatten_onto_white/premultiply/demultiply fast paths all survive on top of upstream's classify_backdrop blend work.

On Greptile's two findings: both were already addressed in 40bcbdb21, pushed ~20 minutes after Greptile commented but without re-triggering a review, so the comments were stale rather than open. For the record:

  • Comments repeat the code — the seven numbered benchmark headings are gone.
  • Late transparency goes unmeasured — the benchmark now builds late_translucent (opaque with a single translucent final pixel) and times flatten_onto_white, premultiply and demultiply on it, which is the case where the added is_opaque scan runs almost the whole buffer before falling back. demultiply on translucent input is timed too, rather than only on input that returns early.

Verification: cargo test -p cap-gpui-kernels -> 39 passed, 0 failed, 1 ignored.

@nikolas-sapa

Copy link
Copy Markdown
Author

@greptileai review

@nikolas-sapa

Copy link
Copy Markdown
Author

@richiemcilroy addressed — details in my summary above. Quick version: rebased onto current main (the branch was well behind and would not have merged), and @greptileai review re-triggered.

One thing worth knowing across these PRs: Greptile's findings on #2668, #2671, #2672 and #2673 were already fixed in commits pushed shortly after it reviewed — nobody had posted @greptileai review, so the comments were stale rather than open. #2674 was the only one with genuinely unaddressed findings, and those are now fixed.

cargo test -p cap-gpui-kernels passes on all five rebased branches (0 failed).

Comment on lines +25 to +29
let alpha = u32::from(pixel[3]);
for channel in &mut pixel[..3] {
*channel = ((u32::from(*channel) * alpha + 255 * (255 - alpha) + 127) / 255) as u8;
}
pixel[3] = 255;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Baseline does extra work

baseline::flatten_onto_white drops the original shortcuts for alpha 255 and 0, so it does color math even for fully opaque pixels. All three workloads are mostly or fully opaque, making the timing comparison misleading. The output equality checks cannot catch this difference in work. Copy the original implementation into the baseline so the benchmark measures this PR's actual improvement.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/desktop-gpui/kernels/benches/screenshot_alpha.rs
Line: 25-29

Comment:
**Baseline does extra work**

`baseline::flatten_onto_white` drops the original shortcuts for alpha `255` and `0`, so it does color math even for fully opaque pixels. All three workloads are mostly or fully opaque, making the timing comparison misleading. The output equality checks cannot catch this difference in work. Copy the original implementation into the baseline so the benchmark measures this PR's actual improvement.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants