Skip to content

feat: stitch frames captured as RGB-scan triplets - #723

Merged
marcinz606 merged 1 commit into
mainfrom
feat/stitch-triplets
Aug 1, 2026
Merged

feat: stitch frames captured as RGB-scan triplets#723
marcinz606 merged 1 commit into
mainfrom
feat/stitch-triplets

Conversation

@marcinz606

Copy link
Copy Markdown
Owner

What

Panorama stitching now works on RGB-scan frames, so a negative captured in overlapping pieces and with narrowband R/G/B illumination assembles into one frame. Each part merges its own triplet first, then the merged parts register and blend as they do today.

No new UI: turn on RGB Scan, select the pieces, right-click → Stitch selected frames.

Why it was blocked

request_stitch_selected refused outright:

if any(f.get("green_path") or f.get("stitch_paths") for f in ordered):
    self.set_status("Stitching RGB-scan or already-stitched frames is not supported", 4000)

The refusal was structural, not arbitrary. StitchConfig carried a single path per part, so _load_source_f32 decoded every part against the same params.rgbscan — each part would have been merged with the primary's green/blue files.

How

StitchConfig gains the per-part triplet, indexed like the existing stitch_transforms / stitch_sizes (primary at 0, unlike stitch_paths):

stitch_triplets: tuple[tuple[str, str], ...] = ()   # (green, blue) per part; ("", "") = plain part
stitch_align: bool = True

Each part is then decoded against its own rgbscan config (_part_params). That one change makes the existing per-part decode tail correct for free — including _decode_oriented_f32's "skip apply_sensor_correction for triplets" branch, which has to be decided per part. An empty tuple means no part is a triplet, so every already-saved composite behaves exactly as before.

Threaded through the places a composite's identity and lifetime live: stitch_token (or the engine cache serves stale renders), the preview loader, PreviewLoadTask, the composite asset dict, session persist/restore, and unstitch — which now hands the parts back as triplet assets instead of nine loose exposures.

Mixed selections (one triplet part, one plain part) fall out working; no guard needed.

StitchWorker is untouched — params_by_path already comes from _batch_params_for, which resolves each part's own triplet, so registration was already decoding merged parts.

Drive-by fix

set_rgb_scan_mode / set_half_frame_mode decomposed only green_path/blue_path when re-running discovery, so toggling either mode dropped a composite's other parts out of the session. Both now go through one _component_paths helper.

Measurements

From the sample capture (Fuji X-T5, X-Trans, two overlapping parts × three exposures):

  • Inter-shot drift R→G→B is 0.02–0.09 px at 1/6 scale (≤0.5 px full-res), near-uniform across quadrants — the existing phase-correlation align in assemble_rgb is plenty, no per-part homography needed.
  • CFA crosstalk is real: under red light the R photosites read ~3312 above black, G ~547, B ~181. Taking the matched demosaiced channel per shot keeps the separation clean.
  • A demosaic-free "treat the sensor as monochrome, gain-equalise the CFA classes" path was tried and dropped: per-class mean residual is tiny (≤0.5 %), but B-pixel noise amplifies ~18× (10 % per-pixel std vs 5.8 %).

Testing

make all green — 2996 passed, 5 skipped, lint + ty clean.

12 new tests in tests/test_stitch.py, including an end-to-end run on real files (skipped when absent) that registers and composites the actual capture and asserts the three channel means diverge — something a red-only merge cannot do.

Checked by hand on the same files: the full-res export path produces an 8101×7776 composite in 21 s, and with matching decode settings its channel means track the preview to three decimals (0.2216 / 0.2485 / 0.2248 vs 0.2213 / 0.2481 / 0.2249).

Docs

USER_GUIDE.md had no stitch section at all — added one covering Stitch selected frames / Unstitch plus the triplet workflow. PIPELINE.md gains a composite-assembly bullet under source corrections, noting that flat-field and unmix run per part and never across the canvas.

Stitching refused any RGB-scan frame. The reason was structural: StitchConfig
carried one path per part, so the decode loop merged every part against the
primary's green/blue exposures.

StitchConfig now carries stitch_triplets — the (green, blue) pair per part,
indexed like stitch_transforms/stitch_sizes with the primary at 0 — plus
stitch_align. Each part is decoded against its own rgbscan config, which makes
the existing per-part tail (triplet merge, flat-field, sensor-unmix skip)
correct without further change. An empty tuple leaves saved composites alone.

The triplet paths are folded into stitch_token, carried on the composite asset
dict, persisted with the session and restored on launch; unstitch hands the
parts back as triplet assets rather than loose exposures.

Also fixes the RGB Scan / Half Frame toggles, which decomposed only
green_path/blue_path and so dropped a composite's other parts from the session
on re-discovery. Both now share one _component_paths helper.
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown

Preview load metrics (github-linux-x64)

format cold load throughput
CR2 0.769 s 29.17 MPx/s
NEF 0.599 s 41.08 MPx/s
ARW 0.214 s 94.37 MPx/s
RAF 0.626 s 25.98 MPx/s
DNG 0.649 s 36.86 MPx/s

fixtures: rawsamples.ch · sha: f4fd607 · full JSON

@marcinz606
marcinz606 merged commit 0353bf2 into main Aug 1, 2026
2 checks passed
@marcinz606 marcinz606 mentioned this pull request Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant