Skip to content

Add Linear Output export — dump decoded source data (Pakon, DNG, camera RAW) as untouched 16-bit TIFF - #722

Merged
marcinz606 merged 4 commits into
marcinz606:mainfrom
thetalkingdrum:feature/linear-output
Aug 1, 2026
Merged

Add Linear Output export — dump decoded source data (Pakon, DNG, camera RAW) as untouched 16-bit TIFF#722
marcinz606 merged 4 commits into
marcinz606:mainfrom
thetalkingdrum:feature/linear-output

Conversation

@thetalkingdrum

@thetalkingdrum thetalkingdrum commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

What

Adds a new export option, "Linear Output," that writes whatever was decoded from a loaded source file as a plain 16-bit TIFF — before any pipeline processing (Normalization, Exposure/Print, Lab, Toning, Finish), no color management, no white balance baked into the pixel data. Inspired by tools like RawDigger, MakeTiff, and RawTherapee's "Save Reference Image" — all of which export a similar linear/unrendered TIFF for sensor calibration, device profiling, or diagnostic purposes.

Supports all three loader types: Pakon proprietary RAW, DNG LinearRaw (SilverFast/VueScan), and camera RAW.

Why

Characterization/profiling work — building an ICC input profile or otherwise measuring a scanner or camera's actual response requires genuinely unprocessed device data, not a normal export, which has already been through white balance, and NegPy's own tone shaping by the time it leaves the app.

Format conversion and archival. Pakon's proprietary .raw format is a headerless binary blob that almost nothing outside NegPy and a few other apps can open. Converting to a standard 16-bit TIFF means the data is readable by any image tool, archivable long-term, and not dependent on a single application's bespoke loader. The same applies to LinearRaw DNG variants — while DNG is a published format, many tools don't handle the LinearRaw IFD, so a plain TIFF is more universally accessible. (Longer term, I'm hoping to add support for a few other exotic scanner formats too — see "Out of scope" below for what's already on the radar.)

RGB triplet/stitch consolidation (planned follow-up, not in this PR). Narrowband/RGB-scan captures currently exist as separate per-band source files — three for a single triplet, and a multiple of three for any stitched multi-part capture. This scales fast: a 4-part stitch is already 12 raw files behind one finished image. The RGB-scan stitch support that just landed (stitch_triplets, one (green, blue) pair per part) makes it practical to collapse all of those into a single combined linear TIFF per finished composite, regardless of how many raw parts went into it. Not included in this first version; planned as a follow-up now that the underlying stitch/merge mechanism exists.

White balance is recorded, not applied. Camera RAW exports write the camera's as-shot WB gains into XMP metadata (MakeTiff-compatible, see Metadata below) and into the human-readable ImageDescription tag (e.g. no WB applied (as-shot: 1.318 1.000 2.891)), rather than baking them into the pixels. This is deliberately safer — once WB gains are multiplied in and rounded to 16-bit integers, any channel that clips can't be perfectly recovered. Recording the gains and letting a downstream tool apply them is exact; reversing an already-applied WB is not.

No per-frame content-dependent scaling. The expansion factor (a linear-domain exposure shift for sources whose ADC precision doesn't fill the 16-bit container) is never a function of an individual frame's own content. This constraint is directly informed by #718 — the exact same category of bug (LibRaw's adjust_maximum_thr silently rescaling by a frame's own brightest pixel, breaking the assumption that a roll/batch shares one consistent scale) would have been trivial to reintroduce here via naive auto-detection, so it was deliberately avoided:

  • Pakon: spec-derived from PAKON_SPECS. F135 (confirmed 14-bit) defaults to ×4 — the standard 2-bit shift factor (2¹⁶/2¹⁴), which also fixes a usability issue where MakeTiff's ×2 leaves thumbnails too dark to distinguish frames. F335 (claimed 16-bit, unverified — no sample file available) defaults to ×1/off to avoid clipping. 2k Square and Panoram default to ×4 (F135's factor) rather than F335's ×1 — these are believed to be F135-family hardware variants (same 14-bit ADC), and F335 units are rare enough, with uncertain panorama capability in the first place, that F135-based is the far more likely source for any real file in either of these specs. ×1/×2/×4 manual override remains available for the rare case this assumption is wrong.
  • DNG (SilverFast/VueScan): manual-only (×1/×2/×4, default off). No reliable per-scanner bit depth is known, so no smart default is attempted.
  • Camera RAW: no expansion — rawpy's own black/white-level stretch already handles headroom using the camera's real levels.

Independent decode path. Camera RAW decode uses its own rawpy.postprocess() call — it does not reuse _decode_sensor_rgb. The rawpy parameters are equivalent to the shared function's linear_raw=True mode; the reason for a standalone path is that linear output needs to capture WB gains and timestamp from the raw object before closing it, which _decode_sensor_rgb doesn't return — and threading those extras through its return type and all its callers wasn't worth it for a single consumer. adjust_maximum_thr=0.0 is set on both paths independently.

IR channel handling

DNG LinearRaw sources with a 4th (IR) sample get split into two output files: RGB as the primary TIFF, IR as a separate grayscale TIFF with an _ir suffix. Not folded as a 4th sample since most TIFF viewers assume a 4th channel is alpha. Pakon and camera RAW never carry IR.

Metadata

  • ColorPerfect/MakeTiff compatibility: as-shot WB gains written to XMP dc:description in MakeTiff's exact format (RAW-WB: R G B, green-normalized to 1.0), source filename in crs:RawFileName. This field is reserved exclusively for that format — nothing else is appended to it.
  • ImageDescription: human-readable summary — source loader/format, expansion factor, and confirmation that no WB and no color management were applied. For camera RAW, also includes the as-shot WB gains in plain text (e.g. no WB applied (as-shot: 1.318 1.000 2.891)).
  • Pakon Make/Model: TIFF tags 271/272 populated from the matched PAKON_SPECS entry (e.g. Make="Pakon", Model="F135 Plus High Res"), since Pakon's format carries no embedded device metadata.
  • Source device metadata (Make, Model, DateTime) carried through from the source file when available. For non-TIFF RAW formats (Fuji RAF, Olympus ORF, etc.) that aren't directly readable as TIFF, Make/Model is extracted from the file's embedded EXIF block.

Out of scope (explicitly deferred)

  • RGB triplet/stitch merge into a single Linear Output file — planned follow-up; the underlying stitch mechanism (stitch_triplets) is now merged, this PR doesn't yet build on it.
  • Batch-aware auto-detection for DNG expansion — only worth building if manual-only proves painful in practice.
  • JXL export — NegPy's JXL color table doesn't have a good way to represent untagged linear data; 16-bit TIFF only for now.
  • Noritsu raw, Imacon .fff, Coolscan NEF loader support — no sample data available yet for the first two; these would be new NegPy loaders entirely, independent of Linear Output, and not part of this PR's scope.

Risk

New export option only — doesn't touch the existing render/export/ICC pipeline. No risk to existing saved edits or normal exports.

Testing

tests/test_linear_output.py — 52 tests:

  • Pakon spec-branching: F135 gets ×4, F335 gets ×1 (test_f335_detected_by_size, test_f135_gets_4x, test_source_type_f335/f135)
  • Synthetic 4-channel LinearRaw DNG: RGB roundtrip, IR split to separate grayscale TIFF, pixel value accuracy (test_4ch_dng_*)
  • 3-channel DNG (SilverFast): roundtrip, no IR file produced (test_3ch_dng_*)
  • Camera RAW decode settings verified via WB tag format (test_build_xmp_maketiff_format, test_normalize_wb_rgb)
  • Metadata: expansion factor in ImageDescription, Pakon Make/Model tags, source device tags
  • Geometry: rotation, flip, orientation, fine-rotation-ignored (7 tests)
  • Basic roundtrip, no ICC profile, output directory creation, unsupported file rejection

docs/USER_GUIDE.md and docs/PIPELINE.md updated.

Mats and others added 4 commits August 1, 2026 12:03
New export intent (Print / Flat / Linear) that decodes the source file
and writes an untagged linear 16-bit TIFF, skipping the entire darkroom
pipeline. Supports Pakon RAW (4× expansion), LinearRaw DNG (3ch
SilverFast + 4ch VueScan with IR), and camera RAW (unity WB, sensor-
native colour). Camera WB is embedded in XMP as RAW-WB (MakeTiff-
compatible format). Source metadata (Make, Model, DateTime) is
preserved.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Export All/Selected now passes the file list through to
  request_linear_output_export instead of always exporting only
  the current frame.
- Remove the save-file dialog; write directly to the export folder
  with overwrite-protection naming (_2, _3, …), matching flat/print.
- Add adjust_maximum_thr=0.0 to _decode_camera_raw so LibRaw uses
  the camera's fixed white level instead of per-frame rescaling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Expansion combo box: source-type-dependent options (Pakon F135: 4×/2×/off,
  F335: off/2×/4×, DNG: off/2×/4×, camera RAW: N/A)
- F335 (16-bit sensor) defaults to no expansion; detected by file size
- 2k Square/Panoram assumed 14-bit (unverified, documented)
- ImageDescription records source loader/format, expansion factor, WB status
- Camera RAW exports include as-shot WB in ImageDescription
- Pakon exports populate TIFF Make/Model from PAKON_SPECS
- Camera RAW exports read Make/Model via tifffile for non-DNG formats
- Updated USER_GUIDE.md and PIPELINE.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- EXIF fallback: scan for embedded EXIF TIFF block in non-TIFF RAW
  formats (RAF, ORF, etc.) to extract Make/Model/DateTime
- Camera RAW exports include as-shot WB gains in ImageDescription
  (e.g. "no WB applied (as-shot: 1.318 1.000 2.891)")

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@thetalkingdrum thetalkingdrum changed the title Title: Add Linear Output export — dump decoded source data (Pakon, DNG, camera RAW) as untouched 16-bit TIFF Add Linear Output export — dump decoded source data (Pakon, DNG, camera RAW) as untouched 16-bit TIFF Aug 1, 2026
@marcinz606
marcinz606 merged commit 6410002 into marcinz606:main Aug 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants