Smartstack: generate preview JPEGs and final e45 products

[timbeccue]

Summary

This is PR 5 of 8 completing the v1 banzai implementation of Smartstacking at site.

PR1 (#469): combine math
PR2 (#470): JPEG utilities
PR3 (#471): DB updates
PR4 (#473): shipper contract
PR5 (#474): smartstack data products      <-- you are here
PR6 (#475): stack worker polling
PR7 (#476): e2e tests, compose cleanup
PR8 (#477): logging

PR1 combine math ───────┐
PR2 JPEG utilities ─────┤
PR3 DB updates ─────────┼──> PR5 data products --> PR6 stack workers --> PR7 E2E --> PR8 logs/docs
PR4 shipper contract ───┘

PRs 1–4 provide the independent foundations for combining images, rendering JPEGs, storing stack state, and shipper integration (this PR). This PR brings them together into the product layer:

ordered reduced e09 stackframes → preview JPEGs or a final e45 FITS product

PR6 will own the lifecycle decisions (stack worker behavior) that call this layer.

Note on merging: this should be merged into main only after PR1-4 are merged. Right now the base is set to simulate that so the diffs are easy to read, but the base should be changed to main before it's actually merged.

What the changes do:

• Opens every persisted stackframe; fails rather than silently building from fewer inputs.
• Sorts members by stack_num.
• Creates a fresh output frame without modifying an input.
• Calls the image stacking code from PR1
• Derives the stack filename from the lowest-numbered available member by replacing e09 with e45.
• Renders 300-pixel and 900-pixel JPEG derivatives using PR2’s utilities.

The final FITS records aggregate and provenance metadata in both the primary and SCI headers, including total EXPTIME, earliest DATE-OBS, NCOMBINE, MOLUID, and ordered IMCOMnnn inputs. It also removes per-exposure MOLFRNUM, scales stack-dependent detector values, and remeasures the combined-image background statistics.

Header changes in stacked products:

While most fits headers are copied over from the stackframes, some are recalculated. I think the procedures make sense, but they're definitely worth a close review.

• EXPTIME: sum of all input exposure times (matches the renormalized-sum flux scale).
• DATE-OBS: earliest input's observation start time.
• UTSTOP: copied from the newest input, so DATE-OBS/UTSTOP bracket the stack's full time span.
• SATURATE: inherited value × N (number of inputs); exact upper bound for trustworthy pixel values on the N-frame flux scale.
• MAXLIN: inherited value × N; same argument for the linearity limit.
• RDNOISE: inherited value × √N; read-noise variances add across a sum.
• L1MEAN / L1MEDIAN / L1SIGMA: remeasured on the stacked SCI using the same Background2D background-map method as SourceDetector, so they're comparable to e09 values.

Preview and final paths

run_preview():

• Builds the current stack in memory.
• Writes JPEGs but no FITS product.
• Publishes through PR4 with fits: null and aggregate thumbnail metadata.

run_final():

• Writes the e45 FITS and both JPEGs.
• Returns the actual written paths.
• Does not publish or change database state; PR6 owns that ordering.

Scope

The PR-local implementation is banzai/smartstack_products.py and its focused tests. The combine, JPEG, database, and publisher layers visible in the cumulative branch are inherited from PR1–4.

This PR builds products but does not poll for work or manage lifecycle state. Tests cover output structure, metadata and provenance, stable naming, invalid inputs, preview publication, and a real FITS write/reopen path.