Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@

## [2.0.0] - Unreleased

### Fluorescence subtraction (1D)

- Add an opt-in absolute-scale fluorescence kernel
(`I_corr = I_abs − β F(q)`) with `constant`, `high_q_mean`,
`high_q_median`, and `measured_profile` methods, a `fluorescence`
correction-ledger token, CLI `subtract-fluorescence`, Workbench Tab 3/Tab 2
1-D hooks, and optional BL19B2 integrate1d post-processing. Default remains
off. Unknown `u(F0)`/`u(β)` keeps combined uncertainty NaN. Detector-space
NIST blank subtraction is unchanged.

### Resource-handle pass (2026-08-17)

- Added `saxsabs.io.detector_images` as the common copy-and-close detector
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ a display server.

| Route | Best for | Start here |
| --- | --- | --- |
| **CLI utilities** | normalization, header and 1D parsing, gated K estimation, gated buffer subtraction | `saxsabs --help` |
| **CLI utilities** | normalization, header and 1D parsing, gated K estimation, gated buffer and fluorescence subtraction | `saxsabs --help` |
| **SAXSAbs Workbench** | interactive K calibration, batch processing, external-1D scaling | `saxsabs-workbench --lang en` |
| **Strict BL19B2 runner** | validated campaign inputs under current BL19B2 conventions | [batch runbook](docs/bl19b2_abs2d_batch_runbook.md) |
| **Python API** | reusable scientific calculations and file I/O | [API reference](docs/api.md) |
Expand Down
611 changes: 608 additions & 3 deletions SASAbs.py

Large diffs are not rendered by default.

16 changes: 16 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,11 @@ subtract_buffer(q_sample, i_sample, err_sample, q_buffer, i_buffer, err_buffer,
*, alpha_uncertainty: float | None = None,
sample_profile: Mapping[str, object],
buffer_profile: Mapping[str, object]) -> BufferSubtractionResult
subtract_fluorescence(q, i_abs, err_abs, *, sample_profile, method,
f0=None, f0_uncertainty=None, beta=1.0, beta_uncertainty=None,
high_q_window=None, q_fluorescence=None, i_fluorescence=None,
err_fluorescence=None, fluorescence_profile=None,
residual_window=None) -> FluorescenceSubtractionResult
propagate_absolute_uncertainty(intensity: np.ndarray, *,
statistical_standard_uncertainty=None, k_relative_standard_uncertainty=None,
standard_relative_standard_uncertainty=None,
Expand All @@ -108,6 +113,12 @@ conflicting evidence remains ambiguous. A unitless metadata label `absolute` is
ambiguous and is not treated as cm$^{-1}$. `subtract_buffer` requires
`sample_profile` and `buffer_profile` provenance, interpolates a buffer onto
the sample q grid when necessary, and propagates supplied uncertainties.
`subtract_fluorescence` is an opt-in additive correction on absolute cm$^{-1}$
data after K, thickness, and optional buffer. Methods are `constant`,
`high_q_mean`, `high_q_median`, and `measured_profile`. A high-q constant is
valid only where elastic SAXS is negligible in that window. Missing `u(F0)` or
`u(β)` keeps combined uncertainty NaN. Negative intensities are reported, not
clipped. This is not detector-dark, NIST-blank, or solvent subtraction.
`propagate_absolute_uncertainty` combines statistical and supplied standard
uncertainty components; relative inputs must be relative standard uncertainties.

Expand Down Expand Up @@ -162,6 +173,10 @@ saxsabs estimate-k --meas <path> [--ref <path>] [--q-col <name>] [--i-col <name>
[--intensity-state relative] [--thickness-cm <cm>]
saxsabs subtract-buffer --sample <path> --buffer <path> [--alpha <value>]
[--alpha-uncertainty <value>]
saxsabs subtract-fluorescence --sample <path> --method <constant|high_q_mean|high_q_median|measured>
[--f0 <cm^-1>] [--f0-uncertainty <value>] [--beta <value>]
[--beta-uncertainty <value>] [--qmin <Å^-1>] [--qmax <Å^-1>]
[--fluorescence <path>]
saxsabs bl19b2-abs2d --input-root <path> (--poni <path>|--pydidas-cali-yaml <path>)
(--mu <cm^-1>|--sample-thickness-cm <cm>)
--monitor-mode <rate|integrated> [workflow options]
Expand All @@ -178,6 +193,7 @@ The main commands are:
| `parse-external1d` | profile path | parsed-profile summary JSON |
| `estimate-k` | relative measured profile; optional reference (built-in SRM 3600 if omitted); optional `--thickness-cm` | K-factor result JSON |
| `subtract-buffer` | absolute sample and buffer profiles with cm⁻¹ units | subtraction diagnostic JSON |
| `subtract-fluorescence` | absolute sample profile; constant, high-q window, or measured F(q) | fluorescence diagnostic JSON |
| `bl19b2-abs2d` | explicit BL19B2 inputs and semantics | batch result JSON and requested files |
| `bl19b2-abs2d-v1-legacy` | explicit migration choices | legacy-compatible batch result with documented assumptions |

Expand Down
32 changes: 22 additions & 10 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
│ saxsabs.core.intensity_state (1D ledger) │
│ saxsabs.core.workbench_preflight_gate │
│ saxsabs.core.buffer_subtraction (BioSAXS) │
│ saxsabs.core.fluorescence_subtraction (1D) │
│ saxsabs.workflows.bl19b2_abs2d / integrate1d │
│ saxsabs.io.parsers (header + 1D I/O) │
│ saxsabs.io.writers (canSAS/NXcanSAS) │
Expand All @@ -31,17 +32,19 @@
- **`src/saxsabs/core`**: pure computation logic (normalization, robust
K-factor estimation, fingerprinted NIST 30 keV material attenuation,
xraydb/Elam diagnostic attenuation, 1D intensity/correction state, Workbench
preflight fingerprints, and buffer subtraction). No GUI side-effects —
preflight fingerprints, buffer subtraction, and optional 1D fluorescence
subtraction). No GUI side-effects —
deterministic and testable.
- **`src/saxsabs/io`**: robust input parsing plus standard-format writers
(canSAS XML and NXcanSAS HDF5).
- **`src/saxsabs/cli.py`**: seven headless subcommands: five focused utilities
- **`src/saxsabs/cli.py`**: eight headless subcommands: six focused utilities
(`norm-factor`, `parse-header`, `parse-external1d`, `estimate-k`,
`subtract-buffer`), the safety-first `bl19b2-abs2d` workflow, and the
explicit `bl19b2-abs2d-v1-legacy` migration entry. `estimate-k` and
`subtract-buffer` apply the intensity-state gates; the first three utilities
remain thin parsers/calculators. The legacy entry requires explicit monitor
and thickness semantics and never silently restores v1 defaults.
`subtract-buffer`, `subtract-fluorescence`), the safety-first `bl19b2-abs2d`
workflow, and the explicit `bl19b2-abs2d-v1-legacy` migration entry.
`estimate-k`, `subtract-buffer`, and `subtract-fluorescence` apply the
intensity-state gates; the first three utilities remain thin
parsers/calculators. The legacy entry requires explicit monitor and
thickness semantics and never silently restores v1 defaults.
- **`src/saxsabs/constants.py`**: pluggable reference-standard registry
(SRM 3600, water, custom curves).
- **`src/saxsabs/workbench_launcher.py`**: packaged launcher used by
Expand Down Expand Up @@ -99,6 +102,14 @@
provenance.
The Workbench calls the shared core `subtract_buffer`; if it is unavailable,
formal subtraction fails closed with no weaker local fallback.
- Fluorescence subtraction is opt-in and absolute-scale only. It is an additive
1-D term `I_corr = I_abs − β F(q)` applied after K, thickness, and optional
buffer. Methods are `constant`, `high_q_mean`, `high_q_median`, and
`measured_profile`. High-q estimation is valid only where elastic SAXS is
negligible in the stated window. The ledger token is `fluorescence`. Unknown
`u(F0)` or `u(β)` keeps combined uncertainty NaN. The kernel never clips
negative intensities. Detector-space NIST blank subtraction is unchanged:
a q-independent 1-D constant must not be written back onto raw counts.
- The NIST 30 keV GUI export is a material-attenuation provenance JSON. It is
invalidated whenever source/energy/preset/composition/density/porosity input
changes. Nominal identity is inferred from the edited composition, not copied
Expand Down Expand Up @@ -139,9 +150,10 @@
estimation, NIST 30 keV material core, Elam diagnostic calculator, 1D
intensity ledger, signed-in-memory Workbench preflight, fixed-thickness
enforcement, disabled legacy/resume controls, exact K-only/Kd/buffer gates,
absolute-buffer validation, provenance-aware scrollable μ UI, disabled Tab 3
raw mode, screen-aware startup, strict BL19B2 workflows, standard writers,
bilingual GUI, CLI, CI, and paper assets. K-only formal scaling requires both
absolute-buffer validation, optional absolute 1D fluorescence subtraction,
provenance-aware scrollable μ UI, disabled Tab 3 raw mode, screen-aware
startup, strict BL19B2 workflows, standard writers, bilingual GUI, CLI, CI,
and paper assets. K-only formal scaling requires both
the inherited-thickness ledger entry and numeric/source provenance.
- **Strict campaign ownership**: formal multi-folder and per-sample campaigns
remain owned by the strict CLI/batch runner. The Workbench is an interactive
Expand Down
8 changes: 8 additions & 0 deletions docs/reviewer-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@ entry point. Reusable scientific and I/O logic lives under `src/saxsabs/`; the
GUI remains separate because the current Workbench and strict BL19B2 campaign
runner have intentionally different ownership boundaries.

## Is fluorescence the same as empty-cell or buffer subtraction?

No. Empty-cell / NIST-blank subtraction happens in detector space before
absolute scaling. Buffer subtraction removes solvent on the absolute
cm$^{-1}$ scale. Fluorescence subtraction is an optional additive 1-D term
applied after K, thickness, and optional buffer. It is off by default, does
not change $K$, and is not written onto 2D raw counts.

## How can this be tested without GUI?

Core logic is exposed as importable APIs and CLI commands. Tests run headlessly in CI.
Expand Down
11 changes: 10 additions & 1 deletion examples/manual-verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,16 @@ front end to the strict BL19B2 campaign runner.
Negative, non-finite, or malformed values must fail closed. Temporarily make
the shared core kernel unavailable and confirm formal subtraction fails closed
rather than using a weaker fallback.
9. Before packaging the repository, confirm `git status` contains no audit
9. Enable fluorescence subtraction on an absolute Tab 3 (or Tab 2 1-D) result.
Confirm it is refused on relative/raw profiles, refused if `fluorescence` is
already in `corrections_applied`, and applied only after K/thickness and
optional buffer. For `constant`, set a planted F0 and confirm
`I_corr = I_abs − F0`. Leave `u(F0)` blank and confirm combined uncertainty
is NaN; set a finite value and confirm it enters the combined column. For
`high_q_mean`, a window with fewer than 3 finite points must fail closed.
Confirm `saxsabs subtract-fluorescence --help` lists the same methods, and
that making the shared kernel unavailable fails closed with no GUI fallback.
10. Before packaging the repository, confirm `git status` contains no audit
outputs, build caches, downloaded literature, or private drive roots. Keep
manual evidence outside the repository and use only anonymized, portable
fixtures for version-controlled examples and tests.
Expand Down
12 changes: 12 additions & 0 deletions src/saxsabs/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,16 @@
IntensityStateAssessment,
assess_intensity_state,
require_absolute_input_for_buffer_subtraction,
require_absolute_input_for_fluorescence_subtraction,
require_relative_input_for_absolute_scaling,
)
from .core.buffer_subtraction import BufferSubtractionResult, subtract_buffer
from .core.fluorescence_subtraction import (
FluorescenceMethod,
FluorescenceSubtractionResult,
parse_fluorescence_method,
subtract_fluorescence,
)
from .core.preflight import evaluate_preflight_gate, PreflightGateSummary
from .core.execution_policy import (
RunPolicy,
Expand Down Expand Up @@ -113,10 +120,15 @@
"IntensityStateAssessment",
"assess_intensity_state",
"require_absolute_input_for_buffer_subtraction",
"require_absolute_input_for_fluorescence_subtraction",
"require_relative_input_for_absolute_scaling",
# buffer subtraction
"BufferSubtractionResult",
"subtract_buffer",
"FluorescenceMethod",
"FluorescenceSubtractionResult",
"parse_fluorescence_method",
"subtract_fluorescence",
# I/O
"parse_header_values",
"parse_header_values_with_meta",
Expand Down
79 changes: 78 additions & 1 deletion src/saxsabs/cli.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""Command-line interface for headless SAXS calibration operations.

Provides seven subcommands: five small utilities plus the safe BL19B2 workflow
Provides eight subcommands: six small utilities plus the safe BL19B2 workflow
and its explicit v1 migration entry.
"""

Expand All @@ -16,6 +16,7 @@

from . import __version__
from .core.buffer_subtraction import subtract_buffer
from .core.fluorescence_subtraction import subtract_fluorescence
from .core.calibration import estimate_k_factor_robust
from .core.intensity_state import require_relative_input_for_absolute_scaling
from .core.normalization import compute_norm_factor
Expand Down Expand Up @@ -384,6 +385,30 @@ def build_parser() -> argparse.ArgumentParser:
p_sub.add_argument("--alpha", type=float, default=1.0)
p_sub.add_argument("--alpha-uncertainty", type=float, default=None)

p_fluo = sub.add_parser(
"subtract-fluorescence",
help="Subtract additive fluorescence from an absolute sample profile",
)
p_fluo.add_argument("--sample", required=True, type=Path)
p_fluo.add_argument(
"--method",
required=True,
choices=["constant", "high_q_mean", "high_q_median", "measured"],
help="How F(q) is obtained",
)
p_fluo.add_argument("--f0", type=float, default=None, help="Constant F0 in cm^-1")
p_fluo.add_argument("--f0-uncertainty", type=float, default=None)
p_fluo.add_argument("--beta", type=float, default=1.0)
p_fluo.add_argument("--beta-uncertainty", type=float, default=None)
p_fluo.add_argument("--qmin", type=float, default=None, help="High-q window minimum")
p_fluo.add_argument("--qmax", type=float, default=None, help="High-q window maximum")
p_fluo.add_argument(
"--fluorescence",
type=Path,
default=None,
help="Measured additive fluorescence profile",
)

p_bl = sub.add_parser(
"bl19b2-abs2d",
help="Process BL19B2 data with explicit monitor and thickness semantics",
Expand Down Expand Up @@ -544,6 +569,58 @@ def main() -> None:
)
return

if args.command == "subtract-fluorescence":
try:
sample = read_external_1d_profile(args.sample)
high_q_window = None
if args.qmin is not None or args.qmax is not None:
if args.qmin is None or args.qmax is None:
raise ValueError("--qmin and --qmax must be provided together")
high_q_window = (args.qmin, args.qmax)
q_fluo = i_fluo = err_fluo = None
fluo_profile = None
if args.fluorescence is not None:
fluo_profile = read_external_1d_profile(args.fluorescence)
q_fluo = fluo_profile["x"]
i_fluo = profile_intensity(fluo_profile)
err_fluo = profile_uncertainty(fluo_profile)
result = subtract_fluorescence(
sample["x"],
profile_intensity(sample),
profile_uncertainty(sample),
sample_profile=sample,
method=args.method,
f0=args.f0,
f0_uncertainty=args.f0_uncertainty,
beta=args.beta,
beta_uncertainty=args.beta_uncertainty,
high_q_window=high_q_window,
q_fluorescence=q_fluo,
i_fluorescence=i_fluo,
err_fluorescence=err_fluo,
fluorescence_profile=fluo_profile,
)
except ValueError as exc:
_die(f"subtract-fluorescence failed: {exc}")
print(
json.dumps(
{
"points": int(result.q.size),
"method": result.method,
"beta": result.beta,
"beta_uncertainty": result.beta_uncertainty,
"f0": result.f0,
"f0_uncertainty": result.f0_uncertainty,
"high_q_residual_mean": result.high_q_residual_mean,
"high_q_check_passed": result.high_q_check_passed,
"high_q_points": result.high_q_points,
"negative_fraction": result.negative_fraction,
},
ensure_ascii=False,
)
)
return

if args.command in {"bl19b2-abs2d", "bl19b2-abs2d-v1-legacy"}:
from .workflows.bl19b2_abs2d import BL19B2Abs2DConfig, run_bl19b2_abs2d

Expand Down
14 changes: 14 additions & 0 deletions src/saxsabs/core/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,18 @@
assess_intensity_state,
parse_correction_ledger,
require_absolute_input_for_buffer_subtraction,
require_absolute_input_for_fluorescence_subtraction,
require_relative_input_for_absolute_scaling,
serialize_correction_ledger,
)
from .buffer_subtraction import BufferSubtractionResult, subtract_buffer
from .fluorescence_subtraction import (
FluorescenceMethod,
FluorescenceSubtractionResult,
parse_fluorescence_method,
subtract_fluorescence,
combine_sequential_standard_uncertainties,
)
from .execution_policy import RunPolicy, parse_run_policy, should_skip_all_existing
from .preflight import PreflightGateSummary, evaluate_preflight_gate
from .reference_matching import (
Expand Down Expand Up @@ -108,10 +116,16 @@
"assess_intensity_state",
"parse_correction_ledger",
"require_absolute_input_for_buffer_subtraction",
"require_absolute_input_for_fluorescence_subtraction",
"require_relative_input_for_absolute_scaling",
"serialize_correction_ledger",
"BufferSubtractionResult",
"subtract_buffer",
"FluorescenceMethod",
"FluorescenceSubtractionResult",
"parse_fluorescence_method",
"subtract_fluorescence",
"combine_sequential_standard_uncertainties",
"RunPolicy",
"parse_run_policy",
"should_skip_all_existing",
Expand Down
Loading