Repository navigation
Lipid RT-order check: assign Confirmed to lipid annotations whose retention times follow the reversed-phase rules - #832
Merged
Conversation
…erministic component The check confirms a lipid annotation when its species-level composition (class, total C, total DB with P- counted as O- plus one, total O) is consistent with the reversed-phase rules among the run's other MS/MS-backed annotations of the class, bracketed on two axes, and not explained as an MS1 artefact of a brighter co-eluting ion (specification v1.0 with the addendum of 2026-10-15). It is a port of the frozen Python reference (lipidrt_final, poset, xpol) and reads no files. - CompMs.MsdialCore.Algorithm.Annotation.LipidRt: candidates from ChromatogramPeakFeature (hook A) and AlignmentSpotProperty (hook B) with the eligibility rule (IsReferenceMatched read from the representative directly; manual annotations, text-library matches and internal standards excluded); species composition via LipidomicsConverter VS2 on the species half, with the reference grammar for chain roles, sterol cores and the ester scale; the Mammalian profile with the chain rules B1-B3 only (no lipid-class bias); steps 1-10 (artefact layer, features, RP gate, exact maximum consistent subset by integer max-flow with warm-started margin flows, adjacent support and ties, class gate, magnitude, outcomes, propagation); step 11 as a pure function over two runs; evidence records, the evidence sentence and the review list. - LipidRtCheckParameter (MessagePack, for ParameterBase Key 18, not wired yet): ClassLevelPolicy defaults to Corroborated, PlausibilityProfile to None, IsEnabled null meaning on for lipidomics projects. - LipidRtVerdict, the persisted record for the integration step. Tests: the 16 synthetic tests of the reference, policies and operating points, parser and profile cases, the max-flow against brute force, determinism, and a parity suite gated on MSDIAL_LIPIDRT_REFERENCE_DIR. On the 2026-10-15 demo the C# reproduces the reference object by object for ClassLevelPolicy Consistent and Corroborated (single run, profile off and Mammalian, and after cross-polarity), and every outcome of the reference's DEV and TEST evaluation runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Integrates the lipid retention-time order check (specification v1.0 with the addendum of 2026-10-15, sections 9-11 and B1-B3) into MS-DIAL: tag provenance in the tag files, one tag protocol for every caller, the persisted verdict, the parameter, hooks A and B, and the Console and mzTab-M outputs. - Tag provenance (B1). PeakSpotTagCollection keeps an optional source per selected tag and a set of suppressed tags. Select/Deselect keep their meaning (a person's path); Select(tag, source) is a tool's; SelectByPerson and DeselectByPerson are what the GUI toggles now call, so turning off a tool's tag suppresses it and turning a tag on makes it the person's. The tag files (.pai2 peaks, alignment, spectrum features) are written and read by one helper, PeakSpotTagXml: a tool's tag gets a Source attribute, a suppression is a <Suppressed> element, and a peak with only suppressions is written. A tag without a source is written exactly as before, a file from before reads as tags with no source, and the previous reader reads a new file as the same selected tags (tested against that reader, copied verbatim). - Tag protocol (SPEC 9 as re-expressed in B1). LipidRtTagProtocol withdraws only the check's own Confirmed (source "MsdialLipidRtCheck/", any rule version), records each object's verdict (null where there is none), and selects Confirmed with source "MsdialLipidRtCheck/1.0" only where the outcome is Confirmed, Confirmed is not already selected, Misannotation is not selected and Confirmed is not suppressed. It never writes Misannotation. - Verdict record (B2). LipidRtVerdict on ChromatogramPeakFeature (key 54, the next after 53) and AlignmentSpotProperty (key 66, after the post-curation keys). Objects written before either key load with no verdict. - Parameter. LipidRtCheckParameter is ParameterBase key 18 (DIMS uses 20, the other modes 150 and up); a project without it gets the defaults. RunsFor says the check runs only for lipidomics. The LC-MS parameter text gets a "# Lipid RT check" section with the method-file keys, the rule version and the parameter hash. - Hooks. Hook A in the LC-MS FileProcess between annotation and saving, per file and stateless; hook B in PeakAligner right after PackingSpots, LC-MS only, and in the Console's alignment light mode, where the verdicts reach only the exports. Both run only when the check is enabled and the target omics is lipidomics; the RP gate decides inside. When the check does not run or fails, its own earlier tags and verdicts are withdrawn and the failure is reported, so nothing stale or half-written is left. - Console. The method-file keys "Lipid RT check" (Auto/True/False), "... operating point", "... chromatography", "... plausibility profile", "... class-level matches", "... minor duplicates" and "... secondary-only features" are read by the LC-MS reader through LcmsConsoleLines, so the key record knows them and every other mode reports them as unrecognised; values that are not one of the choices are recorded as unusable. The LC-MS template writes them with their defaults. The Console writes <alignment stem>_lipidrt_review.tsv next to the alignment exports whenever the check ran, and a summary line per file and for the alignment. - mzTab-M. opt_global_lipid_rt_outcome, _reason and _score after opt_global_Ontology when any spot has a verdict; "null" where a spot has none. Tags keep appearing as "Tag: Confirmed" in the Comment column. Not in this change (v1.1 or a later step): the GUI "Re-run lipid RT check" action, applying cross-polarity corroboration from the GUI or the Console, and the LC-IM-MS, GC-MS and DIMS hooks. Tests: tag collection, tag-file round trips and compatibility both ways, the protocol and hooks on real peaks and spots (person's tags, another tool's tags, Misannotation, suppression, re-runs, switched off, failure, save and load), MessagePack backward compatibility of keys 54, 66 and 18, the mzTab-M columns, the Comment column, the method-file keys, template and parameter text, the review list file, and the GUI toggles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…through undo
Completes the GUI part of the lipid retention-time order check (specification
v1.0 with the addendum of 2026-10-15, B1 and B3).
- Re-run lipid RT check. A ribbon button on the Data visualization tab, next to
MS-CleanR, opens a small window for the loaded LC-MS alignment result with the
five user-facing settings (operating point, chromatography, plausibility
profile, class-level matches, minor duplicates). Run writes them into the
project's LipidRtCheckParameter, as MS-CleanR does with its own, runs the check
on the result's spots through LipidRtCheckHooks.RunOnSpots and with it the
shared tag protocol, tells the spots' views, and saves the alignment result
(spots with their verdicts and the _tags.xml file) through the alignment
model's SaveAsync. A person's tags, another tool's tags, a Misannotation and a
suppressed Confirmed are never touched. The window is modal and stays open
while the check runs, so no tag is edited by hand meanwhile. On a project that
is not lipidomics the window says why and Run is disabled; a project whose
check was switched off ("Lipid RT check: False") is told that running switches
it back to Auto. A failed save is reported in the window, not thrown.
Cross-polarity corroboration is not applied from the GUI (v1.1).
- Verdict and provenance in view. The peak information panel shows a "Lipid RT
check:" row (the evidence sentence) wherever a peak or spot has a verdict, with
the outcome, reason, score, rule version and parameter hash in its tooltip. The
Confirmed tag checkbox of the peak and spot tables gets a tooltip saying who set
it (the lipid RT check, another tool by its source, or a person) or that a
person removed a tool's tag, and the verdict; with nothing to say the shortcut
legend stays.
- Person path (B1). Every tag toggle of the GUI (checkboxes, Ctrl+1..5, Mark all
as confirmed/unconfirmed) goes through the three models' setters, which already
call SelectByPerson/DeselectByPerson. Undo of "Mark all" now puts each tag back
exactly as it was, source and suppression included (PeakSpotTagState), instead
of toggling back as a person: undoing "Mark all as unconfirmed" used to hand
every Confirmed of the check to the person, and undoing "Mark all as confirmed"
forgot what a person had removed. The three identical nested commands are one
MarkAllAsConfirmedCommand<T> over IPeakSpotTagEditable.
Tests (MsdialGuiAppTests, net48): the setting model on a synthetic TG grid (same
tags and reasons as hook B on the same spots, settings written to the project,
person's tags and decisions kept, a re-run withdrawing only the check's own
tags, not lipidomics, switched off, failed save), the view model, Mark all and
its undo for spots and peaks, PeakSpotTagState, the tooltip texts, the
information panel, and the XAML loaded on a WPF dispatcher (the window's
bindings, the panel row, the Confirmed tooltip in the shared tag template, and
the ribbon icon).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e tracking, and withdraw the check where a person changes the annotation (review) Applies the confirmed findings of the branch review. - Peak verdict key (high). ChromatogramPeakFeature keys 54 and 55 are taken in the MS-DIAL 5.6 line (tags MSDIAL-v5.6.250526-alpha, -250820-alpha, -260323-beta and seven branches: RawDataIDType and AccumulatedDataIDType, a SpectrumIDType int enum). The verdict at 54 made 5.6 peak files unreadable here, made every peak file written here unreadable by 5.6, and would collide on a merge. The verdict moves to key 56, the next free one on every tag and remote branch (AlignmentSpotProperty 66 and ParameterBase 18 are free everywhere). Keys 54 and 55 are now kept as plain integers, ReservedRawDataIDType and ReservedAccumulatedDataIDType: a 5.6 file keeps its values when read and saved here, and a file written here holds 0 (Index, which is what this line's raw spectrum IDs are) where a 5.6 reader reads its enum. Without them a 5.6 reader would meet the nil MessagePack writes for a key with no member and fail, so moving the key alone would not have restored that direction. - Hook B and isotope tracking (medium). PeakAligner ran the check before IsotopeTracking.SetIsotopeTrackingID, which clears the annotation of every spot represented by a labelled file, so such spots were saved and exported as unknowns carrying the check's Confirmed and a Confirmed verdict about a lipid they no longer were. The hook now runs after every step that changes annotations and reads the spot collection the tracking replaces. - Manual annotation changes (medium). Compound search, MS-FINDER and "Set unknown" (SetAnnotationUsecase, SetUnknownDoCommand) now withdraw the check's own Confirmed and the verdict on the LC-MS peak or spot (addendum A6); a person's or another tool's Confirmed stays. Undo of "Set unknown" puts both back. ILipidRtCheckedObject carries this for AlignmentSpotPropertyModel and ChromatogramPeakFeatureModel. - Evidence (low). The supporting neighbours kept in the evidence record are still listed by |gap| then direction, but the strongest lower and upper bound and the strongest of each direction are kept before the rest, so a Confirmed's sentence always shows both sides of its bracket (the review counted 48 of 1,274 demo Confirmed showing one side only). Only this free text differs from the Python reference; outcomes and scores do not. - "Mark all as confirmed" (low). Rows whose Confirmed a tool had set are now taken over by the person (SelectByPerson drops the source in place), so a later run of the check no longer withdraws what the person confirmed. Undo gives them back to the check. - Re-run lipid RT check (low). The run clears the alignment's undo history (new UndoManager.Clear), so undoing an earlier "Mark all" can no longer put back a Confirmed credited to the check against its new verdict. The save after the run goes off the UI thread, and the window says it is saving meanwhile. - GUI processing (low). A failure of the check, on an analysis file (hook A, now recorded by FileProcess.LipidRtFailures) or on the alignment (hook B), is told to the person once after processing; before, it went only to a console the GUI does not show. - Window text (low). The re-run window promised that a Confirmed a person removed is never set again. Under addendum B1 a suppression records the removal of a tool's tag only, so a Confirmed the person turned on again and then removed may be set again; the text now says so. The behaviour is unchanged. Tests: the peak file in both directions with the 5.6 layout (a simulated 5.6 reader, and why nil at key 54 cannot be read there); PeakAligner with and without isotope tracking; the evidence selection, end to end and as a unit; takeover in place; the annotation changes and their undo for spots and peaks; Mark all followed by the check's withdrawal; the undo history after a re-run; the save off a UI-like synchronization context; the processing notice; the window text. Each of these fails with its fix reverted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…publishes it and the selection error on the broker LcmsMethodViewModel no longer constructs LipidRtCheckSettingView, calls MessageBox.Show or ShowDialog, or reads Application.Current.MainWindow. The command is a plain DelegateCommand (the ribbon passes no parameter). With no alignment result selected it publishes an ErrorMessageBoxRequest, whose defaults (caption "Error", OK, error icon) are the box shown before. Otherwise it publishes the LipidRtCheckSettingViewModel, as ExportAlignment does with its view model, and MainWindow shows it with ShowChildDialog: modal and owned by the main window, so the view model is still disposed only after the window has closed and the window still refuses to close while the check runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e view for the error box it showed before The view model now publishes the box on the broker instead of calling MessageBox.Show, so pin what the view receives: the same text, the "Error" caption and the OK button, with no setting window, when the ribbon button (which passes no parameter) is pressed with no alignment result selected. The method view model is built over an empty LC-MS project on an application dispatcher thread, as the view tests do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Follow-up after the lead developer's MVVM review: two commits now remove the ViewModel → View dependency this PR had added to
The Model side of this PR was already free of ViewModel and WPF references. Merge order: #833 first, which fixes the failing GUI test behind this PR's red check, then #834 (lock files). 🤖 Generated with Claude Code |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Within one lipid class on reversed-phase LC:
Curators use these rules by eye to reject false positives, and that manual pass is the bottleneck. It gets worse on very sensitive instruments: on ZenoTOF 8600 data, false positives are numerous even at a 0.5 min LBM2 RT tolerance. It is also worse for outside users, whose LC differs from the library's predicted RTs.
This PR lets MS-DIAL check lipid annotations against the rules and add the existing
PeakSpotTag.CONFIRMEDto the consistent ones:ChromatogramPeakFeature);AlignmentSpotProperty).It never removes or overrides a person's tag.
The check (rule version 1.0)
Species level only. It compares the sum composition (class, total C, total DB, total O; P- counted as O- with one more DB). Fatty-acid composition is not used for RT. A class-level match is eligible, but by default it is Confirmed only when a second adduct or the other polarity corroborates it.
Exact order consistency. Within each class, the check finds the maximum-weight consistent subset of the dominance order, exactly, with an integer max-flow (Dilworth antichain). An annotation is Confirmed only if it belongs to every maximum subset and has adjacent directional support (a +C neighbour and a +DB neighbour bracketing it).
Rejection rules:
MS1 artefact layer. A brighter co-eluting partner that explains the annotation means it is not Confirmed. Covered: isotopes, Na/H, NH4/K, acetate/formate/Cl, demethylation, in-source chain or head losses, oxidised ions.
Propagation. A confirmed feature's other adducts ([M+Na]+ included) are Confirmed with it.
Chromatography gate. A positive carbon slope and order concordance are required. HILIC data get no Confirmed.
Optional chain-rule plausibility profile "Mammalian" (off by default):
It has no lipid-class bias, by design.
Outcomes. Each annotation gets a verdict: Confirmed, NotEvaluable (with a reason code) or Inconsistent (with evidence). Inconsistent is reported only; the check never writes Misannotation.
Cross-polarity corroboration (step 11) is implemented as a pure function with tests. Applying it from the GUI or Console is deferred to v1.1.
Evaluation (Python reference; the C# reproduces it object for object)
The truth sets are human curations. The primary held-out set is the author's own curation of ZT Scan DIA mouse liver, scored only on what was curated. Development used plasma.
Where it runs
FileProcess, after annotation and before save. Console and GUI share it.LipidRtCheckParameteratParameterBaseKey 18. Console method-file keys start withLipid RT check…; they are registered, templated and listed in ParametersAsText.<alignment>_lipidrt_review.tsv;opt_global_lipid_rt_outcome,_reasonand_score;Tag: Confirmedin the Comment column.Tag provenance (who set a tag) and compatibility
Provenance lives in the existing
_tags.xmlfiles, readable by people and external tools:<Tag Source="MsdialLipidRtCheck/1.0">1</Tag>marks a tag set by the check;<Suppressed Source="User">1</Suppressed>records that a person removed a tag the check had set.Older readers read only the
<Tag>values, so they still see the same tags.How the check treats tags:
Verdict storage: the verdict is a MessagePack record:
AlignmentSpotPropertyKey 66;ChromatogramPeakFeatureKey 56.ChromatogramPeakFeaturekeys 54 and 55 are reserved as integers for the MS-DIAL 5.6 line'sRawDataIDTypeandAccumulatedDataIDType. Replace the placeholders with those members when the 5.6 line is merged.Backward compatibility: old files load, and master's MsdialCore reads files written by this branch.
Verification
Builds (Release): MsdialCore, MsdialLcMsApi, the Console and MsdialGuiApp all build with 0 errors.
Tests:
MSDIAL_LIPIDRT_REFERENCE_DIR)The GUI failure is
LcmsAnalysisModelWithoutRawTests.ASampleWithoutRawDataOpensFromWhatTheProjectStores. Its cleanup cannot delete a locked temporary.dcl, and it fails the same way before this PR's GUI commits.Parity with the Python reference: 100% agreement, object by object (outcome, reason, score and evidence fields). Covered: demo POS/NEG alignments and per-sample peaks, ClassLevelPolicy Consistent and Corroborated, cross-polarity, and seven evaluation runs.
Console smoke runs on public verification data:
Lipid RT check: False, or on a non-lipidomics method, outputs match master except for version strings, timestamps and paths.Decisions still open
IsEnablednull means on for TargetOmics Lipidomics). An existing lipidomics method file therefore now gets verdicts, Confirmed tags, the review TSV and the mzTab-M columns. An opt-in default is a one-line change.Deferred to v1.1: applying cross-polarity from the GUI or Console (MS-DIAL Interactive's lab pos/neg merge will call it), LC-IM-MS / GC-MS / DIMS hooks, and light-mode persistence.
🤖 Generated with Claude Code