SealTK functionality#1747
Open
BryonLewis wants to merge 6 commits into
Open
Conversation
* Fix spelling error, move duplicate filter IDs to a single variable * Add percentile stretch option to 'Image Enhancements' for thermal imagery. * Fix per-camera image enhancement state and persistence Each camera now maintains independent enhancement values. Viewer.vue loads enhancements from each camera's sub-meta instead of root meta, saves to the correct camera-scoped path via a per-camera debounce map, and flushes pending saves before switching cameras. * Fix Vue 2 reactivity, spurious save, and percentileStretch aliasing in per-camera enhancements * Sync image enhancement sliders when a new camera is selected * client/platform/desktop/backend/media/ * Fix seven correctness and quality issues in display processing backend Float32/Float64 TIFFs now use sort-based percentile instead of the integer histogram, which collapsed sub-integer values to a single bin and produced solid-color output silently. Uint8/Uint16 bin counts are now type-matched (256 vs 65536). Zero-range frames (flat field, dead sensor) now render as mid-gray; the previous check tested only raw[0] against min, so every uniform frame filled with black. Cache keys now include mtimeMs so entries are invalidated when a source file is replaced at the same path, matching the tile cache pattern. fs.statSync replaced with await fs.stat to avoid blocking the event loop on a per-frame hot path. The two try-catch blocks collapsed into one with ENOENT distinguished for a 404. Array.isArray guards added for low and high query params; duplicate params previously passed validation and silently discarded extra values. A low >= high check catches inverted percentile inputs at the boundary. encodePngRgba deduplicated — exported from displayProcessing.ts and imported by geotiffTiles.ts, removing the unused pngjs import. * Wire percentile stretch UI: toggle/sliders in ImageEnhancements, display URL remapping in Viewer * fix: percentile slider bounds, signed-16 histogram, per-camera URL debounce, stale isDefaultImage, TIFF lazy load, and in-flight decode dedup * fix: derive display URL origin from rawUrl to bypass Vite proxy in Electron * fix: stretch original TIFF via dataset id+frame, pass through non-TIFF originals * Reuse the existing viewer when imageData URLs are swapped in place The imageData watcher rebuilt the whole viewer via initializeViewer, which replaces the geojs map and orphans every feature layer and event handler other components attached to it (LayerManager annotation layers crash with 'null.api' on redraw; calibration click handlers go dead). An imageData change on a mounted annotator is a display-URL swap for the same camera (percentile stretch), so rebuild only the image cache and redraw the current frame on the existing map; drawImage already handles dimension changes via resetMapDimensions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Linting * Change stretch to 0-1 on the low end, 99-100 on the high end, more realistic for IR data. * disable percentile stretch on unsupported imagery * add histogram viewer to percentile stretch * web support for percentile stretching * updating docs about image enhancements * support for multicamera IR/EO importing from parent folder for electron and web --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Bryon Lewis <Bryon.Lewis@kitware.com>
* Add batch multicam collect-folder scanner Scans a root folder of collect folders (collect -> camera -> images) and produces per-collect beginMultiCamImport arguments for MML seal surveys, without copying imagery. Camera subfolders are matched case-insensitively across collects; the canonical camera set is the union of image-bearing subfolder names, limited to 2-3 alphanumeric cameras to match the existing multicam subfolder import. Missing/empty/imageless camera folders are reported as per-collect blocking problems; frame-count mismatches are non-blocking warnings (seal collects legitimately drop frames). Default display camera follows pickDefaultMulticamCamera (center/middle alias, else the middle camera of display order). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add batch multicam import UI and desktop wiring New 'MultiCam Batch' entry in the Open Image Sequence dropdown (desktop only, behind a new batch-multi-cam-import prop on ImportButton) opens ImportMultiCamBatchDialog: pick a root folder, review a per-collect validation summary table (cameras, frame counts, problems/warnings), then import all valid collects sequentially via the existing import-multicam-media / finalize-import machinery, continuing past per-collect failures and reporting a final summary. Adds the scan-multicam-batch IPC handler and scanMultiCamBatch frontend API. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * make the dropdown area larger to display full multicam batch import option * editable batch import datasets * add support for web to batch upload multicam datasets * update documentation for batch multi-camera import functionality * linting * ordering multicam imports * prevent stereoscopic loading and calculations when in multicamera mode --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Bryon Lewis <Bryon.Lewis@kitware.com>
* Parse per-frame capture timestamps from filenames
Multicam datasets assume cameras are frame-aligned by position, but
cameras in a rig can independently drop frames, so index i isn't
necessarily the same real-world moment across cameras. Phase I of
SEAL feature 5: derive a best-effort timestamp per frame from its
filename and expose it on FrameImage/MediaResource, so later phases
can build a real aligned timeline instead of assuming index parity.
Parsing is provisional (no real MML sample filenames yet) and fully
optional -- undefined/None is the expected common case and must not
affect existing datasets. No meta.json schema change: the timestamp
is computed at read time from the filename, mirroring how url/
filename are already derived, not persisted.
* Align multicam playback to a global timestamp-derived timeline
Multicam playback assumed cameras were frame-aligned by position, but
cameras in a rig can independently drop frames, so index i wasn't
necessarily the same real-world moment across cameras. Phase II of
SEAL feature 5: when every camera has a timestamp on every frame,
build a global aligned timeline and resolve each camera's own local
frame per slot, blanking a camera's pane when it has none at that
instant. Falls back byte-identically to today's positional broadcast
whenever any camera lacks full timestamp coverage -- zero risk to
existing datasets.
Also fixes a latent gap this surfaced: continuous playback bypassed
the aggregate controller entirely (each camera free-ran its own
frame-advance loop), so aligned mode would have applied to scrubbing
but silently reverted to raw per-camera advancement during playback.
Centralizes a single tick in the aggregate controller when aligned.
* Fix stale annotation overlay and add UI for blanked aligned-timeline panes
Phase III of SEAL feature 5. Phase II blanked a camera's image when the
global aligned timeline has no frame for it at a slot, but left two
gaps: LayerManager kept drawing annotations for whatever local frame
the camera was on before blanking, since seek(undefined) deliberately
leaves data.frame untouched and LayerManager's watchers are keyed on
frame number -- so stale boxes floated over an otherwise-blank pane.
The pane itself also gave no visual indication it was blanked rather
than just failing to load.
Expose the existing per-camera hasFrame flag on MediaController, and
have LayerManager watch it: every prior updateLayers() call site now
goes through a new refreshLayers(), which disables all annotation/edit
layers instead whenever hasFrame is false. ImageAnnotator/VideoAnnotator
now render a 'eNo frame at this instant' overlay over a blanked pane.
* Fix local-vs-global frame bugs in editing/seek and add timeline gap indicator
Track create/edit/delete and seek operations were keying off the global
aligned slot index instead of the selected camera's local frame, writing
detections to wrong frame numbers when alignment is active. Adds
selectedCameraFrame() in useModeManager, seekCameraFrame() with an
inverse aligned index, per-selected-camera time updates in Viewer, and
an aligned-gap indicator bar under the frame slider in Controls.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix aligned timeline never activating on real multicam datasets
imageData was initialized as { singleCam: [] } and multicam loading only
added per-camera keys on top, so the leftover empty singleCam entry made
canAlign() disqualify every multicam dataset. Rebuild imageData with
exactly the dataset's cameras before loading (replacing the whole object,
since adding keys via bracket assignment is non-reactive under Vue 2.7)
and restrict the alignedTimeline computed to the multicam camera list so
stale keys can never disqualify alignment.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Make per-camera controller lookups null-safe during load
loadData calls changeCamera(defaultCamera) before progress.loaded, so no
annotator (and thus no camera controller) exists yet and the aggregate is
the empty stub whose getController() throws. That produced an unhandled
promise rejection on every multicam load and skipped selectCamera's
edit-mode restore and change-camera emit. selectedCameraFrame() and
isCreatingNewDetection() shared the same throw exposure whenever a
camera's annotator never mounted. Wrap all three lookups with a graceful
fallback (skip the time resync / fall back to the aggregate frame).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Route local-frame seek producers through seekCameraFrame
Viewer.vue's seekToFrame (track-seek from TrackItem/TrackList/
TrackDetailsPanel/AttributesSubsection keyframe navigation, BottomPanel)
and ControlsContainer's Timeline chart click-seek were calling the
aggregate seek(), which operates in global slot space under an aligned
timeline, with frames produced in the selected camera's local frame
space. Translate both through seekCameraFrame(selectedCamera, frame)
(a passthrough when alignment is inactive), and drive the Timeline
playhead from time.frame -- which tracks the selected camera's local
frame -- so charts, playhead, and click-seek all agree on frame space.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Center timeline gap-indicator bands on the slider thumb position
The gap gradient painted slot s over [s/maxFrame, (s+1)/maxFrame], a full
cell to the right of the v-slider thumb (which sits at s/maxFrame), and a
trailing gap at s === maxFrame painted [100%, 100+%] -- invisible, since
the widening branch only ran for sub-0.25% cells. Extract the computation
into a pure computeGapGradient() in alignedTimeline.ts that centers each
band on the thumb position ([(s-0.5)/maxFrame, (s+0.5)/maxFrame], clamped
to [0, 100]) and unit-test it, including edge slots and min-width
widening.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Apply the aligned seek when a resolver is installed or a camera mounts
setAlignedFrameResolver() reset alignedCurrentFrame to 0 without seeking,
so a camera that should be blank at slot 0 kept showing its local frame 0
until the first user seek. Perform an aligned seek to slot 0 on resolver
install, and re-apply the current slot (flush: 'post') whenever a camera
registers while a resolver is active -- annotators self-seek to their own
local frame 0 during init, which covers the common ordering where the
resolver is installed before the annotators mount.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Pair identical-timestamp frames positionally and skip no-op timeline reinstalls
Sorting triples camera-name-first swept all of one camera's same-time
frames into consecutive single-camera slots, so a calibration dataset
where every frame shares one collection timestamp produced a timeline of
mostly blank panes instead of positional pairing; tie-break equal
timestamps by local index first. Also key the Viewer's resolver install
on the slot structure so display-URL swaps (percentile stretch) that
leave timestamps unchanged no longer reinstall the resolver and visibly
re-seek every camera.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Allow unequal multicam frame counts via filename frame-index inference
Add an "Infer frame index from filename" toggle to the multicam import
dialog. When enabled, the equal-frame-count validation is skipped so
datasets with dropped frames (e.g. KAMERA, which encodes a capture
timestamp per filename) can be imported and aligned downstream by their
filename timestamps rather than by exact positional index.
Also drop the matching server-side equal-frame-count rejection in
create_multicam, keeping the frame-count call only for its validation
side effect.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Remove provisional language
* Unify frame-timestamp parsing on a single client implementation
Replace the hand-synced Python + TypeScript filename timestamp parsers with
one shared TS module (dive-common/frameTimestamp.ts) used by both platforms.
The girder server previously parsed each frame's timestamp in get_media and
embedded it in the /media response, but nothing server-side ever read it -- it
existed only to reach the client. Move parsing to the client: the web-girder
API layer now attaches timestamps from filenames after fetching media (both the
single-cam /media response and multicam per-camera imageData from /dive_dataset),
mirroring how the desktop backend already parses on media enumeration.
- Add dive-common/frameTimestamp.ts (parseFrameTimestamp + attachFrameTimestamps)
and move its tests out of sharedUtils.spec.ts.
- Point the desktop backend importers at the new shared module.
- Delete server dive_utils/timestamp_parser.py and its test; drop the now-dead
MediaResource.timestamp field and the get_media parsing call; update the
multicam dataset tests that asserted server-side timestamps.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Shrink alignment to 0s, from 0.5s tolerance
* Update tests - matching intended metadata behavior
* documentation update
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Bryon Lewis <Bryon.Lewis@kitware.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Feature branch that will implement most of the functionality of the SealTK tool.
Once branches are merged into this PR this can be merged into main.