Skip to content

Pup retrieval scoring, and experiment protocols that switch analyses on - #199

Merged
gbradham merged 4 commits into
mainfrom
claude/pup-retrieval
Oct 15, 2026
Merged

gbradham merged 4 commits into
mainfrom
claude/pup-retrieval

Conversation

@gbradham

Copy link
Copy Markdown
Member

Summary

GLIDER can now score maternal pup retrieval from video, and experiments declare a protocol that switches the matching analysis on.

Pup retrieval

  • Batch Pose: a Pup segmentation card (shown under the Pup Retrieval protocol) runs a semantic pup-segmentation model alongside the adult pose model and writes <video>_pups.npz. Each frame's mask is split into one blob per pup, the blobs are linked frame to frame, and the fragments are consolidated to the litter size, so every pup keeps one ID through the video.
  • Session Review: Open video… opens a bare video with its pose CSV and pups file; Open folder as cohort… finds pup videos. Videos that share a folder get their own <stem>.review_markers.json.
  • Retrieval tab:
    • Nest outlines: redrawn from the playhead when the dam moves the nest.
    • Trial start and scale (one video or the whole cohort).
    • Thresholds… saved per cohort as retrieval_protocol.json.
    • Results: a per-pup table (contact and retrieval latency, carries, drops) and a litter summary.
    • Hidden pups: a pup hidden in the dam's mouth is followed at her nose until she reaches the nest.
    • Settings files: unreadable or invalid ones open read-only with the reason, and are never overwritten.
  • Viewer and timeline:
    • On the video: the nest, numbered pup outlines, the nose-to-nest line, and a ring for a pup in her mouth.
    • HUD chips ("pup_2 carried", "nose 4.1 cm from nest" / "nose in nest").
    • One lane per pup, a nose-to-nest lane, and read-only trial-start, nest and cutoff flags.
  • Exports:
    • retrieval.csv: one row per pup per session.
    • retrieval_litter.csv: one row per session.
    • Per-frame <stem>_nest_distance.csv.
    • Per-range retrieval numbers in the epoch table.
    • cutoff_s is the effective cutoff, so unretrieved pups can be censored correctly when a video ends early.

Protocols

  • The experiment form's Protocol box is now one dropdown (the old Type box is merged into it). It is a single shared value across the Experiment tab, the Run tab, Session Review's Protocol ▾ and Batch Pose.
  • Protocols are a presets table (core/experiment_types.py), and each lists the analyses it unlocks. Adding a protocol (e.g. an Open Field analysis) is one entry plus its own code.
  • Built-in Pup Retrieval and Open Field run presets ship with GLIDER. Picking one applies its protocol and per-run fields to the open experiment and keeps the rig and flow. If one of your own presets is open, the experiment is detached from it, so that preset is never overwritten.

Also

  • Ultralytics 8.4.19 → 8.4.174 (floor 8.4.136, the first release that loads semantic-segmentation models).
  • atomic_write_bytes in core/fileio.py, which keeps a shared file mode on lab drives.
  • An unscored video keeps its timeline ruler and playhead.
  • Inspector tab titles scroll instead of eliding on macOS.
  • Docs page docs-site/camera-behavior/pup-retrieval.md and a CHANGELOG entry.

Testing

  • Full gate: 6472 passed, 4 skipped, 7 deselected; ruff and black clean.
  • Real video: a real retrieval recording (not committed: the data belongs to the lab that recorded it) went through the batch pipeline with both models (about 30 ms per frame on an M-series Mac). Session Review scoring then matched an earlier frame-by-frame hand check:
    • every pup was retrieved;
    • every contact time matched;
    • every retrieval time matched to within one frame.
  • tests/integration/test_pup_retrieval_real_video.py is opt-in (-m slow). It reads the video, models, nest outline and hand-checked latencies from environment variables, and skips without them.
  • Built test-first with a review of each task, then a whole-branch review and one fix wave. The fixes covered invalid settings files, one animal appearing twice in a cohort, the effective cutoff, and the cost of scoring a cohort on the UI thread.

Known limits / follow-ups

  • Not yet tested on Windows: on Windows and Linux fonts the pup table may scroll sideways a few pixels in a 300 px inspector.
  • Cutoff edge case: a pup carried in within 2 s of the cutoff never counts as retrieved, because the 2 s in-nest hold is checked only inside the window.
  • Undetected pups: a pup the model never detects still counts as scattered.
  • Shared IDs: two pup videos inside one recording folder get the same session ID.
  • Filtering: Batch Pose's pose filtering is off by default. Retrieval scoring uses the nose as written; low-confidence noses were rare (about 1%) on the real clip.
  • Paused: a live Pup Carry input node for closed-loop stimulation during a run.

…tch analyses on

Pup retrieval, end to end:
- Batch Pose can also run a semantic pup-segmentation model and track every
  pup through each video with a stable ID (<video>_pups.npz): masks are split
  into one blob per pup, linked frame to frame and consolidated to the litter
  size.
- Session Review opens a bare video (Open video..., or a folder as a cohort)
  with its pose CSV and pups file. Videos sharing a folder get their own
  marker file (<stem>.review_markers.json).
- A Retrieval tab sets the nest (redrawn from the playhead when the dam moves
  it), the trial start and the scale (one video or the whole cohort), and
  scores each pup's contact and retrieval latency, carries and drops against
  the cohort's thresholds (retrieval_protocol.json). A pup hidden in the
  dam's mouth is followed at her nose until she reaches the nest. Unreadable
  or invalid settings files open read-only and are never overwritten.
- The video shows the nest, each pup's outline, the nose-to-nest line and a
  ring for a pup in her mouth; the timeline gets a lane per pup, a
  nose-to-nest lane and read-only trial/nest/cutoff flags; Export writes
  retrieval.csv, retrieval_litter.csv and per-frame <stem>_nest_distance.csv,
  and the epoch table gains per-range retrieval numbers.

Protocols:
- The experiment form's Protocol box is one dropdown (Type merged into it)
  and is a single shared value across the Experiment tab, the Run tab,
  Session Review and Batch Pose.
- Protocols are a presets table (core/experiment_types.py): each lists the
  analyses it unlocks, so a new protocol is one entry. Pup-retrieval UI
  appears only under Pup Retrieval.
- Built-in Pup Retrieval and Open Field run presets ship with GLIDER and
  apply their protocol and per-run fields to the open experiment.

Also: Ultralytics 8.4.174 (floor 8.4.136, the first to load semantic
segmentation models); an atomic bytes writer that keeps a shared file mode;
an unscored video keeps its timeline ruler and playhead; inspector tab
titles scroll instead of eliding on macOS. Docs page and CHANGELOG updated;
an opt-in acceptance test runs against a real video supplied through
environment variables.
…verflowing

Windows CI: at the inspector's width a setup row (a fixed 78 px label, the
value and its button) was wider than the pane, so the column scrolled
sideways. The rows are now a QFormLayout with WrapLongRows: where label,
value and button do not fit side by side, the value and button wrap under
the label. A test reproduces the overflow by enlarging the text.
…lows

The overflow only happens on Windows CI, which we cannot run locally; the
assertion now names the viewport, the content's minimum, the font and each
row's minimum width so the next failure says which one is too wide.
The widths Windows CI reported settle it: its offscreen platform has no font
(empty family, -1 px) and lays text out 1.6-2.7x wider than any real font,
so 'does real text fit 330 px' cannot be asked there. Those two tests skip on
such a platform, as the shell tests already do; a new structural test checks
on every platform that the setup rows are a QFormLayout wrapping long rows.
@gbradham
gbradham merged commit 220e63f into main Oct 15, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant