Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs-site/building/builder-window.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ of the window:
| **Experiment** | Metadata, mice, zones, lab vocabulary and plugins. Where you say what this experiment *is*. |
| **Dashboard** | This page: the node graph, hardware, camera and properties. |
| **Run** | The operator view — run control, device states, camera, experiment info. |
| **Analyze** | Behavior Analysis, Batch Pose Tracking, Session Review, Multi-Camera Recording and the device check. Each opens in its own window. |
| **Analyze** | Batch Pose Tracking, Behavior Analysis, Session Review, Multi-Camera Recording and the device check. Each opens in its own window. |

New and Open both land you on **Experiment**, because naming the experiment and
entering its animals comes before wiring anything up.
Expand All @@ -28,7 +28,7 @@ From top to bottom:
|---|---|
| **Menu bar** | Five menus: **File**, **Edit**, **Experiment**, **View**, **Help**. |
| **Chrome strip** | One row carrying the experiment name, one dot per board, the centred tabs, the run-state pill once a run is under way, and the ++ctrl+k++ hint. The two panel toggles appear on this tab only, since they drive the panels below. |
| **Left panel** | Tabs: **Nodes**, **Hardware**, **Control**, **Files**. |
| **Left panel** | Tabs: **Nodes**, **Hardware**, **Control**. |
| **The canvas** | The node graph itself. Everything else gets out of its way. |
| **Right panel** | Tabs: **Properties**, **Camera**. |
| **Status bar** | The thin line along the bottom: connection indicator, session state, and transient messages. |
Expand All @@ -47,7 +47,7 @@ panels, one on each side of the canvas.

| Panel | Tabs |
|---|---|
| Left | **Nodes** (the node library), **Hardware** (boards and devices), **Control** (drive a device by hand), **Files** (New, Open, Save, Save As) |
| Left | **Nodes** (the node library), **Hardware** (boards and devices), **Control** (drive a device by hand) |
| Right | **Properties** (settings for the selected node), **Camera** (the live camera panel) |

Click a tab at the top of a panel to switch to it. Only one tab per side is
Expand All @@ -71,7 +71,7 @@ buttons**, one per tab, down the outside edge of the window:

| Panel | Rail buttons |
|---|---|
| Left | **N** Nodes, **H** Hardware, **D** Control, **F** Files |
| Left | **N** Nodes, **H** Hardware, **D** Control |
| Right | **P** Properties, **C** Camera |

Hovering a rail button names the tab. Clicking one does two things at once: it
Expand Down Expand Up @@ -226,7 +226,7 @@ about.
|---|---|---|
| **Hardware** | Add Board…, Add Device…, New Custom Device Type…, Connect All, Disconnect All | **Add Board** and **Add Device** are buttons in the **Hardware** tab, which is where you were probably already looking. The rest: ++ctrl+k++ |
| **Run** | Start (++f5++), Stop (++shift+f5++), Emergency Stop (++ctrl+shift+escape++) | The toolbar, the **Run** tab, the shortcuts, or ++ctrl+k++ |
| **Tools** | Behavior Analysis…, Batch Pose Tracking…, Session Review…, Multi-Camera Recording…, GPU / Device Check… | The **Analyze** tab, which has a card for each — including, when a tool needs an optional dependency stack, the exact install line. **Plugins** moved to the **Experiment** tab instead |
| **Tools** | Batch Pose Tracking…, Behavior Analysis…, Session Review…, Multi-Camera Recording…, GPU / Device Check… | The **Analyze** tab, which has a card for each — including, when a tool needs an optional dependency stack, an **Install** button that adds it and offers to restart GLIDER. **Plugins** moved to the **Experiment** tab instead |

Every row in the palette names the menu it came from, so if you remember that
something "was under Tools", the palette still tells you so.
Expand Down
61 changes: 32 additions & 29 deletions docs-site/camera-behavior/camera.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,35 +145,38 @@ The `source` column tells you what kind of event each row is:
The `frame` column holds the most recent camera frame seen when the event fired,
so events line up with the video too.

## Calibration

Calibration teaches GLIDER how many pixels correspond to a real-world distance, so
tracking can report movement in millimeters instead of pixels.

Open the camera settings dialog, go to the **Tools** tab, and click **Open
Calibration…**. In the calibration window:

1. Click **Capture Frame** to grab a still from the camera (an object of known
size should be visible — for example the width of your arena).
2. Click **Draw Line**, then click two points to draw a measurement line across
that known distance. Clicking near an existing endpoint snaps to it.
3. Enter a **Name** (for example `Apparatus Width`), the real **Length**, and the
**Unit**. Supported units are millimeters (`mm`), centimeters (`cm`), meters
(`m`), inches (`in`), and feet (`ft`).
4. Click **Add Line**. You can add several lines; GLIDER averages them to compute
a **pixels/mm** ratio, shown in the info panel.

Use **Save…** / **Load…** to store a calibration as a JSON file and reuse it later.

Once a calibration is active, the tracking CSV's `distance_mm` and `cumulative_mm`
columns are filled with real distances. Without calibration, those columns simply
mirror the pixel values.

!!! note "Calibration is tied to resolution, not to a specific camera"
A calibration is stored against the frame size it was drawn at and scales if
the resolution changes. There is a single active calibration per session
rather than one per camera, so in multi-camera setups it applies to the view
you drew it on.
## Arena (calibration)

The arena tells GLIDER where the floor is and how big it is. That one outline
also sets the scale, so tracking reports movement in millimeters instead of
pixels. There is no separate measuring step.

Open the camera settings dialog, go to the **Tools** tab, and click **Arena…**.
In the arena window:

1. Pick the **Shape**: **Rectangle** or **Circle**.
2. Enter the floor's real size: **Width** and **Height** for a rectangle, or the
**Diameter** for a circle.
3. Outline the floor on the live frame, clicking where the bedding meets the
wall:
- **Rectangle:** click the four corners in order (top-left, top-right,
bottom-right, bottom-left). Corners outside the picture can be placed in
the margin around it.
- **Circle:** click the centre, then a point on the edge, then a second edge
point about a quarter-turn round. The outline is an ellipse, so it still
fits a camera that is slightly tilted. Drag the centre to move it, or an
edge point to fit it.
4. Check the status line. It shows the scale at the centre, and for a rectangle
it warns when the corners look misplaced.

The arena is remembered between sessions. Once it is set, the tracking CSV's
`distance_mm` and `cumulative_mm` columns are filled with real distances.
Without one, those columns simply mirror the pixel values.

!!! note "The scale is tied to resolution, not to a specific camera"
An arena is stored against the frame size it was drawn at and scales if the
resolution changes. There is a single arena per session rather than one per
camera, so in multi-camera setups it applies to the view you drew it on.

## Zones (regions of interest)

Expand Down
6 changes: 0 additions & 6 deletions docs-site/camera-behavior/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,3 @@ only install them if you need them:
followed the [Installation](../getting-started/installation.md) guide you
already have it. Preview, tracking, and data logging work without it.

!!! tip "Two different things are called 'Behavior Analysis'"
GLIDER has a **live, movement-based** behavior classifier (built into the
camera settings, described in [Tracking](tracking.md)) and a **supervised
machine-learning** workflow (a separate window, described in
[Behavior Analysis](behavior.md)). They are unrelated tools. The pages below
point out which is which so you don't confuse them.
4 changes: 2 additions & 2 deletions docs-site/camera-behavior/pup-retrieval.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,8 @@ click one to jump to it, or select it and press **Remove**.
**Trial start.** Scrub to the moment the dam is released and press **Set at
playhead**. Everything is timed from here.

**Scale.** **Set scale…** opens the calibration tool on a frame of the video:
measure something of known length. The scale lives in that video's setup. If
**Scale.** **Set scale…** opens the [arena](camera.md#arena-calibration) tool on a
frame of the video: outline the floor and give its real size. The scale lives in that video's setup. If
more than one pup video is open it asks whether to use it for every session in
the cohort, and afterwards tells you about any session whose file couldn't be
read and so kept its old scale.
Expand Down
21 changes: 3 additions & 18 deletions docs-site/camera-behavior/tracking.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ the `_tracking.csv` file. Its columns are:
frame, timestamp, elapsed_ms, flow_elapsed_ms, object_id, class,
x, y, w, h, confidence, center_x, center_y,
distance_px, distance_mm, cumulative_mm,
zone_ids, behavioral_state, velocity_px_frame
zone_ids, velocity_px_frame
```

Key columns:
Expand All @@ -116,14 +116,13 @@ Key columns:
| `x, y, w, h` | Bounding box in pixels (top-left corner, width, height) |
| `center_x, center_y` | Bounding-box center, in pixels |
| `distance_px` | Distance moved since the previous frame, in pixels |
| `distance_mm`, `cumulative_mm` | The same distances in millimeters, when a [calibration](camera.md#calibration) is active (otherwise they mirror pixels) |
| `distance_mm`, `cumulative_mm` | The same distances in millimeters, when an [arena](camera.md#arena-calibration) is set (otherwise they mirror pixels) |
| `zone_ids` | Names of the [zones](camera.md#zones-regions-of-interest) containing the object |
| `behavioral_state` | Movement state, when the live classifier is on (see below) |
| `velocity_px_frame` | Speed in pixels per frame |

!!! note "Coordinates are in pixels"
Positions and box sizes are always in pixels. The millimeter columns are only
real-world distances when you have [calibrated](camera.md#calibration) the
real-world distances when you have [set an arena](camera.md#arena-calibration) for the
camera; otherwise they carry the pixel values unchanged.

A row with `object_id` of `-1` and class `motion` marks a frame where motion was
Expand Down Expand Up @@ -155,20 +154,6 @@ A batch run writes several files into the chosen folder:
(`frame ÷ fps`), so the results are reproducible regardless of how fast the
pass runs.

## Live movement-based behavioral state

The camera settings dialog has an **Enable Behavior Analysis** option (with
**Freeze**, **Immobile**, and **Darting** thresholds measured in pixels per
frame). This is a simple, rule-based classifier: it labels each tracked object's
movement each frame as freezing, immobile, moving, or darting, and writes that
into the `behavioral_state` column.

!!! note "This is not the machine-learning workflow"
This live, threshold-based classifier is separate from the supervised
[Behavior Analysis](behavior.md) tool, which learns behaviors from your own
labeled examples. If you want to score complex behaviors (rearing, grooming,
and so on), that is the tool to use.

## Pose and keypoint tracking

GLIDER can also run YOLO **pose** models, which return body **keypoints** (for
Expand Down
8 changes: 0 additions & 8 deletions docs-site/reference/ecosystem-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,14 +200,6 @@ The Camera panel can process a recorded video without a live camera. The batch r

## Behavior analysis: annotation, training, and application

GLIDER has two distinct behavior features that should not be conflated.

### Live movement-state labels

The camera settings support a light-weight, rule-based live state classifier. It uses movement thresholds to label tracked objects as states such as freezing, immobile, moving, or darting. These labels are useful for immediate feedback and are stored in tracking output when enabled. They are not a trained model for complex behavior categories.

### Supervised behavior workflow

The Behavior Analysis tool is an optional workflow built around pose data. It proceeds in three stages:

1. **Annotate:** create a vocabulary and label selected video clips. Labels are saved alongside pose data.
Expand Down
4 changes: 2 additions & 2 deletions docs-site/runner/runner.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,13 +83,13 @@ message won't bury it; tap it once you've read it.
## The Experiment tab

The Experiment tab is where you say what this experiment *is*. A row of
**section chips** runs across the top — **Metadata**, **Mice**, **Zones**, **Lab
**section chips** runs across the top — **Metadata**, **Animals**, **Zones**, **Lab
Vocabulary**, and **Plugins** — and tapping one shows that section at full width
below. Each section scrolls by dragging with a finger.

These are the same editors the desktop's **Experiment** tab uses, so what you
change here is saved in the same `.glider` file. See
[Subjects & Lab Vocabulary](../building/subjects.md) for the mice and the
[Subjects & Lab Vocabulary](../building/subjects.md) for the animals and the
vocabulary lists and [Plugins](../building/plugins.md) for the plugin catalogue;
zones are covered under [Camera & Recording](../camera-behavior/camera.md#zones-regions-of-interest).

Expand Down
3 changes: 2 additions & 1 deletion src/glider/analysis/behavior/scale_guard.py
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,8 @@ def calibration_spread_warning(
return None

by_resolution: dict[tuple[int, int], list[float]] = {}
for calibration in cal_set.entries.values():
# Arenas carry the line's scale interface, and win where a video has both.
for calibration in {**cal_set.entries, **cal_set.arenas}.values():
scale = float(getattr(calibration, "pixels_per_mm", 0.0) or 0.0)
width = int(getattr(calibration, "calibration_width", 0) or 0)
height = int(getattr(calibration, "calibration_height", 0) or 0)
Expand Down
3 changes: 2 additions & 1 deletion src/glider/analysis/behavior/units.py
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,8 @@ def _borrow_from_folder_mate(cal_set, video: Path) -> float | None:

folder = _folder_name(video)
candidates = []
for stored, calibration in cal_set.entries.items():
# An arena carries the same scale interface as a line, and wins where both exist.
for stored, calibration in {**cal_set.entries, **cal_set.arenas}.items():
if not folder or _folder_name(stored) != folder:
continue
if (calibration.calibration_width, calibration.calibration_height) != resolution:
Expand Down
12 changes: 8 additions & 4 deletions src/glider/core/glider_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
from glider.core.flow_engine import FlowEngine
from glider.core.hardware_manager import HardwareManager
from glider.core.live_signals import LiveSignalBus
from glider.vision.arena import ArenaCalibration
from glider.vision.audio_recorder import AudioRecorder, mux_audio_video
from glider.vision.calibration import CameraCalibration
from glider.vision.camera_manager import CameraManager
from glider.vision.cv_processor import CVProcessor, CVSettings
from glider.vision.multi_camera_manager import MultiCameraManager
Expand Down Expand Up @@ -90,7 +90,7 @@ def __init__(self):
self._video_recorder = VideoRecorder(self._camera_manager)
self._tracking_logger = TrackingDataLogger()
self._audio_recorder = AudioRecorder()
self._calibration = CameraCalibration()
self._calibration: ArenaCalibration | None = None

# Multi-camera support
self._multi_camera_manager = MultiCameraManager()
Expand Down Expand Up @@ -232,10 +232,14 @@ def event_logger(self) -> DeviceEventLogger:
return self._event_logger

@property
def calibration(self) -> CameraCalibration:
"""Camera calibration instance."""
def calibration(self) -> ArenaCalibration | None:
"""The live camera's arena, which carries its real-world scale (#212)."""
return self._calibration

@calibration.setter
def calibration(self, arena: ArenaCalibration | None) -> None:
self._calibration = arena

@property
def multi_camera_manager(self) -> MultiCameraManager:
"""Multi-camera manager instance."""
Expand Down
30 changes: 19 additions & 11 deletions src/glider/gui/behavior/analysis_window.py
Original file line number Diff line number Diff line change
Expand Up @@ -1486,37 +1486,45 @@ def _redraw_nest(self) -> None:
self._refresh_retrieval()

def _run_scale_dialog(self, video: Path) -> tuple[float, bool] | None:
"""Lines measured on *video*: ``(px_per_mm, for_the_whole_cohort)``, or None."""
from glider.gui.dialogs.calibration_dialog import CalibrationDialog
from glider.vision.calibration import CameraCalibration
"""The arena outlined on *video*: ``(px_per_mm, for_the_whole_cohort)``, or None."""
from glider.gui.dialogs.arena_dialog import ArenaDialog
from glider.vision.frame_provider import VideoFrameProvider

provider = VideoFrameProvider(video)
if not provider.is_connected:
grabbed = provider.get_frame() if provider.is_connected else None
if grabbed is None:
provider.release()
QMessageBox.warning(self, "Set scale", f"{video.name} could not be opened.")
return None

def scrub(index: int):
provider.seek(index)
frame = provider.get_frame()
return None if frame is None else frame[0]

dialog = None
try:
dialog = CalibrationDialog(
frame_provider=provider,
calibration=CameraCalibration(),
dialog = ArenaDialog(
grabbed[0],
title=video.name,
parent=self,
show_file_buttons=False,
frame_count=provider.frame_count,
on_scrub=scrub,
)
dialog.setWindowTitle(f"Set scale — {video.name}")
if dialog.exec() != QDialog.DialogCode.Accepted:
return None
calibration = dialog.get_calibration()
calibration = dialog.calibration()
finally:
if dialog is not None:
dialog.deleteLater()
provider.release()
if calibration is None:
return None
px_per_mm = float(calibration.pixels_per_mm) if calibration.is_calibrated else 0.0
if px_per_mm <= 0:
return None
# Asked after the measuring rather than as a box inside it: the
# calibration dialog is shared with the rig and the batch tool.
# arena dialog is shared with the rig and the batch tool.
whole = sum(view.pups is not None for _path, view in self._cohort) > 1 and (
QMessageBox.question(
self,
Expand Down
Loading
Loading