Skip to content

Releases: fourMs/MGT-python

1.32.0 — the notebook teaches, and the pose cache tells the truth

Choose a tag to compare

@alexarje alexarje released this 03 Sep 20:31

The tutorial notebook is redesigned as a gentle eight-part introduction — from a video in your hands to sound against motion — executed once with images only (6.8 MB where 16 stood), with a Colab button in the README's badge row and an install cell that brings the [pose] extra.

Fixed

  • The pose cache reached the posture and action layers in the wrong shape: postures_from_pose() counted zero postures for every recording and the actions pose path failed outright. pose_cache_landmarks() restores the (frames, 33, 3) trajectory contract with below-threshold refusals marked invisible; the actions path unpacks pose_qom correctly and masks refusals as NaN.
  • A failed pose-weights download now raises one clear RuntimeError naming the fix (pip install "musicalgestures[pose]") instead of printing, returning, and letting the caller's next line die on an attribute that was never set.

Added

  • ReID v2.1: associate_fragments(embeddings=...) links each segment's movers into persistent chains across breaks by the same strictly-more-separated appearance rule, with a new chains key and per-mover chain ids. Measured on the co-located dance day: the longest per-dancer aggregate grows from a 6.6-minute segment to a 12.7-minute chain over five segments; refusal remains the rule.

Full details in the CHANGELOG.

1.31.0

Choose a tag to compare

@alexarje alexarje released this 02 Sep 22:59

Eye tracking joins the toolbox, on the video's clock.

Added

  • Pupil Labs Neon exports on the video's clock — new _pupillabs module. read_pupil_export reads a Pupil Cloud timeseries export (gaze at 200 Hz, 3D eye states, IMU, fixations, saccades, blinks, events, scene camera intrinsics) and keeps every table in seconds after the recording start; pupil_to_frames places a video on that clock by naming the event it starts at (or an offset) and bins everything to the video's frames — gaze position and angular velocity, worn, fixation/saccade/blink flags with the export's ids, pupil size masked around blinks and interpolated, head gyroscope magnitude, dynamic acceleration and orientation — so eye and head data sit in the same frame-indexed table as quantity of motion. eye_events lists the events inside the video as EyeEvent spans, eyetracking_rates gives per-bin rates (onset-counted) and means, and gazegram draws where the wearer looked per time bin as the gaze counterpart of a motiongram. Gaze inside blinks is set to missing rather than trusted, and the Cloud face detector's boxes are read but not interpreted. Bound on MgVideo as eyetracking(), gazegrams() and eyetracking_timeline().

Changed

  • Cross-modal names match their level of description — body_audio_coupling() is now motion_audio_coupling() (attribute motion_audio_coupling_figure), and warp_audiovisual_beats() is now warp_audiomotion_beats(). The old names keep working through one release and warn.

Full changelog: https://github.com/fourMs/MGT-python/blob/master/CHANGELOG.md

1.30.0 — the static row: postures found, poses proposed

Choose a tag to compare

@alexarje alexarje released this 01 Sep 14:04

The toolbox's motion/action/gesture ladder gains its static counterpart — position, posture, pose, following Sound Actions and a five-field terminology review (plans/2026-09-01-position-posture-pose-terminology.md).

Added

  • _postures: segment_postures cuts landmark trajectories into held configurations on a Movement–Hold stationarity criterion, in body-normalised torso lengths per second so thresholds transfer between bodies and spaces; key_postures groups the configurations a body returns to, without a fixed catalogue of standing and sitting; average_posture gives a recording's habitual carriage; match_postures proposes a pose by example, as a label attached to matching postures — segmentation kept apart from recognition, exactly as in _actions. Bound on MgVideo as postures_from_pose().
  • Detector-agnostic normalisation: normalise_poses recognises the landmark topology by count (MediaPipe-33, YOLO/COCO-17, OpenPose BODY_25/COCO-18/MPI-15) with an anchors= override, so the higher-level analysis works unchanged when the tracking model is swapped.

Changed

  • Jitter-robust stationarity: configuration change is measured as the displacement of the window-median configuration across a 0.4 s stride, after corpus footage showed a seated dancer's high-confidence knee fluttering within 1.2 px — flat across lags, pure detector jitter — yet reading as 0.15 torso lengths per second on a 43 px foreshortened torso.

Full details in the CHANGELOG.

1.29.0 — the plate distrusts its own refinement, and the associator learns to look

Choose a tag to compare

@alexarje alexarje released this 30 Aug 22:34

The plate distrusts its own refinement, and the associator learns to look.

room_plate no longer hands back a plate its own refinement made worse. On
material where the subject rarely leaves the frame — standstill recordings — the
frames most like the first plate are the ones with the subject in place, and
re-taking the median over them made the subject solid where the full sample had
washed them out. Under a median first pass the kept frames are the ones that
AGREE with the plate, so a material change is concentration, not cleaning:
room_plate now measures that change and returns the unrefined plate with a
warning past the new max_refine_change argument (default 0.02, above a body's
residue and below a body). Measured on a 2012 standstill performance, the old
default changed 3.7% of the plate; the fixed default hands back exactly the
refine=False plate. Ordinary corpora are untouched.

associate_fragments gains appearance: fragment_embeddings collects one
appearance vector per track fragment in a single sequential pass, an ambiguous
crossing is decided by appearance only when one candidate is strictly more
separated than the material's own within-appearance spread, and refusal remains
output — identical appearances decide nothing.

Full details in the CHANGELOG.

1.28.0 — the third family, and fragments that become movers

Choose a tag to compare

@alexarje alexarje released this 30 Aug 15:35

The third family, and fragments that become movers.

RTMPose joins as the third pose extractor on the same trajectory-array contract —
extract_pose_landmarks_rtmpose, riding rtmlib over ONNX runtime with no MMPose
stack, Apache-licensed end to end (pip install musicalgestures[rtmpose]). Its
separate person detector held 100% detection on a dark dance stage where small
single-stage models flickered. All three extractors — MediaPipe, YOLO, RTMPose —
now feed the same detector-agreement tooling, and the posetools docs page carries
the measured which-to-reach-for guidance.

associate_fragments chains identity-track fragments into persistent movers using
position and time only, under an exclusivity constraint and a plausibility bound
measured from the material itself — and it refuses where honesty demands: a
crossing at a fragment boundary becomes a recorded break for a human to
adjudicate, never a guess. The design document records both the method and its own
measured mispredictions.

Full details in the CHANGELOG.

1.27.0 — the qualities of movement, and identities that outlast confidence

Choose a tag to compare

@alexarje alexarje released this 30 Aug 13:55

The qualities of movement, and identities that outlast confidence.

The Effort layer is MGT's own operationalisation of Laban's four factors — Time,
Weight, Space, Flow — as continuous windowed indices, every claim measured before
the docs made it: SPARC implemented from the paper and agreeing with the
originating lab's implementation on the canonical profiles, the tracker-noise
robustness that justifies it, and Laban's eight basic effort actions as
median-anchored proposals in his own vocabulary (via Haga, 2008). The docs' new
Effort page carries the definitions, the grounding, and the limits — including
the measured ones.

YOLO pose gains per-person tracking: extract_pose_tracks_yolo returns every
identity's trajectory separately, and track=True follows the most persistent
body — the cure for per-frame top-confidence selection teleporting between two
dancers, or between a dancer and their life-size projection on a videoconference
screen. And skeleton_timeline draws posture at sampled moments on a real time
axis: stick figures, torso-normalised, honest about tracking gaps.

Full details in the CHANGELOG.

1.26.0 — the last deprecation batch, and a second pair of eyes

Choose a tag to compare

@alexarje alexarje released this 30 Aug 09:31

The last deprecation batch, and a second pair of eyes.

extract_pose_landmarks_yolo is the Ultralytics twin of the MediaPipe extractor, on
the same trajectory-array contract — same decode pipe, same result dictionary — so
two detectors can be compared on a shared clock. The topology is the 17-point COCO
set, the third channel is keypoint confidence, and a zero-confidence keypoint is NaN
rather than a fabricated origin point. Install with pip install musicalgestures[yolo].

motiongram_data speaks x and y like the rest of the toolbox, with the old
"vertical"/"horizontal" values warning until 2.0. This completes the deprecation set:
everything scheduled for removal at 2.0 now warns.

The motion envelope (MgMotionVectorViews.magnitude against .time) is documented
as what it is: the encode's own ~12.6 Hz reference cadence for a tenth of a decode's
cost, median r = 0.82 against exact quantity of motion, 0.57 where little moves,
B-frames excluded by default.

Full details in the CHANGELOG.

1.25.0 — texture, floors, and pages of a time range

Choose a tag to compare

@alexarje alexarje released this 30 Aug 07:03

The room's texture, the floor made affordable, and pages of a time range.

texture_mask says which cells of a picture carry enough texture to trust motion
vectors on: an encoder's motion search is unconstrained where nothing textures the
block, so masking the flattest cells of the room removes motion that was never a
measurement.

The zoomable page plays its recording (player=), builds for any video in one call
(MgVideo.zoompage()), and pages any time range of a session on its own clock
(start_s=) — the cached track is sliced and tier spans shift with it, so one
section of a long recording can be studied alone.

And measuring a noise floor no longer holds every magnitude in memory: BoundedSample
keeps a uniform sample of the stream whatever its length, so the quantile survives
and the 8 GB does not.

Full details in the CHANGELOG.

MGT-python 1.24.0

Choose a tag to compare

@alexarje alexarje released this 29 Aug 11:10

The grams put right, in number and in name.

Motion-vector displacements are corrected for the reference cadence: FFmpeg's source carries only the sign of a vector's reference, so a P-frame following a run of B-frames had reported its whole multi-frame displacement as one frame's — 3× to 4× over on B-heavy encodes. Every motion-vector number on such encodes changes; arrays saved by earlier versions should be re-derived.

The grams are drawn in the classic orientations and named by the position axis each keeps: the x-motiongram keeps horizontal position, shows sideways travel, and is the tall picture with time downward; the y-motiongram keeps vertical position and is the wide one. Every picture-named attribute keeps working with a deprecation warning until 2.0.

multishot(animate=True) writes the chronophotograph as a looping GIF of its own build-up. zoomable_page gains an audio band — waveform and spectrogram on the shared clock — and named, switchable video strips, so a videogram and a motiongram can be compared in place.

extract_tracks_parallel no longer leaks one live ffmpeg per finished chunk; 51 of them once held 23 GB and took a 32 GB machine down mid-extraction.

Full details in the CHANGELOG.

MGT-python 1.23.0

Choose a tag to compare

@alexarje alexarje released this 29 Aug 10:55

Postures over time, and a capability that turned out not to be there.

pose_timeline draws postures and trajectories in three flat views: postures at regular instants side by side, skeletons where they actually stood, and per-region joint angles so a held posture is a flat band — which a posegram cannot show, since it carries speed and a held limb has none. Gaps stay gaps rather than being interpolated into invented posture.

multishot absorbed stroboscope(), which made the same picture a different way; the old name still works, warns, and goes at 2.0. plate() and multishot() are methods like every other view.

MediaPipe segmentation had been silently dead since MediaPipe 0.10 — the Solutions API it asked for was removed, the lookup raised, a bare except swallowed it, and every caller got background subtraction while being told otherwise. Rebuilt on the Tasks API, and the fallback now says so out loud.

Note that posegram's rows are reordered — head, torso, arms, hands, legs — so a posegram made before this release is not comparable row for row with one made after.

Full details in the CHANGELOG.