You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
export_dive_as_coco (server/dive_utils/serializers/kwcoco.py:582, mirrored in client/platform/desktop/backend/serializers/coco.ts) emits info, images, annotations, categories, and conditionally videos. There is no tracks
key on either platform.
Meanwhile each annotation gets four values read from the track, not the
feature — so they are constant across the track and written once per frame:
Field
Server
Client
track_id
kwcoco.py:674
coco.ts:596
prob
kwcoco.py:668
coco.ts:599
dive_confidence_pairs
kwcoco.py:671
coco.ts:600
dive_track_attributes
kwcoco.py:678
coco.ts:602
Three consequences:
There's nowhere to put track-level data. DIVE groups annotations by track_id, the original COCO tracking convention — self-consistent, but it
gives a track no identity of its own. kwcoco made tracks first-class in
v0.7.0 (2023-08-15)
Track-level data is duplicated per frame — a 500-frame track writes its
attributes and confidence pairs 500 times. This is a specific instance of
kwcoco #7
("annotations are often duplicated in every frame of a video").
prob is dense over the whole category list — one float per category per
annotation, for a track that usually has one or two real pairs. Scales as categories × detections, so it worsens as hierarchical types grow the
category count (Integrate Baseline VLM and Hierarchical Models - P6 #1587). Also worth knowing: CocoDataset.union remaps category_id and keypoint_category_id but never prob, so unioning
datasets with different category orders silently misaligns every array.
Proposed changes
Emit a tracks table, unconditionally — every DIVE annotation belongs to a Track (a "detection" is a one-feature track), so there is no DIVE dataset
without tracks. Not gated on emit_video, which keys on whether the source
was a video, not on whether tracks exist.
Move confidence pairs and track attributes onto the track entry.
Stop writing the dense per-annotation prob. It's optional in the schema and category_id/score still gives any reader the primary label.
Read a tracks table on import — currently nothing on either platform reads
one (load_coco_as_tracks_and_attributes, kwcoco.py:437), so a file from
another tool loses its track names. This is the round-trip half of #1089.
What's standardized
The tracks table with {id, name} is standard. No invention.
category_scores and attributes on a track are not. kwcoco's TRACK is
exactly {id, name}, and kwcoco has no track-level classification concept at
all — categories attach to annotations only. So hoisting these is an
efficiency fix, not a standardization one; they remain DIVE extensions
wherever they sit.
Problem
export_dive_as_coco(server/dive_utils/serializers/kwcoco.py:582, mirrored inclient/platform/desktop/backend/serializers/coco.ts) emitsinfo,images,annotations,categories, and conditionallyvideos. There is notrackskey on either platform.
Meanwhile each annotation gets four values read from the track, not the
feature — so they are constant across the track and written once per frame:
track_idkwcoco.py:674coco.ts:596probkwcoco.py:668coco.ts:599dive_confidence_pairskwcoco.py:671coco.ts:600dive_track_attributeskwcoco.py:678coco.ts:602Three consequences:
There's nowhere to put track-level data. DIVE groups annotations by
track_id, the original COCO tracking convention — self-consistent, but itgives a track no identity of its own. kwcoco made tracks first-class in
v0.7.0 (2023-08-15)
Track-level data is duplicated per frame — a 500-frame track writes its
attributes and confidence pairs 500 times. This is a specific instance of
kwcoco #7
("annotations are often duplicated in every frame of a video").
probis dense over the whole category list — one float per category perannotation, for a track that usually has one or two real pairs. Scales as
categories × detections, so it worsens as hierarchical types grow thecategory count (Integrate Baseline VLM and Hierarchical Models - P6 #1587). Also worth knowing:
CocoDataset.unionremapscategory_idandkeypoint_category_idbut neverprob, so unioningdatasets with different category orders silently misaligns every array.
Proposed changes
trackstable, unconditionally — every DIVE annotation belongs to aTrack(a "detection" is a one-feature track), so there is no DIVE datasetwithout tracks. Not gated on
emit_video, which keys on whether the sourcewas a video, not on whether tracks exist.
prob. It's optional in the schema andcategory_id/scorestill gives any reader the primary label.trackstable on import — currently nothing on either platform readsone (
load_coco_as_tracks_and_attributes,kwcoco.py:437), so a file fromanother tool loses its track names. This is the round-trip half of
#1089.
What's standardized
trackstable with{id, name}is standard. No invention.category_scoresandattributeson a track are not. kwcoco'sTRACKisexactly
{id, name}, and kwcoco has no track-level classification concept atall — categories attach to annotations only. So hoisting these is an
efficiency fix, not a standardization one; they remain DIVE extensions
wherever they sit.