feat(sdk): host surfaces for derived objects and builtin field tables (0.36.0)

.claude/skills/plotjuggler-plugin/references/toolbox.md

@@ -78,6 +78,29 @@ PJ_TOOLBOX_PLUGIN(MyToolbox,
   `dataset_source:topic/field` lookup rules are in
   `pj_plugins/docs/toolbox-guide.md` → "Playback, viewport, and owned tabs".
 
+## Detecting host features (SDK 0.36.0)
+
+One rule, in the header comment of `pj_base/plugin_data_api.h`; never test a
+version string:
+
+- ABI service feature: a named `hasX()` on the view
+  (`DataProcessorsHostView::hasTypedRequests()`, `PlotTabHostView::hasSceneTabs()`,
+  `ToolboxHostView::hasCatalogSnapshotV2()`).
+- Flag-bit feature (`PJ_DATA_PROCESSOR_FLAG_INFER_OUTPUTS`): do not probe; an older
+  host rejects the bit. Its floor is the `hasX()` of the slot carrying it.
+- Build-dependent behaviour (Python for on_demand): probe by doing it,
+  `validateScript("on_demand", "python", "return {}")`.
+- Dialog feature (`scene_view`): `hostHas(DialogHostCapability::kEmbedsSceneViews)`
+  inside the dialog.
+- Manifest metadata (`badge`, `custom_topics_editor`): declarative, no probe.
+
+On-demand requests: `createV2` returns `<owner>/<id>/<name>` for object and number
+outputs (read a number series by exactly that string) and the bare name for a
+string output. `submitEvaluation` + `pollEvaluation`: check `coverage.stopped` and
+`coverage.error`; an empty `bundles` list is not "no sample". Lifetime: persistent
+by default, `EPHEMERAL` = preview that undo/redo does not end, `HISTORY_EXEMPT` =
+persisted but outside history. See `toolbox-guide.md` for the full text.
+
 ## Reading a series (Arrow)
 
 ```cpp

CHANGELOG.md

@@ -3,6 +3,180 @@
 All notable changes to `plotjuggler_sdk` are recorded here. Versioning policy is in
 [`CLAUDE.md`](./CLAUDE.md) → "Release Versioning".
 
+## [0.36.0] — Unreleased
+
+Host contract: extended: PJ_toolbox_host_vtable_t::acquire_catalog_snapshot_v2, PJ_data_processors_host_vtable_t::create_data_processor_v2, PJ_data_processors_host_vtable_t::submit_evaluation, PJ_data_processors_host_vtable_t::poll_evaluation, PJ_data_processors_host_vtable_t::release_evaluation, pj.plot_tabs.v1 tail slots (create_tab_v2, attach_topic, detach_topic, focus_tab), PJ_DATA_PROCESSOR_TIME_FLAG_WINDOW, PJ_DATA_PROCESSOR_TIME_FLAG_INSTANT, PJ_DATA_PROCESSOR_FLAG_INFER_OUTPUTS (floor 0.36.0)
+
+- Add `uint64_t reserved[2]` (must be 0) to `PJ_data_processor_output_t`, which is
+  an array element with a fixed stride: the struct grows from 32 to 48 bytes. The
+  C++ wrapper zeroes it; a host rejects an output with nonzero `reserved`.
+- Deprecate the `"<name>:<type>"` output suffix of `create_data_processor`: it
+  still works, new clients use `create_data_processor_v2` with
+  `PJ_data_processor_output_t`; it will be removed in a future version.
+- Evaluation budgets and live-handle / report-byte limits are host policy: `0` =
+  host default, each field is clamped to a host-defined maximum, and a submit is
+  never rejected for asking more (`coverage.stopped` says which budget ended the
+  run). The documented default and cap numbers are removed. Docs only.
+- Document the unknown-value rule of the evaluation report: an unrecognised string
+  value in a key the client knows (e.g. `coverage.stopped`, `outputs[].status`)
+  is an error, never success; `outputs[].type` `"unknown"` is a defined value.
+  The names inferred by `INFER_OUTPUTS` are chosen by the host and must be read
+  from the report. Docs only.
+- Document the optional `done` / `total` keys of the on_demand `series` block of
+  `data_processor_config` (instants evaluated / known total); a client tolerates
+  their absence. Docs only.
+- Document the visibility rule for ephemeral data processors: `list_data_processor_ids`
+  hides them, but `data_processor_config` by exact id answers the owning plugin
+  (hosts no longer reject it), so a plugin can read its preview's `series` progress.
+  Docs only: no ABI change.
+- Document the `chart_auto_zoom` semantics (`setChartAutoZoom`): omitted fits until
+  the user zooms or pans, `true` fits now and resumes auto-fit, `false` keeps the
+  user's view. Docs only: no ABI change.
+- Add `WidgetData::setSceneView` / `setSceneTopics` / `clearSceneView` (keys
+  `scene_view`, `scene_topics`) and the matching `WidgetDataView::sceneView` /
+  `sceneTopics`: a QFrame carrying `scene_view` becomes an embedded 3D/2D object
+  view bound to object topics and following the playback cursor. Hosts without
+  support leave the frame empty. Dialog protocol only: no ABI change.
+- Add `PJ_DATA_PROCESSOR_FLAG_INFER_OUTPUTS` (on_demand): a transient evaluation
+  with no declared outputs infers each output's name and type from the script's
+  returned value and reports them in a root `"outputs"` array of the
+  `poll_evaluation` report; on create/create_v2 the declared outputs are a
+  binding hint learned from such a trial.
+- Add compile-time field tables (`pj_base/builtin/field_table.hpp`) describing
+  the members of builtin object structs, so a generic binder (e.g. a script
+  engine) can read/write any described field by name without per-type glue
+  code. Per-type specializations live in `frame_transforms_fields.hpp`,
+  `image_annotations_fields.hpp`, `point_cloud_fields.hpp`, and
+  `scene_entities_fields.hpp`; `field_table_registry.hpp` exposes
+  `describe(BuiltinObjectType)` to look one up from the runtime tag carried
+  by a `BuiltinObject`. `FrameTransforms`, `ImageAnnotations`, `PointCloud`,
+  and `SceneEntities` were the first four tabled types (the next entry adds four
+  more); `PointCloud`
+  exposes its packed-bytes `data` field through a `kBuffer` descriptor
+  (`buffer<>()`) resolving a `BufferLayout` view rather than a plain
+  get/set pair. Client-side only: no ABI or wire-format change.
+- Add field tables for `Image`, `DepthImage`, `CameraInfo`, and `VideoFrame`
+  (`image_fields.hpp`, `depth_image_fields.hpp`, `camera_info_fields.hpp`,
+  `video_frame_fields.hpp`); `describe(BuiltinObjectType)` now covers 8
+  types (there is no count constant: `FieldTableTest.DescribeCoversExactlyTheTabledTypes`
+  pins the set). `Image::data`/`DepthImage::data`/`VideoFrame::data` each get a
+  `kBuffer` descriptor whose layout is derived from the encoding/format
+  string: a raw `Image` encoding (e.g. "rgb8") or a recognized `DepthImage`
+  encoding ("16UC1"/"32FC1") resolves a static per-pixel `record_step`;
+  a compressed encoding or a `VideoFrame`'s codec bitstream has none, so
+  `record_step`/`record_count` are 0 and the encoding/format string still
+  comes through the buffer's sole channel name. `FieldKind` gains two new
+  cases to describe these structs honestly rather than faking them:
+  `kOptionalNumber` (a nullable number, `has_value()` + `get_number`/
+  `set_number`, for `Image::compressed_depth_min`/`compressed_depth_max`)
+  and a fixed-size `kList` (for a `std::array<double, N>` member — e.g.
+  `CameraInfo::K`/`R`/`P`, `DepthImage::K` — written through the new
+  `list_replace` accessor instead of `list_emplace`/`list_clear`, since the
+  element count never changes). Client-side only: no ABI or wire-format
+  change.
+- Add catalog snapshot v2: `PJ_toolbox_host_vtable_t::acquire_catalog_snapshot_v2`
+  (`ToolboxHostView::catalogSnapshotV2`) returns a second, ABI-VERSIONED
+  snapshot struct (`PJ_catalog_snapshot_v2_t`) carrying the scalar catalog of
+  `acquire_catalog_snapshot` PLUS every object topic — dataset, builtin type,
+  entry count, raw time range (`PJ_object_topic_info_t`). A new struct rather
+  than a tail-append, because object topics are an ARRAY ELEMENT with a FIXED
+  STRIDE: a field that cannot be zero-defaulted needs a new struct + slot,
+  never a layout change to an existing one.
+- Add the typed data-processor request vocabulary (`PJ_data_processor_request_t`,
+  `PJ_data_processor_output_t`, `PJ_evaluation_budget_t`) and
+  `PJ_data_processors_host_vtable_t::create_data_processor_v2`
+  (`DataProcessorsHostView::createV2`): typed outputs, a human-readable label,
+  and — with `PJ_DATA_PROCESSOR_TIME_FLAG_INSTANT` — a pinned evaluation time
+  for an on_demand finding. This is the ABI surface the `kind="on_demand"`
+  doc-comment on `PJ_data_processors_host_vtable_t` previously described as
+  not existing yet; `create_data_processor` (v1) stays valid for the
+  `"<name>:<type>"` output-suffix grammar.
+- Add asynchronous on_demand evaluation: `submit_evaluation`/`poll_evaluation`/
+  `release_evaluation` (`DataProcessorsHostView::submitEvaluation`/
+  `pollEvaluation`/`releaseEvaluation`), gated by the new
+  `PJ_DATA_PROCESSOR_TIME_FLAG_WINDOW`/`PJ_DATA_PROCESSOR_TIME_FLAG_INSTANT`
+  bits. A submitted evaluation returns a handle; a host may complete the work
+  before returning or in the background, and `poll_evaluation` is the only way
+  to read the result either way, as a JSON report
+  (`{"coverage":{...},"bundles":[...]}` — one bundle per requested instant, one
+  entry per output, `*_ns` values as raw int64 dataset nanoseconds).
+- Add `DataProcessorsHostView::hasTypedRequests()`: true iff the host serves
+  `create_data_processor_v2` and `submit_evaluation`/`poll_evaluation`/
+  `release_evaluation`, so a plugin can gate typed-request UI without probing.
+- Scene 3D/2D tabs are kinds of `pj.plot_tabs.v1` tabs: tail slots
+  `create_tab_v2`, `attach_topic`, `detach_topic`, `focus_tab` (C++:
+  `PlotTabHostView::createV2/attachTopic/detachTopic/focus/hasSceneTabs`). The
+  slots are NULL when the host has no scene workspace; the plot `tab_config`
+  JSON is unchanged.
+- Add the `pj_snapshot` object-topic metadata key
+  (`ObjectTopicMetadataBuilder::snapshot`): marks a SceneEntities/
+  ImageAnnotations topic whose every entry is a complete clear-and-replace
+  snapshot, so a stateless consumer may render each entry alone without
+  replaying the topic's history.
+- Add the optional manifest string `badge` (`PluginDescriptor::badge`, "" when
+  absent): a short label a host may show next to objects the plugin creates.
+  Older manifests and hosts are unaffected.
+- Record the `struct_size` rule of the typed data-processor request and of
+  `PJ_evaluation_budget_t` as READ-PREFIX: a host reads the prefix it knows,
+  rejects a `struct_size` below the v1 layout and ACCEPTS a larger one; a field
+  appended later is zero-defaultable and announced by a new `flags`/`time_flags`
+  bit, so an older host rejects the bit, never the size. The previous text
+  (reject any `struct_size` larger than the host's own) would have made the
+  first appended field break every newer plugin on a 0.36 host. The C++
+  wrapper keeps sending `sizeof`, which is correct under this rule.
+- `create_data_processor_v2` / `create_data_processor` (`kind="on_demand"`):
+  `out_topics` now returns `<owner>/<id>/<name>` for number outputs as well as
+  object outputs (the series key of series mode; absent from the catalog when
+  the recipe cannot run in series mode); string outputs stay the bare name.
+  A reader must use the returned string, not rebuild it from the output name.
+- `DataProcessorsHostView::pollEvaluation` returns an error for an unknown
+  evaluation state ("unknown evaluation state N") instead of reading it as
+  pending, which made a caller poll forever.
+- Document `coverage.error` (present only when `coverage.stopped == "error"`)
+  and `coverage.gaps` in the `poll_evaluation` report, plus the 64-handle /
+  64 MiB limits and that polling needs the host event loop to turn.
+- Add `ToolboxHostView::hasCatalogSnapshotV2()`: true iff the host serves
+  `acquire_catalog_snapshot_v2`. One `hasX()` per ABI service feature is the
+  capability rule, stated once in the `plugin_data_api.h` header comment
+  (ABI feature = tail slot behind a `hasX()`; flag-bit feature = no probe, an
+  older host rejects the bit; build-dependent behaviour = probe by doing it;
+  dialog feature = `PJ_dialog_host_info_t::capabilities`; manifest metadata =
+  no probe).
+- Add the dialog host capability `PJ_DIALOG_HOST_EMBEDS_SCENE_VIEWS`
+  (`1 << 5`, C++ `DialogHostCapability::kEmbedsSceneViews`): the host embeds
+  `scene_view` / `scene_topics` frames. `DialogPluginBase` (and so
+  `DialogPluginTyped`) gains `hostHas(capability)`
+  over the host info already delivered by the existing `set_host_info` slot;
+  a host that never delivers it reports 0. A dialog gates its scene UI on the
+  bit instead of probing a different service. No ABI change.
+- Add the optional manifest flag `custom_topics_editor`
+  (`PluginDescriptor::custom_topics_editor`, false when absent): the toolbox
+  that edits the host's user-defined topics. A host keys the "+" button and
+  the ownership of those rows on it instead of a hard-coded plugin id.
+- Add `sdk::kDerivedMetadataKey` (`"pj_derived"`) and `sdk::kDerivedOnDemandValue`
+  (`"on_demand"`) in `object_topic_metadata.hpp`: the topic metadata key a host
+  sets on object topics derived by an on_demand data processor.
+- Add `sdk::unprojectPixel` (`depth_image_utils.hpp`): unprojects a rectified
+  pixel with positive metric depth through a pinhole `K`; rejects singular or
+  non-finite intrinsics and skew/projective terms. Client-side only.
+- RENAME (the 0.36 line is unreleased, so nothing shipped breaks):
+  `PlotTabHostView::createV2` -> `createTabV2` and `focus` -> `focusTab`, so
+  they no longer read like `DataProcessorsHostView::createV2` or a UI focus.
+  `feature_floors.json` keys follow.
+- Every slot, struct and wrapper added in 0.36 now carries `@since 0.36.0`.
+- Docs: the contract text now states, once each, the lifetime of a processor by
+  flag and kind (persistent / EPHEMERAL / HISTORY_EXEMPT / pinned), what each
+  kind does with `label`, WINDOW and INSTANT, that Python is an optional
+  per-host on_demand language probed with `validateScript`, that a plot-tab
+  re-create keeps the host's layout and clears the curves, that `attach_topic`
+  is idempotent, that the two halves of catalog snapshot v2 are not atomic,
+  that a `scene_view` frame exists in panels only and a failed attach is not
+  retried until the topic set changes, and that the empty-badge fallback is host
+  policy (at most 8 characters advised). `data_processor_config` reports
+  `history_exempt` for every kind, not only transforms and markers.
+- ImageAnnotations wire carries the top-level timestamp and image_topic;
+  additive, old readers skip them.
+
 ## [0.35.0]
 
 Host contract: unchanged (no floor impact) — a client-side codec helper, no new

VERSION

@@ -1 +1 @@
-0.35.0
+0.36.0

docs/builtin_type.md

@@ -673,6 +673,56 @@ splice form and decodes with an empty span; the host runs `validateGridMap()`
 right after attaching the spliced bytes, and consumers that index cells run it
 again defensively.
 
+## Field tables
+
+`pj_base/builtin/field_table.hpp` describes a builtin struct's members at
+compile time — name, shape (`FieldKind`), and type-erased get/set accessors —
+so a generic consumer (e.g. a script binder) can read or write any described
+field by name without hand-written per-type glue. A struct opts in with a
+`FieldTable<T>` specialization built from `field<&T::member>("member")` calls
+(`buffer<>()` for a packed record buffer like `PointCloud::data`); nested
+structs and lists of described structs link to their own table through
+`FieldDescriptor::nested`, so a consumer walks an arbitrarily deep struct tree
+with one generic recursive routine. `field_table_registry.hpp` exposes
+`describe(BuiltinObjectType)` to look up a table from the runtime tag a
+`BuiltinObject` carries.
+
+`FrameTransforms`, `ImageAnnotations`, `PointCloud`, `SceneEntities`, `Image`,
+`DepthImage`, `CameraInfo`, and `VideoFrame` are tabled today
+(`frame_transforms_fields.hpp`, `image_annotations_fields.hpp`,
+`point_cloud_fields.hpp`, `scene_entities_fields.hpp`, `image_fields.hpp`,
+`depth_image_fields.hpp`, `camera_info_fields.hpp`, `video_frame_fields.hpp`;
+`describe()` in `field_table_registry.hpp` returns a table for exactly these eight). `Image::data`,
+`DepthImage::data`, and `VideoFrame::data` each expose their packed pixel /
+bitstream bytes through a `kBuffer` descriptor the same way `PointCloud::data`
+does, with the per-record layout derived from the encoding/format string
+rather than a channel list (see each header's doc comment for how a raw vs.
+compressed encoding differs). `CameraInfo`'s `K`/`R`/`P` (and `DepthImage`'s
+`K`) are `std::array<double, N>` members, described as a fixed-size `kList`
+written through `list_replace` rather than `list_emplace`/`list_clear` (the
+element count never changes); `Image::compressed_depth_min`/
+`compressed_depth_max` are `kOptionalNumber` — a nullable number read/written
+like `kNumber` but guarded by `has_value()`. A field table is hand-maintained
+alongside its struct: adding, renaming, or retyping a member requires updating the matching
+`FieldTable<T>` specialization in the same change, or the table silently
+drifts from the struct it claims to describe. `pj_base/tests/field_table_test.cpp`
+enforces this — its generic copy-through-table round trip is compared against
+the struct's own `operator==` and its canonical wire-codec bytes, so a member
+the table forgot to list shows up as a round-trip mismatch rather than a
+silent gap.
+
+An object topic carrying a field-tabled type can mark itself with the
+`pj_snapshot` object-topic metadata key (`PJ::sdk::kSnapshotMetadataKey`,
+`ObjectTopicMetadataBuilder::snapshot()`, SDK 0.36.0). Value `"true"` means
+every entry on the topic is a complete clear-and-replace snapshot — a
+generic field-table consumer may render each entry alone, without replaying
+the topic's history to reconstruct state first. This matters for
+SceneEntities/ImageAnnotations producers in particular, whose entries can
+otherwise be incremental (add/update/delete deltas); a producer that instead
+republishes its whole set each time sets this flag so a stateless consumer
+(and `set_object_topic_retention(topic, 1)`, which keeps only the latest
+snapshot) can treat each entry as the full picture.
+
 ## Conversion Examples
 
 | Source type | Canonical builtin type | Conversion intent |

docs/image_annotations_format.md

@@ -66,16 +66,17 @@ the renderable annotation fields and ignores the rest:
 | `ImageAnnotations.circles` | Mapped to `ImageAnnotations::circles`. |
 | `ImageAnnotations.points` | Mapped to `ImageAnnotations::points`. |
 | `ImageAnnotations.texts` | Mapped to `ImageAnnotations::texts`. |
-| Top-level `timestamp` | Not serialized or decoded today. |
+| Top-level `timestamp` | Serialized when non-zero, decoded into `ImageAnnotations::timestamp`. |
+| `image_topic` (field 6, string) | Serialized when non-empty, decoded into `ImageAnnotations::image_topic`. |
 | Top-level `metadata` | Not serialized or decoded today. |
 | Per-annotation `timestamp` | Not serialized or decoded today. |
 | Per-annotation `metadata` | Not serialized or decoded today. |
 | `TextAnnotation.background_color` | Not represented by the SDK type; skipped on decode and not emitted on encode. |
 
-`PJ::sdk::ImageAnnotations::image_topic` is also not part of this payload. It is
-runtime association metadata used by PlotJuggler to attach overlays to an image
-stream. Adapters that need to preserve it across storage or transport must store
-it outside the `PJ.ImageAnnotations` bytes.
+`PJ::sdk::ImageAnnotations::image_topic` is part of the payload: it is written
+as field 6 when non-empty, so runtime association between an annotation set
+and its image stream can be carried on the wire. A reader predating this
+addition ignores the unknown field.
 
 ## Codec Rules
 

pj_base/CMakeLists.txt

@@ -186,6 +186,7 @@ if(PJ_BUILD_TESTS)
     tests/frame_transforms_codec_test.cpp
     tests/image_annotations_codec_test.cpp
     tests/image_annotations_decoder_test.cpp
+    tests/field_table_test.cpp
     tests/media_metadata_test.cpp
     tests/object_topic_metadata_test.cpp
     tests/push_message_test.cpp