diff --git a/.claude/skills/plotjuggler-plugin/SKILL.md b/.claude/skills/plotjuggler-plugin/SKILL.md index 93d737ef..798f525d 100644 --- a/.claude/skills/plotjuggler-plugin/SKILL.md +++ b/.claude/skills/plotjuggler-plugin/SKILL.md @@ -168,6 +168,56 @@ The host routes payloads by these case-sensitive names. DataSource may add `"file_extensions": [".csv"]`. `pj_emit_plugin_manifest` (the pre-0.25 name) still works but is deprecated. +### `min_sdk_required` — the host floor, not the build pin + +`min_sdk_required` declares the oldest **host** SDK contract your plugin +functions on. It is independent of the SDK you build against: `pj_base` / +`pj_source` are static libraries compiled into your plugin, so adopting a new +helper (`parseIso8601Utc`, `sliderToWindow`, a codec) **never** raises the +floor. Only **host-contract** surfaces do — runtime-host tail slots, named +services, wire contracts — and the authority for which release introduced each +is `feature_floors.json`, shipped in the package at +`share/plotjuggler_sdk/feature_floors.json`. + +When you adopt a surface newer than your floor, pick one: + +- **Raise `min_sdk_required`** to that surface's release — mandatory for a + non-negotiated (hard protocol) surface, and right whenever the feature is + essential to what your plugin is for. The reverse is NOT symmetric: the + checker sees only runtime host surfaces, so a floor above anything it + matched may still be correct (compile-time SDK APIs are outside the check) + — never lower a floor just because the checker matched nothing. +- **Declare the degraded range** — when the surface is runtime-negotiated and + the wrapper's error return is acceptable (ignore it and carry on). Two + manifest fields, required together and forbidden when nothing exceeds the + floor: + +```json +"min_sdk_required": "0.28.0", +"suggested_sdk_version": "0.32.0", +"floor_test": "McapDatasetMetadata.MetadataPublishDegradesCleanlyOnFloorLevelHost" +``` + +`suggested_sdk_version` is the full-feature floor (exactly the max introducing +release over every host surface you use — the checker validates it), so hosts +can tell users which PlotJuggler unlocks your whole feature set. `floor_test` +names one gtest, existing in your `tests/`, proving the DEGRADED PATH works +against a floor-level host — drive it with a `struct_size`-truncated host +vtable and poison the slots past the truncation so a missing guard fails the +test. Helper-level granularity is acceptable when the full entry point needs a +live host; the test comment must then state the routing fact (which entry +point routes through the helper). Neither field is ever a gate: +`min_sdk_required` remains the only admission criterion, and +`min_plotjuggler_version` is deprecated. + +Check your own plugin with the tool the SDK package ships next to the table: + +```bash +python3 /share/plotjuggler_sdk/feature_floor_check.py \ + --table /share/plotjuggler_sdk/feature_floors.json \ + --manifest my_plugin/manifest.json --sources my_plugin +``` + ## Step 6 — The rules that silently break a plugin These cut across all families. The family references add family-specific traps diff --git a/.gitignore b/.gitignore index 96461698..ebb48291 100644 --- a/.gitignore +++ b/.gitignore @@ -20,3 +20,4 @@ pj_media/testdata/ # git worktrees live here by policy (see the global CLAUDE.md) .worktrees/ +__pycache__/ diff --git a/CHANGELOG.md b/CHANGELOG.md index ccd3dd79..f76077cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,25 @@ All notable changes to `plotjuggler_sdk` are recorded here. Versioning policy is in [`CLAUDE.md`](./CLAUDE.md) → "Release Versioning". +## [0.33.0] — unreleased + +Host contract: unchanged (no floor impact) — this release adds the floor +table itself and the `suggested_sdk_version` manifest field, no new host surfaces. + +- Ship `share/plotjuggler_sdk/feature_floors.json`: the host-contract surface + table (surface → introducing release, runtime-negotiated or not) that a + plugin floor checker validates `min_sdk_required` against. A new host + surface (tail slot, versioned service id, wire flag) must be classified in + the table in the same change — `tools/feature_floors/check_feature_floors.py` + runs in CI and fails on any unclassified surface. +- New optional manifest fields `suggested_sdk_version` (decoded into + `PluginDescriptor::suggested_sdk_version`) and `floor_test`: a plugin whose + used surfaces exceed its functioning floor declares the full-feature floor + (validated by the floor checker to equal max introducing release over its + matched surfaces) plus the gtest proving it functions against a floor-level + host. `min_sdk_required` remains the only admission gate; + `min_plotjuggler_version` is deprecated (host-side removal lands in PJ4). + ## [0.32.0] - Add the `set_dataset_metadata` tail slot to the data-source runtime host diff --git a/CLAUDE.md b/CLAUDE.md index c02efb2b..5405d2e8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -73,6 +73,25 @@ Propose a release only according to plugin impact: Tail-append optional slots and gate them by `struct_size`; keep minimum vtable sizes frozen. `abidiff` must show additions only for a MINOR. + +The SDK version is **build identity only** — it names what a plugin compiled +against, never what host it runs on. A release that only adds client-side +helpers (static-library code compiled into the plugin) does NOT change runtime +host compatibility and never mints a `pj_base/feature_floors.json` entry; only +host-contract extensions (host-vtable tail slots, versioned service ids, wire +contracts) do, and those MUST be classified there in the same PR (CI-enforced +by `tools/feature_floors/check_feature_floors.py`). The version stays one +monotonic identity; `feature_floors.json` carries the per-release compatibility +semantics; `PJ_ABI_VERSION` remains the orthogonal hard-break counter. +The plugin compatibility contract is `min_sdk_required` (the functioning +floor, the only admission gate) plus the informational +`suggested_sdk_version` (full-feature floor) and `abi_major` for hard +breaks; `min_plotjuggler_version` is deprecated — the decoder keeps reading +it for back-compat, and the host-side gate removal happens in PJ4. +Changelog convention: every release section carries one `Host contract:` line — +either `unchanged (no floor impact)` or `extended: (floor X.Y.Z)`; +the floor checker requires the `extended` form whenever a surface newer than +the last released version exists. `pj_base/abi/baseline.abi` is refreshed only for an intentional MAJOR break. See [ABI evolution rules](pj_plugins/docs/ARCHITECTURE.md#0a-abi-stability-and-evolution-rules-v5). diff --git a/CMakeLists.txt b/CMakeLists.txt index f9d77a11..45421984 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -113,6 +113,13 @@ find_package(FastFloat REQUIRED) if(PJ_BUILD_TESTS) enable_testing() + # Host-surface classification gate: a new tail slot / service id / wire flag + # must be classified in pj_base/feature_floors.json in the same change. + find_package(Python3 QUIET COMPONENTS Interpreter) + if(Python3_Interpreter_FOUND) + add_test(NAME feature_floors_check + COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/tools/feature_floors/check_feature_floors.py) + endif() endif() add_subdirectory(pj_base) @@ -145,6 +152,12 @@ if(PJ_INSTALL_SDK) DESTINATION share/plotjuggler_sdk/docs ) + # Host-surface floor table + the reusable checker that consumes it: the + # machine-readable authority (and tool) a plugin validates min_sdk_required + # against (see tools/feature_floors/). + install(FILES pj_base/feature_floors.json tools/feature_floors/feature_floor_check.py + DESTINATION share/plotjuggler_sdk) + # Fixture sources come from the authoritative PJ_SDK_TEST_FIXTURES list # (cmake/PjSdkTestFixtures.cmake, included above). foreach(_fixture IN LISTS PJ_SDK_TEST_FIXTURES) diff --git a/pj_base/feature_floors.json b/pj_base/feature_floors.json new file mode 100644 index 00000000..92c8c77b --- /dev/null +++ b/pj_base/feature_floors.json @@ -0,0 +1,125 @@ +{ + "schema_version": 1, + "minimum_supported_floor": "0.28.0", + "since": { + "0.30.0": { + "PJ_INGEST_COMPLETION_FLAGS_V1_MASK": "", + "PJ_INGEST_COMPLETION_FLAG_ATTESTS_EMPTY_TOPICS": "", + "PJ_INGEST_COMPLETION_FLAG_NONE": "", + "completeIngest": "PJ_data_source_runtime_host_vtable_t::complete_ingest", + "discardParserIngest": "PJ_toolbox_runtime_host_vtable_t::discard_parser_ingest" + }, + "0.32.0": { + "setDatasetMetadata": "PJ_data_source_runtime_host_vtable_t::set_dataset_metadata" + } + }, + "baseline": [ + "PJ_DATA_PROCESSOR_FLAG_EPHEMERAL", + "PJ_DESCRIPTOR_IMPORT_START_FLAGS_V1_MASK", + "PJ_DESCRIPTOR_IMPORT_START_FLAG_NONE", + "PJ_MODULE_PARSE_INPUT_FLAG_HAS_TIMESTAMP", + "PJ_PARSER_ROUTE_FLAG_OBJECT_V1", + "PJ_PARSER_ROUTE_FLAG_SCALAR_V1", + "PJ_colormap_registry_vtable_t::register_map", + "PJ_colormap_registry_vtable_t::unregister_map", + "PJ_data_processors_host_vtable_t::create_data_processor", + "PJ_data_processors_host_vtable_t::data_processor_config", + "PJ_data_processors_host_vtable_t::list_data_processor_ids", + "PJ_data_processors_host_vtable_t::remove_data_processor", + "PJ_data_processors_host_vtable_t::validate_data_processor_script", + "PJ_data_source_runtime_host_vtable_t::attach_source_record", + "PJ_data_source_runtime_host_vtable_t::ensure_parser_binding", + "PJ_data_source_runtime_host_vtable_t::is_stop_requested", + "PJ_data_source_runtime_host_vtable_t::list_available_encodings", + "PJ_data_source_runtime_host_vtable_t::notify_available_topics", + "PJ_data_source_runtime_host_vtable_t::notify_state", + "PJ_data_source_runtime_host_vtable_t::progress_finish", + "PJ_data_source_runtime_host_vtable_t::progress_start", + "PJ_data_source_runtime_host_vtable_t::progress_update", + "PJ_data_source_runtime_host_vtable_t::push_message", + "PJ_data_source_runtime_host_vtable_t::report_message", + "PJ_data_source_runtime_host_vtable_t::request_stop", + "PJ_data_source_runtime_host_vtable_t::show_message_box", + "PJ_object_read_host_vtable_t::entry_count", + "PJ_object_read_host_vtable_t::get_bytes", + "PJ_object_read_host_vtable_t::list_topics", + "PJ_object_read_host_vtable_t::lookup_topic", + "PJ_object_read_host_vtable_t::lookup_topic_on_dataset", + "PJ_object_read_host_vtable_t::read_latest_at", + "PJ_object_read_host_vtable_t::release_bytes", + "PJ_object_read_host_vtable_t::time_range", + "PJ_object_read_host_vtable_t::topic_metadata", + "PJ_object_write_host_vtable_t::push_lazy", + "PJ_object_write_host_vtable_t::push_owned", + "PJ_object_write_host_vtable_t::register_topic", + "PJ_object_write_host_vtable_t::set_retention_budget", + "PJ_parser_object_write_host_vtable_t::push_lazy", + "PJ_parser_object_write_host_vtable_t::push_owned", + "PJ_parser_runtime_host_vtable_t::report_diagnostic", + "PJ_parser_write_host_vtable_t::append_arrow_stream", + "PJ_parser_write_host_vtable_t::append_bound_record", + "PJ_parser_write_host_vtable_t::append_record", + "PJ_parser_write_host_vtable_t::ensure_field", + "PJ_playback_host_vtable_t::get_state", + "PJ_playback_host_vtable_t::pause", + "PJ_playback_host_vtable_t::play", + "PJ_playback_host_vtable_t::seek", + "PJ_playback_host_vtable_t::set_playback_rate", + "PJ_playback_host_vtable_t::to_display_time", + "PJ_playback_host_vtable_t::to_display_time_for_source", + "PJ_plot_tab_host_vtable_t::add_curve", + "PJ_plot_tab_host_vtable_t::clear_tab", + "PJ_plot_tab_host_vtable_t::close_tab", + "PJ_plot_tab_host_vtable_t::create_tab", + "PJ_plot_tab_host_vtable_t::list_tab_ids", + "PJ_plot_tab_host_vtable_t::remove_curve", + "PJ_plot_tab_host_vtable_t::tab_config", + "PJ_service_registry_vtable_t::get_service", + "PJ_settings_store_vtable_t::contains", + "PJ_settings_store_vtable_t::get_string", + "PJ_settings_store_vtable_t::get_string_list", + "PJ_settings_store_vtable_t::remove_key", + "PJ_settings_store_vtable_t::set_string", + "PJ_settings_store_vtable_t::set_string_list", + "PJ_source_promotion_host_vtable_t::promote_to_file_source", + "PJ_source_write_host_vtable_t::append_arrow_stream", + "PJ_source_write_host_vtable_t::append_bound_record", + "PJ_source_write_host_vtable_t::append_record", + "PJ_source_write_host_vtable_t::ensure_field", + "PJ_source_write_host_vtable_t::ensure_topic", + "PJ_toolbox_host_vtable_t::acquire_catalog_snapshot", + "PJ_toolbox_host_vtable_t::append_arrow_stream", + "PJ_toolbox_host_vtable_t::append_bound_record", + "PJ_toolbox_host_vtable_t::append_record", + "PJ_toolbox_host_vtable_t::create_data_source", + "PJ_toolbox_host_vtable_t::ensure_field", + "PJ_toolbox_host_vtable_t::ensure_topic", + "PJ_toolbox_host_vtable_t::push_owned_object", + "PJ_toolbox_host_vtable_t::read_series_arrow", + "PJ_toolbox_host_vtable_t::register_object_topic", + "PJ_toolbox_host_vtable_t::register_object_topic_on_dataset", + "PJ_toolbox_host_vtable_t::set_object_topic_retention", + "PJ_toolbox_runtime_host_vtable_t::create_parser_ingest", + "PJ_toolbox_runtime_host_vtable_t::notify_data_changed", + "PJ_toolbox_runtime_host_vtable_t::release_parser_ingest", + "PJ_toolbox_runtime_host_vtable_t::report_message", + "PJ_viewport_host_vtable_t::zoom_reset", + "PJ_viewport_host_vtable_t::zoom_to_time_range", + "pj.colormap.v1", + "pj.data_processors.v1", + "pj.parser_object_write.v1", + "pj.parser_runtime.v1", + "pj.parser_write.v1", + "pj.playback.v1", + "pj.plot_tabs.v1", + "pj.runtime.v1", + "pj.settings.v1", + "pj.source_object_write.v1", + "pj.source_promotion.v1", + "pj.source_write.v1", + "pj.toolbox_object_read.v1", + "pj.toolbox_runtime.v1", + "pj.toolbox_write.v1", + "pj.viewport.v1" + ] +} diff --git a/pj_plugins/docs/ARCHITECTURE.md b/pj_plugins/docs/ARCHITECTURE.md index d039c523..2e511cdc 100644 --- a/pj_plugins/docs/ARCHITECTURE.md +++ b/pj_plugins/docs/ARCHITECTURE.md @@ -5,6 +5,14 @@ Seven rules the loader and every plugin author rely on. Breaking any of these is an ABI break and requires a future `PJ_ABI_VERSION` bump. +One classification obligation rides alongside these rules: **a new host +surface — a host-vtable tail slot, a versioned service/extension id, or a +wire-contract flag — must be classified in `pj_base/feature_floors.json` in +the same PR.** CI enforces it (`tools/feature_floors/check_feature_floors.py` +diffs the extracted host-surface inventory against the checked-in snapshot), +and the installed copy of the table is what downstream plugin repositories +validate `min_sdk_required` against. + 1. **Boot-level ABI symbol.** Every plugin .so exports `pj_plugin_abi_version` as a `uint32_t` symbol independent of any vtable. The host `dlsym`s it BEFORE fetching the family vtable; diff --git a/pj_plugins/include/pj_plugins/host/plugin_catalog.hpp b/pj_plugins/include/pj_plugins/host/plugin_catalog.hpp index cd9fbbc0..86a02e9c 100644 --- a/pj_plugins/include/pj_plugins/host/plugin_catalog.hpp +++ b/pj_plugins/include/pj_plugins/host/plugin_catalog.hpp @@ -55,6 +55,12 @@ struct PluginDescriptor { /// Optional SDK contract floor; "" means undeclared. This is a hard host-side /// load gate, distinct from both the application release and the compile SDK. std::string min_sdk_required; + /// Optional full-feature floor: the SDK release at which every capability + /// this plugin uses is available ("" in manifests that do not declare it). + /// min_sdk_required remains the FUNCTIONING floor; a host between the two + /// runs the plugin with some optional features inactive. Informational, for + /// host degradation display — never an admission criterion. + std::string suggested_sdk_version; }; /// Diagnostic for a candidate DSO that could not produce a valid descriptor. diff --git a/pj_plugins/src/plugin_catalog.cpp b/pj_plugins/src/plugin_catalog.cpp index dbde92e5..7bff9483 100644 --- a/pj_plugins/src/plugin_catalog.cpp +++ b/pj_plugins/src/plugin_catalog.cpp @@ -257,6 +257,10 @@ Expected decodeManifest( if (!min_plotjuggler_version) { return unexpected(min_plotjuggler_version.error()); } + auto suggested_sdk_version = optional_string("suggested_sdk_version"); + if (!suggested_sdk_version) { + return unexpected(suggested_sdk_version.error()); + } auto min_sdk_required = optional_string("min_sdk_required"); if (!min_sdk_required) { return unexpected(min_sdk_required.error()); @@ -283,6 +287,7 @@ Expected decodeManifest( d.category = *category; d.min_plotjuggler_version = *min_plotjuggler_version; d.min_sdk_required = *min_sdk_required; + d.suggested_sdk_version = *suggested_sdk_version; d.file_extensions = *file_extensions; d.capabilities = *capabilities; diff --git a/pj_plugins/tests/plugin_catalog_test.cpp b/pj_plugins/tests/plugin_catalog_test.cpp index 89d4fb2b..65a72929 100644 --- a/pj_plugins/tests/plugin_catalog_test.cpp +++ b/pj_plugins/tests/plugin_catalog_test.cpp @@ -280,6 +280,21 @@ TEST_F(PluginCatalogTest, MinSdkRequiredDefaultsToEmptyWhenAbsent) { EXPECT_TRUE(descriptor->min_sdk_required.empty()); } +TEST_F(PluginCatalogTest, SuggestedSdkVersionRoundTripsAndDefaultsEmpty) { + // Informational full-feature floor: round-trips when declared, and a + // manifest without the field still decodes (back-compat). + auto with_field = decodeManifest( + "static:sdk-test", PluginFamily::kDataSource, + R"({"id":"sdk-test","name":"SDK Test","version":"1.0.0","suggested_sdk_version":"0.33.0"})"); + ASSERT_TRUE(with_field.has_value()) << with_field.error(); + EXPECT_EQ(with_field->suggested_sdk_version, "0.33.0"); + + auto without_field = decodeManifest( + "static:sdk-test", PluginFamily::kDataSource, R"({"id":"sdk-test","name":"SDK Test","version":"1.0.0"})"); + ASSERT_TRUE(without_field.has_value()) << without_field.error(); + EXPECT_TRUE(without_field->suggested_sdk_version.empty()); +} + TEST_F(PluginCatalogTest, MinSdkRequiredAcceptsEmptyString) { auto descriptor = decodeManifest( "static:sdk-test", PluginFamily::kDataSource, diff --git a/test_sdk_install.sh b/test_sdk_install.sh index cc50eb04..d9c146df 100755 --- a/test_sdk_install.sh +++ b/test_sdk_install.sh @@ -140,5 +140,32 @@ if nm -C "$STAGING_DIR/lib/libpj_base.a" | grep -E " [TWuVvW] " | grep -q "fmt:: exit 1 fi +# --------------------------------------------------------------------------- +# 7. Verify the feature-floors table + checker are installed and work +# --------------------------------------------------------------------------- + +echo "" +echo "--- Step 7: Verify feature_floors.json and feature_floor_check.py install ---" + +FLOORS="$STAGING_DIR/share/plotjuggler_sdk/feature_floors.json" +FLOOR_CHECK="$STAGING_DIR/share/plotjuggler_sdk/feature_floor_check.py" +for f in "$FLOORS" "$FLOOR_CHECK"; do + if [[ ! -f "$f" ]]; then + echo "ERROR: $f not installed" + exit 1 + fi +done +# The installed checker is the one parser of the installed table: loading the +# table through it validates both, in the exact combination users receive. +python3 -c " +import importlib.util, pathlib, sys +spec = importlib.util.spec_from_file_location('feature_floor_check', sys.argv[1]) +module = importlib.util.module_from_spec(spec) +sys.modules['feature_floor_check'] = module +spec.loader.exec_module(module) +table = module.load_surface_table(pathlib.Path(sys.argv[2])) +assert table.surfaces, 'empty feature_floors.json' +" "$FLOOR_CHECK" "$FLOORS" + echo "" echo "=== plotjuggler_sdk install test PASSED ===" diff --git a/tools/feature_floors/check_feature_floors.py b/tools/feature_floors/check_feature_floors.py new file mode 100644 index 00000000..e75da95c --- /dev/null +++ b/tools/feature_floors/check_feature_floors.py @@ -0,0 +1,118 @@ +#!/usr/bin/env python3 +# Copyright 2026 Davide Faconti +# SPDX-License-Identifier: Apache-2.0 +"""CI gate for the host-surface inventory and feature_floors.json. + +Fails when a host surface exists in the headers but is not classified, so a +new tail slot / service / flag cannot merge without its floor entry. Table +schema validation itself lives in feature_floor_check.py (the shipped, +reusable parser); this gate layers the repo-side obligations on top: +snapshot/classification consistency, unique release assignment, and the +changelog coupling. Exit 0 = consistent. +""" + +import re +import sys +from collections import Counter +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from extract_host_surfaces import extract # noqa: E402 +from feature_floor_check import CheckError, format_version, load_surface_table, parse_version # noqa: E402 + + +def _unreleased_changelog_section(changelog: str) -> str: + """The text of the unreleased section, or empty when none exists. + + Restricting the 'Host contract:' obligation to this section keeps a + historical extension line from satisfying a new release's duty. Residual + gap, accepted: bumping VERSION to the surface's release in the same PR + retires the obligation — the release commit is where that is reviewed. + """ + headings = list(re.finditer(r"^## .*$", changelog, flags=re.MULTILINE)) + for index, heading in enumerate(headings): + if "unreleased" in heading.group(0).lower(): + start = heading.end() + end = headings[index + 1].start() if index + 1 < len(headings) else len(changelog) + return changelog[start:end] + return "" + + +def main() -> int: + repo = Path(__file__).resolve().parents[2] + errors: list[str] = [] + + inventory = extract(repo) + snapshot_path = repo / "tools/feature_floors/host_surfaces.snapshot" + snapshot = snapshot_path.read_text().split() + if inventory != snapshot: + added = sorted(set(inventory) - set(snapshot)) + removed = sorted(set(snapshot) - set(inventory)) + for surface in added: + errors.append( + f"NEW host surface not classified: {surface}\n" + f" -> add it to pj_base/feature_floors.json (under 'since'/, " + f"or 'baseline' if it predates the minimum supported floor)\n" + f" -> then refresh the snapshot: tools/feature_floors/extract_host_surfaces.py --write" + ) + for surface in removed: + errors.append(f"host surface removed from headers but still in snapshot: {surface}") + if not added and not removed: + errors.append("snapshot is stale (ordering): refresh with extract_host_surfaces.py --write") + + table_path = repo / "pj_base/feature_floors.json" + sdk_version = parse_version((repo / "VERSION").read_text().strip(), "VERSION file") + try: + table = load_surface_table(table_path) + except CheckError as exc: + print(f"feature-floors check FAILED:\n{exc}") + return 1 + + above_floor = [ + surface for surface in table.surfaces.values() if surface.identifier not in table.baseline + ] + + # Unique release assignment: the parser already rejects an identifier in + # two groups and duplicate baseline entries; a declaration reused under a + # second identifier would still assign two versions to one ABI entity. + declaration_counts = Counter(surface.declaration for surface in above_floor) + declaration_counts.update(table.baseline) + for declaration, count in sorted(declaration_counts.items()): + if count > 1: + errors.append(f"surface declaration classified {count} times: {declaration}") + + # since may exceed VERSION during development: a surface added for the + # next, not-yet-released version. That requires the changelog's unreleased + # section to declare the host-contract extension. + unreleased = sorted( + surface.identifier for surface in above_floor if surface.since > sdk_version + ) + if unreleased: + section = _unreleased_changelog_section((repo / "CHANGELOG.md").read_text()) + if "Host contract: extended" not in section: + errors.append( + "surfaces newer than VERSION exist (" + ", ".join(unreleased) + ") but the " + "unreleased CHANGELOG.md section has no 'Host contract: extended: ...' line" + ) + + # Every inventoried surface is classified exactly once: as some surface's + # declaration, or as baseline. Nothing is silently ignored. + classified = {surface.declaration for surface in above_floor} | set(table.baseline) + for surface in snapshot: + if surface not in classified: + errors.append(f"snapshot surface neither a declaration nor baseline: {surface}") + for surface in sorted(classified - set(snapshot)): + errors.append(f"classified surface no longer in the snapshot: {surface}") + + if errors: + print("feature-floors check FAILED:\n" + "\n".join(errors)) + return 1 + print( + f"feature-floors check OK: {len(snapshot)} surfaces, {len(above_floor)} above-floor entries " + f"(minimum {format_version(table.minimum_supported_floor)})" + ) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/feature_floors/extract_host_surfaces.py b/tools/feature_floors/extract_host_surfaces.py new file mode 100644 index 00000000..5c2f0530 --- /dev/null +++ b/tools/feature_floors/extract_host_surfaces.py @@ -0,0 +1,216 @@ +#!/usr/bin/env python3 +# Copyright 2026 Davide Faconti +# SPDX-License-Identifier: Apache-2.0 +"""Inventory the SDK's HOST-CONTRACT surfaces from the protocol headers. + +The inventory is what feature_floors.json must classify: every surface a +plugin can require from the HOST at runtime. Three kinds are extracted: + + 1. Callable members of host-side vtables (top-level function-pointer + members, including typedef'd callable types) — emitted as + "Struct::member". Callback parameters nested inside a member's + parameter list are NOT members and are skipped. + 2. Versioned host-served service ids ("pj..v"). Plugin-provided + extension ids are excluded: the HOST queries those from the plugin, so + an older host simply never asking for a new one degrades by + construction. Every discovered id must be classified in one of the two + lists below — an unknown id fails the extraction. + 3. PJ_*_FLAG* macros in the protocol headers (wire-contract flags a host + must understand). + +Plugin-side vtables (PJ_data_source_vtable_t, ...) are deliberately absent: +the HOST calls those, so an old host not calling a new plugin slot degrades +by construction and never constrains a plugin's min_sdk_required. +Client-side static-library helpers are equally absent — they travel inside +the plugin binary. Every discovered vtable must be classified as host-side +or plugin-side below; an unknown one fails the extraction. + +Run with --write to refresh host_surfaces.snapshot; CI diffs a fresh +extraction against the checked-in snapshot (see check_feature_floors.py). +""" + +import argparse +import re +import sys +from pathlib import Path + +# Host-side vtables beyond the PJ_*host_vtable_t naming convention. +EXTRA_HOST_VTABLES = { + "PJ_service_registry_vtable_t", + "PJ_colormap_registry_vtable_t", + "PJ_settings_store_vtable_t", +} +# Plugin-implemented vtables: the host calls these, they never constrain floors. +PLUGIN_SIDE_VTABLES = { + "PJ_data_source_vtable_t", + "PJ_message_parser_vtable_t", + "PJ_toolbox_vtable_t", + "PJ_joinable_job_vtable_t", +} +# Versioned ids the HOST serves through the service registry (host surfaces). +HOST_SERVICE_IDS = { + "pj.colormap.v1", + "pj.data_processors.v1", + "pj.parser_object_write.v1", + "pj.parser_runtime.v1", + "pj.parser_write.v1", + "pj.playback.v1", + "pj.plot_tabs.v1", + "pj.runtime.v1", + "pj.settings.v1", + "pj.source_object_write.v1", + "pj.source_promotion.v1", + "pj.source_write.v1", + "pj.toolbox_object_read.v1", + "pj.toolbox_runtime.v1", + "pj.toolbox_write.v1", + "pj.viewport.v1", +} +# Versioned ids of PLUGIN-provided extensions (the host queries the plugin). +PLUGIN_EXTENSION_IDS = { + "pj.descriptor_import.v1", + "pj.parser_functional.v1", + "pj.parser_functional.v2", + "pj.parser_route_claims.v1", + "pj.topic_subscription.v1", +} + +VTABLE_TYPEDEF_RE = re.compile(r"typedef\s+struct\s+(\w+)?\s*\{") +FN_PTR_MEMBER_RE = re.compile(r"\(\s*\*\s*(?:const\s+)?(\w+)\s*\)\s*\(") +FN_PTR_TYPEDEF_RE = re.compile(r"typedef\s+[^;{]*?\(\s*\*\s*(\w+)\s*\)\s*\(") +TYPED_MEMBER_RE = re.compile(r"^\s*(?:const\s+)?(\w+)\s+(\w+)\s*$") +SERVICE_ID_RE = re.compile(r'"(pj\.[a-z_0-9.]+\.v\d+)"') +FLAG_RE = re.compile(r"#\s*define\s+(PJ_\w*_FLAG\w*)[\s(]") + +PROTOCOL_HEADER_GLOBS = [ + ("pj_base/include/pj_base", "*.h"), +] +SERVICE_ID_GLOBS = [ + ("pj_base/include/pj_base", "**/*.h*"), + ("pj_plugins/include", "**/*.h*"), + ("pj_plugins/dialog_protocol/include", "**/*.h*"), +] + + +def strip_comments(text: str) -> str: + text = re.sub(r"/\*.*?\*/", "", text, flags=re.S) + return re.sub(r"//[^\n]*", "", text) + + +def _struct_bodies(text: str, path: Path) -> list[tuple[str, str]]: + """Every ``typedef struct [name] { body } name;`` as (name, body).""" + results: list[tuple[str, str]] = [] + for match in VTABLE_TYPEDEF_RE.finditer(text): + start = text.index("{", match.start()) + depth = 0 + for pos in range(start, len(text)): + if text[pos] == "{": + depth += 1 + elif text[pos] == "}": + depth -= 1 + if depth == 0: + break + else: + raise SystemExit(f"unterminated struct in {path}") + body = text[start + 1 : pos] + name = match.group(1) + if name is None: + trailing = re.match(r"\s*(\w+)\s*;", text[pos + 1 :]) + if not trailing: + continue + name = trailing.group(1) + results.append((name, body)) + return results + + +def _top_level_members(body: str) -> list[str]: + """Member declarations split on ';' at parenthesis/brace depth zero.""" + members: list[str] = [] + depth = 0 + current: list[str] = [] + for char in body: + if char in "({[": + depth += 1 + elif char in ")}]": + depth -= 1 + if char == ";" and depth == 0: + members.append("".join(current).strip()) + current = [] + else: + current.append(char) + return [member for member in members if member] + + +def _callable_member_name(declaration: str, fn_ptr_typedefs: set[str]) -> str | None: + fn_ptr = FN_PTR_MEMBER_RE.search(declaration) + if fn_ptr: + return fn_ptr.group(1) + typed = TYPED_MEMBER_RE.match(declaration) + if typed and typed.group(1) in fn_ptr_typedefs: + return typed.group(2) + return None + + +def extract(repo: Path) -> list[str]: + surfaces: set[str] = set() + header_texts: list[tuple[Path, str]] = [] + for rel, glob in PROTOCOL_HEADER_GLOBS: + for header in sorted((repo / rel).glob(glob)): + header_texts.append((header, strip_comments(header.read_text()))) + + fn_ptr_typedefs: set[str] = set() + for _, text in header_texts: + fn_ptr_typedefs.update(FN_PTR_TYPEDEF_RE.findall(text)) + + for path, text in header_texts: + surfaces.update(FLAG_RE.findall(text)) + for struct_name, body in _struct_bodies(text, path): + if not struct_name.endswith("_vtable_t"): + continue + is_host = struct_name in EXTRA_HOST_VTABLES or re.fullmatch( + r"PJ_\w*host_vtable_t", struct_name + ) + if not is_host: + if struct_name not in PLUGIN_SIDE_VTABLES: + raise SystemExit( + f"unclassified vtable {struct_name} in {path}: add it to " + "EXTRA_HOST_VTABLES or PLUGIN_SIDE_VTABLES in extract_host_surfaces.py" + ) + continue + for declaration in _top_level_members(body): + member = _callable_member_name(declaration, fn_ptr_typedefs) + if member is not None: + surfaces.add(f"{struct_name}::{member}") + + for rel, glob in SERVICE_ID_GLOBS: + for header in sorted((repo / rel).glob(glob)): + for service_id in SERVICE_ID_RE.findall(strip_comments(header.read_text())): + if service_id in PLUGIN_EXTENSION_IDS: + continue + if service_id not in HOST_SERVICE_IDS: + raise SystemExit( + f"unclassified versioned id {service_id!r} in {header}: add it to " + "HOST_SERVICE_IDS or PLUGIN_EXTENSION_IDS in extract_host_surfaces.py" + ) + surfaces.add(service_id) + return sorted(surfaces) + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--repo", type=Path, default=Path(__file__).resolve().parents[2]) + parser.add_argument("--write", action="store_true", help="refresh host_surfaces.snapshot") + args = parser.parse_args() + + inventory = extract(args.repo) + snapshot = args.repo / "tools/feature_floors/host_surfaces.snapshot" + if args.write: + snapshot.write_text("\n".join(inventory) + "\n") + print(f"wrote {len(inventory)} surfaces to {snapshot}") + return 0 + sys.stdout.write("\n".join(inventory) + "\n") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/feature_floors/feature_floor_check.py b/tools/feature_floors/feature_floor_check.py new file mode 100644 index 00000000..ca04990d --- /dev/null +++ b/tools/feature_floors/feature_floor_check.py @@ -0,0 +1,562 @@ +#!/usr/bin/env python3 +# Copyright 2026 Davide Faconti +# SPDX-License-Identifier: Apache-2.0 +"""Reusable min_sdk_required floor checking over feature_floors.json. + +This module is the single owner of the table schema and of the rules a plugin +floor claim is validated against. It ships with the SDK package next to +feature_floors.json so any plugin author — in-tree or third-party — can run +the same check the official-plugins CI runs: + + python3 feature_floor_check.py --table feature_floors.json \ + --manifest my_plugin/manifest.json --sources my_plugin + +The contract: ``min_sdk_required`` (the functioning floor) must cover every +matched NON-negotiated surface — a hard surface above the floor is always a +failure. When any matched surface's introducing release exceeds the floor, the +manifest must declare ``suggested_sdk_version`` (the full-feature floor — +exactly max(since) over all matched surfaces) plus ``floor_test`` (one gtest, +existing in the plugin's tests, proving the plugin functions against a +floor-level host); both are forbidden when nothing exceeds the floor. Floors +below the table's ``minimum_supported_floor`` fail; over-declared floors are +the caller's business (this module reports the effective maximum so drivers +can warn). + +Scanning is token matching with identifier boundaries over comment- and +string-literal-stripped sources; the host-surface universe is a few dozen +identifiers, so token precision is sufficient. Known limitation: an identifier +assembled by token-pasting macros (``a ## b``) is invisible — a Clang-based +extractor is the documented upgrade path if a real ambiguity ever appears. +Scope boundary: only RUNTIME host-contract surfaces are visible; compile-time +SDK library APIs never constrain (or justify lowering) a floor. + +Standard library only. No repository-layout assumptions: callers hand in the +table path, source directories, and test directories. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from dataclasses import dataclass +from pathlib import Path +from typing import Iterable, Sequence + +SOURCE_SUFFIXES = { + ".c", + ".cc", + ".cpp", + ".cxx", + ".h", + ".hh", + ".hpp", + ".hxx", + ".inl", + ".ipp", + ".tpp", +} +VERSION_RE = re.compile(r"^\d+\.\d+\.\d+$") +RAW_STRING_START_RE = re.compile(r'(?:u8|u|U|L)?R"([^ ()\\\t\r\n]{0,16})\(') + + +class CheckError(Exception): + """A configuration or input shape error that must fail the check.""" + + +def parse_version(value: object, context: str) -> tuple[int, int, int]: + """Parse a plain X.Y.Z release version (the only form floors may name).""" + if not isinstance(value, str) or not VERSION_RE.fullmatch(value): + raise CheckError(f"{context} must name a stable release as X.Y.Z, got {value!r}") + parts = value.split(".") + return int(parts[0]), int(parts[1]), int(parts[2]) + + +def format_version(version: tuple[int, int, int]) -> str: + return ".".join(str(part) for part in version) + + +@dataclass(frozen=True) +class Surface: + identifier: str + since: tuple[int, int, int] + negotiated: bool + declaration: str + # Every spelling that counts as usage: the C++ wrapper identifier plus, + # for vtable surfaces, the raw ABI member token (host.vtable->member is a + # supported call spelling and must not evade the floor). + tokens: tuple[str, ...] + matcher: re.Pattern[str] + + @property + def since_text(self) -> str: + return format_version(self.since) + + +@dataclass(frozen=True) +class SurfaceTable: + surfaces: dict[str, Surface] + baseline: frozenset[str] + minimum_supported_floor: tuple[int, int, int] + path: Path + + @property + def minimum_text(self) -> str: + return format_version(self.minimum_supported_floor) + + +def _reject_duplicate_json_keys(pairs: list[tuple[str, object]]) -> dict[str, object]: + result: dict[str, object] = {} + for key, value in pairs: + if key in result: + raise CheckError(f"duplicate JSON key {key!r}") + result[key] = value + return result + + +def _token_pattern(token: str) -> str: + pattern = re.escape(token) + if token[0].isalnum() or token[0] == "_": + pattern = r"(? re.Pattern[str]: + if not tokens or not all(tokens): + raise CheckError("surface identifiers must not be empty") + return re.compile("|".join(_token_pattern(token) for token in tokens)) + + +def _surface_tokens(identifier: str, declaration: str) -> tuple[str, ...]: + member = declaration.rsplit("::", 1)[1] if "::" in declaration else "" + if member and member != identifier: + return (identifier, member) + return (identifier,) + + +def load_surface_table(path: Path) -> SurfaceTable: + """Parse the compact version-grouped table (the only schema parser). + + Shape: ``{"schema_version": 1, "minimum_supported_floor": "X.Y.Z", + "since": {"X.Y.Z": {identifier: entry}}, "baseline": [identifier]}`` + where entry is a declaration string (``""`` = the declaration IS the + identifier, as with flag macros) or ``{"declaration": ..., + "negotiated": false}`` for a hard surface. ``negotiated`` defaults true. + """ + try: + raw = path.read_text(encoding="utf-8") + except OSError as exc: + raise CheckError(f"cannot read surface table {path}: {exc}") from exc + try: + document = json.loads(raw, object_pairs_hook=_reject_duplicate_json_keys) + except json.JSONDecodeError as exc: + raise CheckError(f"invalid JSON in surface table {path}: {exc}") from exc + + if not isinstance(document, dict): + raise CheckError(f"surface table {path} must contain a JSON object") + expected_keys = {"schema_version", "minimum_supported_floor", "_comment", "since", "baseline"} + unknown_keys = set(document) - expected_keys + if unknown_keys: + raise CheckError( + f"surface table {path} has unknown top-level keys: {', '.join(sorted(unknown_keys))}" + ) + if document.get("schema_version") != 1: + raise CheckError(f"surface table {path} must use schema_version 1") + + minimum = parse_version( + document.get("minimum_supported_floor"), f"minimum_supported_floor in {path}" + ) + + since_raw = document.get("since") + if not isinstance(since_raw, dict) or not since_raw: + raise CheckError(f"surface table {path} 'since' must be a non-empty object of version groups") + + surfaces: dict[str, Surface] = {} + for since_text, group in since_raw.items(): + since = parse_version(since_text, f"since group in {path}") + if since <= minimum: + raise CheckError( + f"since group {since_text} in {path} is at or below minimum_supported_floor — " + "those surfaces belong in 'baseline'" + ) + if not isinstance(group, dict) or not group: + raise CheckError(f"since group {since_text} in {path} must be a non-empty object") + for identifier, value in group.items(): + if not isinstance(identifier, str) or identifier != identifier.strip() or not identifier: + raise CheckError(f"surface identifier {identifier!r} must be a single trimmed token") + if identifier in surfaces: + raise CheckError( + f"surface {identifier!r} appears in more than one since group in {path}" + ) + negotiated = True + if isinstance(value, str): + declaration = value or identifier + elif isinstance(value, dict): + unknown = set(value) - {"declaration", "negotiated"} + if unknown: + raise CheckError( + f"surface {identifier!r} in {path} has unknown keys: {', '.join(sorted(unknown))}" + ) + declaration_raw = value.get("declaration", "") + if not isinstance(declaration_raw, str): + raise CheckError(f"surface {identifier!r} in {path} needs a string 'declaration'") + declaration = declaration_raw or identifier + negotiated = value.get("negotiated", True) + if not isinstance(negotiated, bool): + raise CheckError(f"surface {identifier!r} in {path} needs a boolean 'negotiated'") + else: + raise CheckError( + f"surface {identifier!r} in {path} must map to a declaration string or an object" + ) + tokens = _surface_tokens(identifier, declaration) + surfaces[identifier] = Surface( + identifier, since, negotiated, declaration, tokens, surface_matcher(tokens) + ) + + # Baseline surfaces predate (or equal) the minimum supported floor, so they + # can never raise a floor at or above it — listed so verdicts can name them + # and so an exception naming one is "known" (and rejected as unnecessary + # via the stale-claim rule when nothing matches). + baseline_raw = document.get("baseline", []) + if not isinstance(baseline_raw, list): + raise CheckError(f"surface table {path} baseline must be an array of identifiers") + for identifier in baseline_raw: + if not isinstance(identifier, str) or identifier != identifier.strip() or not identifier: + raise CheckError(f"baseline identifier {identifier!r} must be a single trimmed token") + if identifier in surfaces: + raise CheckError( + f"surface {identifier!r} in {path} is listed twice (surfaces/baseline overlap " + "or duplicate baseline entry)" + ) + surfaces[identifier] = Surface( + identifier, minimum, True, identifier, (identifier,), surface_matcher((identifier,)) + ) + return SurfaceTable(surfaces, frozenset(baseline_raw), minimum, path) + + +# --- C/C++ source scanning -------------------------------------------------- + + +def _blank_except_newlines(text: str) -> str: + return "".join("\n" if char == "\n" else " " for char in text) + + +def strip_cpp_comments(text: str) -> str: + """Blank C/C++ comments AND string/char-literal contents, preserving offsets. + + Literal contents must not count as identifier usage (a help string naming a + surface is not a call), so they are blanked exactly like comments. + """ + output: list[str] = [] + index = 0 + quote: str | None = None + escaped = False + while index < len(text): + if quote is not None: + char = text[index] + if escaped: + escaped = False + output.append(" " if char != "\n" else "\n") + elif char == "\\": + escaped = True + output.append(" ") + elif char == quote: + quote = None + output.append(char) + else: + output.append(" " if char != "\n" else "\n") + index += 1 + continue + + raw_match = RAW_STRING_START_RE.match(text, index) + if raw_match: + closing = ")" + raw_match.group(1) + '"' + end = text.find(closing, raw_match.end()) + end = len(text) if end < 0 else end + len(closing) + output.append(_blank_except_newlines(text[index:end])) + index = end + continue + if text.startswith("//", index): + end = text.find("\n", index) + end = len(text) if end < 0 else end + output.append(_blank_except_newlines(text[index:end])) + index = end + continue + if text.startswith("/*", index): + end = text.find("*/", index + 2) + end = len(text) if end < 0 else end + 2 + output.append(_blank_except_newlines(text[index:end])) + index = end + continue + char = text[index] + if char in {'"', "'"}: + quote = char + output.append(char) + index += 1 + return "".join(output) + + +def _splice_cpp_lines(text: str) -> tuple[str, list[int]]: + """Apply phase-2 backslash-newline removal, retaining an offset mapping.""" + output: list[str] = [] + original_offsets: list[int] = [] + index = 0 + while index < len(text): + if text.startswith("\\\r\n", index): + index += 3 + continue + if text.startswith("\\\n", index): + index += 2 + continue + output.append(text[index]) + original_offsets.append(index) + index += 1 + return "".join(output), original_offsets + + +def _original_line(text: str, original_offsets: Sequence[int], spliced_offset: int) -> int: + original_offset = ( + original_offsets[spliced_offset] if spliced_offset < len(original_offsets) else len(text) + ) + return text.count("\n", 0, original_offset) + 1 + + +def scan_source_file(path: Path, table: SurfaceTable) -> list[tuple[str, int]]: + """All (surface identifier, 1-based line) matches in one source file.""" + try: + raw_source = path.read_text(encoding="utf-8") + except (OSError, UnicodeDecodeError) as exc: + raise CheckError(f"cannot read C/C++ source {path}: {exc}") from exc + spliced_source, original_offsets = _splice_cpp_lines(raw_source) + searchable_source = strip_cpp_comments(spliced_source) + matches: list[tuple[str, int]] = [] + for surface in table.surfaces.values(): + # Fast native substring scan first: absence of every matchable token + # proves the boundary-aware regex cannot match (measured ~55x). + if not any(token in searchable_source for token in surface.tokens): + continue + for match in surface.matcher.finditer(searchable_source): + matches.append( + (surface.identifier, _original_line(raw_source, original_offsets, match.start())) + ) + return matches + + +def iter_source_files(directory: Path, excluded_dirs: frozenset[str] = frozenset()) -> Iterable[Path]: + for path in sorted(directory.rglob("*")): + if not path.is_file() or path.suffix.lower() not in SOURCE_SUFFIXES: + continue + relative_parts = path.relative_to(directory).parts[:-1] + if any(part in excluded_dirs or part.startswith("build-") for part in relative_parts): + continue + yield path + + +# --- Exception + floor validation ------------------------------------------- + + +def validate_manifest_shape(manifest: dict) -> list[str]: + """Table-free shape rules for the floor-contract manifest fields. + + ``suggested_sdk_version`` is the FULL-FEATURE floor — the SDK release at + which every capability the plugin uses is available — while + ``min_sdk_required`` stays the functioning floor. ``floor_test`` names the + ONE gtest proving the plugin functions against a floor-level host + (degradations active). The two travel together: declaring a degraded range + without its proof, or a proof without the range, is malformed. Whether the + values are CORRECT against real surface usage is the table-aware check. + """ + errors: list[str] = [] + if manifest.get("sdk_floor_exceptions") is not None: + errors.append( + "sdk_floor_exceptions is no longer supported — declare " + "suggested_sdk_version + floor_test instead" + ) + suggested = manifest.get("suggested_sdk_version") + floor_test = manifest.get("floor_test") + if suggested is not None: + if not isinstance(suggested, str) or not VERSION_RE.fullmatch(suggested): + errors.append( + f"suggested_sdk_version must name a stable release as X.Y.Z, got {suggested!r}" + ) + else: + minimum = manifest.get("min_sdk_required") + if isinstance(minimum, str) and VERSION_RE.fullmatch(minimum): + if tuple(map(int, suggested.split("."))) < tuple(map(int, minimum.split("."))): + errors.append( + f"suggested_sdk_version {suggested} is below min_sdk_required {minimum}" + ) + if floor_test is None: + errors.append( + "floor_test is required alongside suggested_sdk_version: name the gtest " + "proving the plugin functions against a floor-level host" + ) + if floor_test is not None: + if not isinstance(floor_test, str) or "." not in floor_test: + errors.append("floor_test must name a gtest as 'Suite.Name'") + if suggested is None: + errors.append("floor_test is only meaningful alongside suggested_sdk_version — remove it") + return errors + + +def test_exists(test_dirs: Iterable[Path], test_name: str) -> bool: + """True when `Suite.Name` is a real gtest TEST/TEST_F/TEST_P declaration. + + Sources are comment-stripped first: a commented-out test must not satisfy + the floor_test obligation. + """ + if "." not in test_name: + return False + suite, _, name = test_name.partition(".") + pattern = re.compile( + r"TEST(?:_F|_P)?\s*\(\s*" + re.escape(suite) + r"\s*,\s*" + re.escape(name) + r"\s*[,)]" + ) + for directory in test_dirs: + if not directory.is_dir(): + continue + for path in iter_source_files(directory): + try: + text = path.read_text(encoding="utf-8") + except (OSError, UnicodeDecodeError): + continue + spliced, _ = _splice_cpp_lines(text) + if pattern.search(strip_cpp_comments(spliced)): + return True + return False + + +def floor_problems( + manifest: dict, + declared: tuple[int, int, int], + table: SurfaceTable, + matched: dict[str, str], + test_dirs: Iterable[Path], +) -> list[str]: + """The whole floor contract, table-aware. + + 1. ``min_sdk_required`` >= since of every matched NON-negotiated surface — + a hard surface above the floor is always a failure, nothing waives it. + 2. ``suggested_sdk_version`` is REQUIRED exactly when a matched surface's + since exceeds the floor, and must equal max(since) over ALL matched + surfaces; it is forbidden otherwise (it would repeat the floor). + 3. ``floor_test`` (required with suggested) must name a gtest that exists + in the plugin's tests — the proof the degraded configuration works. + """ + problems = validate_manifest_shape(manifest) + if problems: + return problems + if declared < table.minimum_supported_floor: + problems.append( + f"min_sdk_required {format_version(declared)} is below the supported floor " + f"{table.minimum_text} (older history is not classified)" + ) + + above_floor = sorted( + (table.surfaces[identifier] for identifier in matched if table.surfaces[identifier].since > declared), + key=lambda surface: (surface.since, surface.identifier), + ) + for surface in above_floor: + if not surface.negotiated: + problems.append( + f"{surface.identifier} (since {surface.since_text}) exceeds floor " + f"{format_version(declared)} at {matched[surface.identifier]} and is not " + "runtime-negotiated — raise min_sdk_required" + ) + + suggested = manifest.get("suggested_sdk_version") + if above_floor: + expected = format_version( + max(table.surfaces[identifier].since for identifier in matched) + ) + if suggested is None: + newest = above_floor[-1] + problems.append( + f"suggested_sdk_version is missing: {newest.identifier} " + f"(since {newest.since_text}, at {matched[newest.identifier]}) exceeds floor " + f"{format_version(declared)} — declare suggested_sdk_version {expected} " + "(+ floor_test) or raise min_sdk_required" + ) + elif suggested != expected: + problems.append( + f"suggested_sdk_version is {suggested} but the full-feature floor over the " + f"matched surfaces is {expected} — declare exactly that" + ) + else: + floor_test = manifest["floor_test"] + if not test_exists(test_dirs, floor_test): + problems.append( + f"floor_test {floor_test!r} not found in the plugin's tests" + ) + elif suggested is not None: + problems.append( + "suggested_sdk_version is only meaningful when a used surface exceeds " + "min_sdk_required (it would just repeat the floor) — remove it" + ) + return problems + + +def effective_maximum(table: SurfaceTable, matched: dict[str, str]) -> tuple[int, int, int] | None: + """The highest introducing release among matched surfaces (for over-declared warnings).""" + return max((table.surfaces[identifier].since for identifier in matched), default=None) + + +# --- Standalone CLI for out-of-tree plugin authors --------------------------- + + +def main(argv: Sequence[str] | None = None) -> int: + parser = argparse.ArgumentParser( + description="Check one plugin's min_sdk_required against the host surfaces its sources use." + ) + parser.add_argument("--table", type=Path, required=True, help="Path to feature_floors.json") + parser.add_argument("--manifest", type=Path, required=True, help="Path to the plugin's manifest.json") + parser.add_argument( + "--sources", + type=Path, + action="append", + required=True, + help="Source directory to scan (repeatable)", + ) + parser.add_argument( + "--tests", + type=Path, + action="append", + help="Test directory for fallback-test lookup (default: /tests and /test)", + ) + args = parser.parse_args(argv) + + try: + table = load_surface_table(args.table) + manifest = json.loads(args.manifest.read_text(encoding="utf-8")) + if not isinstance(manifest, dict): + raise CheckError(f"{args.manifest} must contain a JSON object") + declared = parse_version(manifest.get("min_sdk_required"), "min_sdk_required") + + matched: dict[str, str] = {} + for directory in args.sources: + for path in iter_source_files(directory, frozenset({"build", "test", "tests"})): + for identifier, line in scan_source_file(path, table): + matched.setdefault(identifier, f"{path}:{line}") + + test_dirs = args.tests or [args.manifest.parent / "tests", args.manifest.parent / "test"] + problems = floor_problems(manifest, declared, table, matched, test_dirs) + except CheckError as exc: + print(f"ERROR: {exc}", file=sys.stderr) + return 1 + + if problems: + print(f"FAIL (floor {format_version(declared)}):", file=sys.stderr) + for problem in problems: + print(f" - {problem}", file=sys.stderr) + return 1 + summary = ", ".join(sorted(matched)) if matched else "no host surfaces matched" + suggested = manifest.get("suggested_sdk_version") + degraded_note = f", full features at {suggested}" if suggested else "" + print(f"PASS (floor {format_version(declared)}{degraded_note}): {summary}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/feature_floors/host_surfaces.snapshot b/tools/feature_floors/host_surfaces.snapshot new file mode 100644 index 00000000..8ed8d06d --- /dev/null +++ b/tools/feature_floors/host_surfaces.snapshot @@ -0,0 +1,113 @@ +PJ_DATA_PROCESSOR_FLAG_EPHEMERAL +PJ_DESCRIPTOR_IMPORT_START_FLAGS_V1_MASK +PJ_DESCRIPTOR_IMPORT_START_FLAG_NONE +PJ_INGEST_COMPLETION_FLAGS_V1_MASK +PJ_INGEST_COMPLETION_FLAG_ATTESTS_EMPTY_TOPICS +PJ_INGEST_COMPLETION_FLAG_NONE +PJ_MODULE_PARSE_INPUT_FLAG_HAS_TIMESTAMP +PJ_PARSER_ROUTE_FLAG_OBJECT_V1 +PJ_PARSER_ROUTE_FLAG_SCALAR_V1 +PJ_colormap_registry_vtable_t::register_map +PJ_colormap_registry_vtable_t::unregister_map +PJ_data_processors_host_vtable_t::create_data_processor +PJ_data_processors_host_vtable_t::data_processor_config +PJ_data_processors_host_vtable_t::list_data_processor_ids +PJ_data_processors_host_vtable_t::remove_data_processor +PJ_data_processors_host_vtable_t::validate_data_processor_script +PJ_data_source_runtime_host_vtable_t::attach_source_record +PJ_data_source_runtime_host_vtable_t::complete_ingest +PJ_data_source_runtime_host_vtable_t::ensure_parser_binding +PJ_data_source_runtime_host_vtable_t::is_stop_requested +PJ_data_source_runtime_host_vtable_t::list_available_encodings +PJ_data_source_runtime_host_vtable_t::notify_available_topics +PJ_data_source_runtime_host_vtable_t::notify_state +PJ_data_source_runtime_host_vtable_t::progress_finish +PJ_data_source_runtime_host_vtable_t::progress_start +PJ_data_source_runtime_host_vtable_t::progress_update +PJ_data_source_runtime_host_vtable_t::push_message +PJ_data_source_runtime_host_vtable_t::report_message +PJ_data_source_runtime_host_vtable_t::request_stop +PJ_data_source_runtime_host_vtable_t::set_dataset_metadata +PJ_data_source_runtime_host_vtable_t::show_message_box +PJ_object_read_host_vtable_t::entry_count +PJ_object_read_host_vtable_t::get_bytes +PJ_object_read_host_vtable_t::list_topics +PJ_object_read_host_vtable_t::lookup_topic +PJ_object_read_host_vtable_t::lookup_topic_on_dataset +PJ_object_read_host_vtable_t::read_latest_at +PJ_object_read_host_vtable_t::release_bytes +PJ_object_read_host_vtable_t::time_range +PJ_object_read_host_vtable_t::topic_metadata +PJ_object_write_host_vtable_t::push_lazy +PJ_object_write_host_vtable_t::push_owned +PJ_object_write_host_vtable_t::register_topic +PJ_object_write_host_vtable_t::set_retention_budget +PJ_parser_object_write_host_vtable_t::push_lazy +PJ_parser_object_write_host_vtable_t::push_owned +PJ_parser_runtime_host_vtable_t::report_diagnostic +PJ_parser_write_host_vtable_t::append_arrow_stream +PJ_parser_write_host_vtable_t::append_bound_record +PJ_parser_write_host_vtable_t::append_record +PJ_parser_write_host_vtable_t::ensure_field +PJ_playback_host_vtable_t::get_state +PJ_playback_host_vtable_t::pause +PJ_playback_host_vtable_t::play +PJ_playback_host_vtable_t::seek +PJ_playback_host_vtable_t::set_playback_rate +PJ_playback_host_vtable_t::to_display_time +PJ_playback_host_vtable_t::to_display_time_for_source +PJ_plot_tab_host_vtable_t::add_curve +PJ_plot_tab_host_vtable_t::clear_tab +PJ_plot_tab_host_vtable_t::close_tab +PJ_plot_tab_host_vtable_t::create_tab +PJ_plot_tab_host_vtable_t::list_tab_ids +PJ_plot_tab_host_vtable_t::remove_curve +PJ_plot_tab_host_vtable_t::tab_config +PJ_service_registry_vtable_t::get_service +PJ_settings_store_vtable_t::contains +PJ_settings_store_vtable_t::get_string +PJ_settings_store_vtable_t::get_string_list +PJ_settings_store_vtable_t::remove_key +PJ_settings_store_vtable_t::set_string +PJ_settings_store_vtable_t::set_string_list +PJ_source_promotion_host_vtable_t::promote_to_file_source +PJ_source_write_host_vtable_t::append_arrow_stream +PJ_source_write_host_vtable_t::append_bound_record +PJ_source_write_host_vtable_t::append_record +PJ_source_write_host_vtable_t::ensure_field +PJ_source_write_host_vtable_t::ensure_topic +PJ_toolbox_host_vtable_t::acquire_catalog_snapshot +PJ_toolbox_host_vtable_t::append_arrow_stream +PJ_toolbox_host_vtable_t::append_bound_record +PJ_toolbox_host_vtable_t::append_record +PJ_toolbox_host_vtable_t::create_data_source +PJ_toolbox_host_vtable_t::ensure_field +PJ_toolbox_host_vtable_t::ensure_topic +PJ_toolbox_host_vtable_t::push_owned_object +PJ_toolbox_host_vtable_t::read_series_arrow +PJ_toolbox_host_vtable_t::register_object_topic +PJ_toolbox_host_vtable_t::register_object_topic_on_dataset +PJ_toolbox_host_vtable_t::set_object_topic_retention +PJ_toolbox_runtime_host_vtable_t::create_parser_ingest +PJ_toolbox_runtime_host_vtable_t::discard_parser_ingest +PJ_toolbox_runtime_host_vtable_t::notify_data_changed +PJ_toolbox_runtime_host_vtable_t::release_parser_ingest +PJ_toolbox_runtime_host_vtable_t::report_message +PJ_viewport_host_vtable_t::zoom_reset +PJ_viewport_host_vtable_t::zoom_to_time_range +pj.colormap.v1 +pj.data_processors.v1 +pj.parser_object_write.v1 +pj.parser_runtime.v1 +pj.parser_write.v1 +pj.playback.v1 +pj.plot_tabs.v1 +pj.runtime.v1 +pj.settings.v1 +pj.source_object_write.v1 +pj.source_promotion.v1 +pj.source_write.v1 +pj.toolbox_object_read.v1 +pj.toolbox_runtime.v1 +pj.toolbox_write.v1 +pj.viewport.v1