Skip to content

Add HIF/Python interoperability and reproducible ecosystem benchmarks - #8

Merged
bregydoc merged 9 commits into
mainfrom
codex/hif-python-interoperability
Oct 4, 2026
Merged

bregydoc merged 9 commits into
mainfrom
codex/hif-python-interoperability

Conversation

@bregydoc

@bregydoc bregydoc commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Hyper can exchange scientific datasets through HIF without silently losing metadata or unsupported incidence semantics. A Rust-backed typed Python package supports HIF import/export and launches the native viewer. This PR also adds ecosystem research and reproducible benchmark evidence to guide positioning and roadmap decisions.

  • Validate against a bundled, pinned HIF schema offline; preserve complete documents, typed IDs, weights and metadata. Viewer conversion rejects unsupported semantics with field locations.
  • Detect HIF through the shared loader and support explicit --format for initial loads and watched reloads, preserving native/legacy behavior.
  • Add PyO3/maturin bindings, generated stubs, an asynchronous viewer handle with snapshot cleanup, XGI/Julia examples and platform wheel CI.
  • Ship hypergraph-viz (import hyper_viz) core wheels and a standalone source archive that excludes Bevy. Add hypergraph-viz-viewer native wheels through the exact-version [viewer] extra; discover installed executables through RECORD metadata without requiring environment activation. Explicit overrides take priority, then the companion, then PATH.
  • Exclude unused Bevy audio/gamepad backends and audit Linux binaries without external library grafting, avoiding a pinned maturin binary-repair relocation defect. Link the Windows C runtime statically and inspect PE imports to avoid a separate Visual C++ runtime installer.
  • Publish the companion before the core using configured Trusted Publishers, only after all core/source/desktop integration checks pass. Ordinary PR and manual runs never publish.
  • Add an opt-in Bevy benchmark feature, core CPU example, canonical dataset generation and adapters for scientific figures, browser rendering and desktop workflows.
  • Record 445 measured attempts across 89 configurations on one M5 Pro Mac, comparing Hyper with HyperNetX, XGI, Hypergraphx, 3d-force-graph, Cytoscape.js, Sigma, HyperGodot and Graphia. Include raw evidence, dataset/source checksums, failure counts, comparison tables and CSV, plus archived report regeneration.
  • Document ecosystem findings in docs/ecosystem-comparison.md and the benchmark archive. Add docs/NOTEBOOK_ROADMAP.md: a future anywidget Canvas2D transport/selection prototype using a 7-node, 4-edge fixture, followed by a separate Bevy/WASM feasibility probe. No notebook API ships here.

Validation

  • 71 core Rust tests and 52 Bevy library tests pass, along with all-target Clippy, formatting, generated-stub drift checks and strict Python typing.
  • All 17 tests pass against locally installed 0.1.2 core and desktop wheels; executable discovery works with empty PATH. Strict PyPI metadata and wheel-content checks pass.
  • Local macOS arm64 graphics capture, Python launch/close and temporary snapshot cleanup pass. The first desktop matrix passed all four native binary builds and installed-extra checks, including Linux software Vulkan/Xvfb graphics capture. Windows and macOS Intel checks cover CLI startup and discovery, without claiming GUI coverage.
  • Linux wheel ELF inspection found only standard system runtime dependencies, with no ALSA/udev or grafted libraries. The previous Windows artifact exposed a Visual C++ runtime dependency; the new PE audit rejects it and gates the static-runtime rebuild.
  • Core CI verifies clean CPython 3.10/3.14 installs and builds a wheel from the standalone source archive. The corrected 0.1.2 tag workflow passed every build and integration gate, then both Trusted Publishing uploads. A fresh public PyPI install verified typed IDs, metadata, load/save, exceptions, typing, empty-PATH discovery, and real macOS render/launch/close/cleanup. The final Windows PE imports only system DLLs, with no external Visual C++ runtime.
  • Benchmark work includes 8 harness tests, 24 independent scientific conversion checks, archived checksums and byte-for-byte report regeneration. The Julia example was not runtime-tested.

Measurement limits

The tracks perform different work: scientific rasterization, browser callback cadence, native application intervals and Graphia import/save are separate metrics. Display cadence was not locked; these results do not establish a universal speed ranking. Native runs retain 16 timeouts across 120 attempts, including one on a small fixture; their cause remains unresolved. A source audit identified a quadratic hub-status lookup for follow-up profiling. Incorrect early Sigma refresh and clamped-window HyperGodot measurements are excluded diagnostics. Human task accuracy, input-to-visible latency, GPU duration and cross-platform benchmark rendering remain unmeasured.

Release status

hypergraph-viz 0.1.0 is published on PyPI. The unpublished python-v0.1.1 release was canceled after inspecting its Windows runtime dependency; that tag remains unchanged. Both hypergraph-viz 0.1.2 and hypergraph-viz-viewer 0.1.2 are published. The release workflow passed all gates and uploads; PyPI lists four wheels for each package and a core source archive. Its desktop installation command is python -m pip install --upgrade "hypergraph-viz[viewer]"; there are no downloads during show().

Related: ATO-2428. See docs/HIF.md, docs/PYTHON_RELEASE.md and scripts/benchmarks/README.md for integration, releases and reruns.

@bregydoc bregydoc changed the title Add HIF interoperability and typed Python integration Add HIF/Python interoperability and reproducible ecosystem benchmarks Oct 4, 2026
@bregydoc
bregydoc merged commit 43d6034 into main Oct 4, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant