Skip to content

Add fake demo runtime vertical slice - #433

Merged
jarcherNV merged 51 commits into
dev/jarcher/inference-runtime-apifrom
dev/jarcher/unified-demo-runtime
Aug 10, 2026
Merged

Add fake demo runtime vertical slice#433
jarcherNV merged 51 commits into
dev/jarcher/inference-runtime-apifrom
dev/jarcher/unified-demo-runtime

Conversation

@jarcherNV

Copy link
Copy Markdown
Collaborator

No description provided.

@greptile-apps

greptile-apps Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds a shared fake demo-runtime vertical slice and adopts it across replay and WebRTC paths, including model-worker lifecycle hardening and OmniDreams integration.

  • Introduces batch and realtime session drivers, run modes, validation, timing, output sinks, metrics, and warmup primitives.
  • Expands shared WebRTC encoding, media, service, and lifecycle infrastructure.
  • Adds OmniDreams demo adapters, replay/WebRTC entry points, artifact validation workflow, and broad runtime test coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
flashdreams/flashdreams/runtime/demo/drivers.py Adds batch and realtime session orchestration with cancellation-resistant, worker-affine cleanup and host quarantine on cleanup failure.
flashdreams/flashdreams/runtime/demo/run_modes.py Defines session edges, admission, error policies, metrics, results, and warmup behavior used by the new drivers.
flashdreams/flashdreams/runtime/demo/host.py Adds runtime-host lifecycle, model-worker dispatch, health tracking, and warmup coordination.
flashdreams/flashdreams/serving/webrtc/manager.py Integrates the shared demo runtime into WebRTC session management and transport lifecycle.
integrations/omnidreams/omnidreams/demo/runtime.py Adapts OmniDreams model execution to the shared demo runtime contracts.
.github/workflows/omnidreams-demo-runtime.yml Adds GPU CI coverage for null, precomputed MP4, and Ludus replay modes with output validation.

Sequence Diagram

sequenceDiagram
  participant Client
  participant Mode as Demo Run Mode
  participant Driver as Session Driver
  participant Host as Runtime Host
  participant Model as Model Session
  participant Output as Output Sink
  Mode->>Driver: run session
  Driver->>Host: create provider and session
  Host->>Model: execute model-affine work
  loop Each step
    Driver->>Host: execute pipeline step
    Host->>Model: generate output
    Model-->>Output: frames or artifacts
  end
  Driver->>Host: close session and provider
  Driver->>Output: finalize result
  alt Cleanup fails or times out
    Driver->>Host: mark unhealthy
    Host-->>Mode: reject subsequent admission
  end
Loading

Reviews (51): Last reviewed commit: "docs: record validated OmniDreams demo c..." | Re-trigger Greptile

Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Close session edges and record a failed run result when DriverInvariantError
escapes after edge creation, and apply the ruff formatting fixes required by
CPU CI.
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Add StepRequirements for model-authored per-step requirements and adapt
legacy StepRequest values at the shared demo driver boundary. Move demo
input, pipeline, and realtime timing contracts to driver-owned input
windows, with CPU coverage for validation, deterministic slicing,
realtime slicing, and legacy compatibility.
Add Mp4OutputSink and build_output_sink for the shared demo runtime while
preserving the legacy OutputTarget builder for replay. Make null sink step
recording lightweight, keep sink close idempotent, and cover MP4 artifacts,
sink construction, payload ownership, setup-open failures, and cleanup-close
failures with CPU tests.
Add provider and run-mode capability contracts, resolved run validation, and
first-class InferenceConfig.seed for deterministic demo runs. Wire validation
into shared demo session helpers and cover raw input schemas, mapping-backed
providers, MP4/WebRTC compatibility checks, realtime/batch source validation,
determinism resolution, and reset coordination with CPU fakes.
Replace the demo timing module's concrete KeyboardResampler import with a
structural resampler protocol, keeping runtime/demo independent of new serving
imports. Also include Ruff formatting fixes needed by the PR lint check.
Add an optional adapter-owned model warmup hook, worker-dispatched warmup plan
construction, and a shared run-context warmup helper that separates model
runtime warmup from run-mode output/transport warmup. Cover temporary providers,
worker-thread input construction, runtime/session warmup calls, transport-only
warmup, and metrics exclusion with fake CPU tests.
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Make run-level cleanup error recording best-effort so provider cleanup failures
cannot replace the original session assembly outcome. Cover sync and async
no-edge cleanup paths where provider close and cleanup telemetry both fail.
Refactor run_inference_session into a compatibility wrapper over the shared
BatchSessionDriver and StepPipeline while preserving legacy mapping, output
target, timing metrics, and cleanup behavior.

Add an explicit legacy StepRequest adaptation opt-in and focused CPU coverage
for wrapper delegation and driver-owned user-window handling.
Move the default replay demo path onto a private replay RunMode backed by
RunContext, SessionEdges, BatchSessionDriver, StepPipeline, and the shared
MP4/null OutputSink implementations.

Preserve old runner/output-target injection as a compatibility path, return
RunResult from replay, and make DemoApplication map failed replay results to a
non-zero exit. Add CPU coverage for sink adoption, legacy payload parity, runner
compatibility, and replay failure exits.
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py
Comment thread flashdreams/flashdreams/runtime/demo/drivers.py Outdated
Add a standalone GPU workflow for the migrated OmniDreams demo runtime paths.
The workflow exercises null output, precomputed HDMap MP4, and Ludus recorded-trace
MP4 runs, validates logs and MP4 metadata, and uploads generated artifacts for
inspection.
Reduce the OmniDreams demo runtime GPU workflow MP4 runs from one minute to
roughly 20 seconds. Keep the same null, precomputed HDMap, and Ludus validation
coverage while reducing GPU CI runtime and artifact size.
Move the canonical OmniDreams runtime/session into demo/runtime.py while
preserving replay compatibility aliases. Split the default shared WebRTC
entrypoint from the legacy compatibility facade, share WebRTC config through
demo/webrtc_config.py, and cover shared-vs-legacy routing plus lazy legacy
imports in CPU tests.
Update the OmniDreams demo README with the remote GPU setup and validated
null, precomputed MP4, Ludus MP4, and WebRTC commands used during migration
testing. Remove the untested explicit benchmark-asset example from the main
command list.
@jarcherNV
jarcherNV merged commit 6239fdc into dev/jarcher/inference-runtime-api Aug 10, 2026
8 checks passed
@jarcherNV
jarcherNV deleted the dev/jarcher/unified-demo-runtime branch August 10, 2026 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant