Skip to content

Own live provider and socket coordination in the compiled module - #33

Closed
senamakel wants to merge 63 commits into
shrink-core-live-voice-7414from
shrink-core-live-runtime-7414
Closed

senamakel wants to merge 63 commits into
shrink-core-live-voice-7414from
shrink-core-live-runtime-7414

Conversation

@senamakel

Copy link
Copy Markdown
Member

Move portable live control parsing, provider event projection and the client/provider coordinator into the compiled TinyVoice module. A host reserves a bounded lease, supplies prepared connection material and explicit tool declarations, and binds mechanical socket/persistence callbacks. The module executes the existing TinyAgentsLive provider/tool loop, preserves transcript-frame-before-persistence order, and owns polling, replay, delivery retries and cleanup.

Tool callbacks retain exact native call IDs and separately generated cancellation IDs. Current host policy remains authoritative; an unknown or revoked tool never receives a synthesized executor. Input, output, queues, leases and callback deadlines are bounded. Client EOF, callback errors, cancellation, explicit release and module drop stop owned tasks; unknown host cleanup contexts cannot fabricate success. Stored callback access uses TinyBus weak connections to avoid a served-object ownership cycle.

This PR is deliberately draft until TinyBus #50 lands and TinyVoice #32 lands. It targets #32's published branch to keep the runtime diff separate. The recorded TinyBus source provides the proposed non-owning capability; this is not a released host dependency. Preserve-merge the landed prerequisites and retarget main before marking ready.

Validation:

  • Actual native bus regressions RED→GREEN for missing live operations, retained module bus name, precise dropped-callback cancellation, oversized provider startup errors, multi-waiter cancellation and aborted cleanup.
  • Full native module suite: 163 passed.
  • Real dynamic artifact verifies the exact 42-method manifest, provider/socket duplex loop, current-policy denial, native tool provenance, unknown-tool failure, ordered persistence, unacknowledged replay, lost-delivery reply deduplication and shutdown.
  • Instrumented native fixtures plus the loaded artifact: each runtime source file exceeds 90% line coverage (actor 90.23%, callback 97.83%, prepare 94.25%, projection 93.10%, table 91.41%, transport 94.19%). A new CI gate executes this same proof.
  • Owner workspace per-file coverage gate passes at 90%, including contract redaction/wire fixtures; strict workspace/module Clippy, formatting, docs, release-version script, shell syntax and workflow YAML checks pass.
  • Providers are local scripted fixtures; no test contacts an external provider.

Part of tinyhumansai/openhuman#7414. OpenHuman credential/config policy, native local-brain execution, progress eligibility, correlation admission, transcript storage and product delivery stay with the host. Realtime acknowledgment/deferred-turn coordination is a separate remaining owner slice; host adoption waits for an actual released, admitted artifact. This PR does not close that issue or update an OpenHuman release pin.

senamakel and others added 30 commits October 11, 2026 19:58
Add the tinyagents repository as a git submodule under vendor so it can be
consumed alongside the existing vendored dependencies.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Document the intended design and rollout plan for the live module
runtime so the implementation work has a shared reference.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a test asserting that parsing a live start frame keeps the default
input sample rate and carries the thread and client identifiers through
the reply.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a PrepareLiveInput bus method and bump the contract version to 1.7 so
hosts can parse portable live control frames without applying host policy.
A new LiveProtocol error reports frames that violate the control protocol.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…ice-module/Cargo.toml,crates/ti

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The live websocket control types now live under a live submodule instead of a
single file, and the module service advertises the PrepareLiveInput method so
hosts can reach it. The verify_module example exercises a start frame to confirm
the default sixteen kHz input rate is preserved.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…lper

The verify_module example now calls a dedicated verify_live_input function instead of inlining the live start framing check, and the service module registers the new live submodule. This keeps the example's main flow readable and gives the live projection code a place to live.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add the wire types the host uses to drive a live session: provider
connection material, provider configuration, the start request, session
commands and the handle request. Credentials and configuration are kept
out of Debug output so secrets do not leak into logs.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Live tool declarations now include a prefer_markdown flag so the host's
compact result rendering preference survives the trip to the provider, and
the live service gains a callback module to hold that plumbing.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The ready projection test now compares the whole mapped frame against an expected value instead of destructuring a single field, so any change to the other fields of the ready frame is caught as well.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Live sessions now carry a host-selected tool-call cap and string lease
identities, and a new FinishLiveContext method lets the host idempotently
revoke a known context and cancel its work. Tool callback requests and
replies are also rejected when their serialized form exceeds one mebibyte,
so oversized payloads fail dispatch instead of being relayed.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a test asserting that a live reservation can be created before the
provider starts and then released, verifying the reserved and closed
states along with the handle id length.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Wire up the new actor, prepare, and table modules in the live service and
re-export the session table for use by the runtime.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Session teardown now waits for the actor to acknowledge that host context
cleanup finished, and reports an error if the worker exits unexpectedly or
the acknowledgement never arrives. Startup validation also rejects connect
and tool timeouts above their upper bounds so oversized values cannot stall
a session indefinitely.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add LiveReserve, LiveStart, LiveSend, LiveRead, LiveRelease and LiveShutdown
methods to the Voice interface, backed by a shared live session table wired to
the bus connection at setup. This lets clients reserve, drive and tear down
bounded live sessions through the module rather than only preparing input
frames.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The start path now checks the session's closed flag and the lease's
cancelled flag before admitting a reservation, so a session that was
shut down or cancelled can no longer be started. Visibility of the
session methods was also widened to the service module to match their
actual callers.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Adds a test module for the live session table and enables tokio's
test-util feature in dev-dependencies so tests can control time.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The two shutdown tests now hold the session lease in a local binding
before passing it to the blocked task, so the lease outlives the call
instead of being dropped as a temporary.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a test asserting that aborting an in-flight release does not detach the
owned provider worker, so the worker is still dropped when the lease goes away.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Shutdown now stops all admitted sessions in parallel via a JoinSet so the
deadline no longer scales with session count, and cleanup futures are aborted
if the caller cancels. Provider worker handles are wrapped so dropping a
cleanup future aborts the task instead of detaching it.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Wire the callback tool's test file into the module tree so its tests are
compiled and run.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Adds a test module for the live actor and switches the callback test
fixture to the connection's serve_at helper, which is the current API.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Live event projection now rejects oversized session ids, provider errors and
remote close reasons before they reach the host, and context cleanup takes the
tool port directly so repeated or late finish calls only revoke a known host
context once. Tests cover the unresponsive-host timeout and idempotent
revocation paths, and the module docs now list the 41 methods under contract
1.7.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The verify_module example now runs a live runtime check against the bus,
using tokio-tungstenite and futures-util as dev-dependencies. The bus
constants are imported from tinyvoice_bus instead of tinyvoice_module so
the example exercises the same names the module serves.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
…k_tests.rs

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a CancelLiveTool callback and a fresh invocation id on ExecuteLiveTool so
the module can tell the host to revoke exactly one known invocation when its
native dispatch future is dropped, rather than leaving host work detached.
Cancellation is bounded and best effort, so hosts must still cancel their own
invocation leases on endpoint or context teardown.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The live verification example now uses tokio's async mutex for its call
and finish counters instead of std's blocking mutex, and the output
checker became async to match. This keeps the fixture from blocking the
runtime while it holds the lock.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
senamakel and others added 20 commits October 11, 2026 21:30
The oversized-event check and the projection call are now combined in a
single bounded_projection helper, so the startup error path and the
streaming path share the same bounding logic instead of duplicating it.
This also ensures provider errors on the startup path are bounded before
projection.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Extend the live verification example to bind a client transport and exercise
input reads, effect delivery with a lost reply, and terminal close output.
Add a regression test asserting the callback cancellation guard can be
dropped outside its creating runtime context.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The invocation guard now stores the current tokio runtime handle and uses it to spawn the cancellation call on drop, instead of relying on the ambient runtime. This ensures the cancellation request is still dispatched when the guard is dropped outside of a runtime context.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
CI and release workflows now fetch submodules recursively so nested
submodules are present during builds. Sessions::with_connection takes the
connection by reference instead of cloning it, and the callback test
fixture records the current runtime handle.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The spec now describes binding a LiveTransport endpoint to a reserved
handle, the acknowledgment-based read protocol, ordered effect delivery
with stable IDs, and the shutdown and weak-connection rules. The verify
example and transport tests gained counters to assert that a bound
socket is polled and closed exactly once.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The transport test fixture now guards its committed and effects state with
tokio's async mutex, so the delivery path no longer blocks a runtime thread
while holding a lock. Assertions and the capabilities accessor were simplified
to match.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the vendored tinybus submodule to a newer revision.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the vendored tinybus submodule to a newer revision.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add tests for the live session table covering native input handling and
output batching. The input test verifies that malformed base64, odd-length
audio, and oversized audio or text payloads are rejected while valid audio,
interrupt, and stop commands are forwarded to the native session. The output
test verifies that batches split between atomic groups and that a single
group exceeding the batch limit is rejected.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add actor tests that verify oversized native audio is split into bounded
chunks before the session closes, and that cancelling before connect still
finishes the known host context. Also annotate the base64 helper closure in
the table tests so it accepts borrowed slices.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add tests for the live transport covering control and audio input
ordering, rejection of oversized text and audio, bounded retries when
delivery is rejected, and expiry of forgotten bound reservations. The
socket fixture gains a reject_delivery flag so the retry path can be
exercised.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add tests asserting that oversized provider errors are detected while
timeouts are not, and that a malformed finish endpoint cannot acknowledge
host cleanup. Also cover live admission rejecting changed, expired, and
oversized inputs before any connection is opened.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a test asserting that cancelling an active idle session terminates the
native provider driver and runs host cleanup exactly once. The transport
fixture's two independent delivery flags are also folded into a single
DeliveryMode enum so the mutually exclusive cases are expressed directly.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a test asserting that when the output consumer disappears while a
provider is active, native audio stops and the session ends in the closed
state without leaving provider work behind. The table test module is made
visible to the live actor tests so they can reuse a helper that closes the
output reader.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Register the new session_tests and transport_tests files as cfg(test) modules so the live session and transport suites are compiled and run with the crate.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a README section and module-level docs describing how the compiled
module owns the live provider loop through TinyAgentsLive and the bounded
client/provider coordinator, while hosts supply connection material, tool
authority, socket callbacks and persistence. Also fix rustdoc links for the
Gemini and ElevenLabs connection variants.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
The CI workflow now builds the live module and runs its boundary
verification example, then enforces the 90% per-file coverage threshold
against the live module coverage report and uploads that report alongside
the existing one. A doc comment in the bus session was also reworded to
satisfy the lint on the TinyTools name.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Wrap the TinyAgentsLive reference in backticks in the module docs and
mark the portable relay coordinator step as a separate owner follow-up
so the plan reflects that it is not part of the current extraction.

Auto-committed-on: dragonfly
Co-authored-by: Medulla <medulla@tinyhumans.ai>
@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

This review includes 44 billable files and costs up to $11.00.

Or wait 26 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 87fb5935-f405-448b-bcaa-b184c5750ef6

📥 Commits

Reviewing files that changed from the base of the PR and between 103f9d6 and bc57421.


⛔ Files ignored due to path filters (1)
  • crates/tinyvoice-module/Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (44)
  • .github/scripts/check-live-module-coverage.sh
  • .github/workflows/ci.yml
  • .github/workflows/release.yml
  • .gitmodules
  • MODULE.md
  • README.md
  • crates/tinyvoice-bus/src/lib.rs
  • crates/tinyvoice-bus/src/live/mod.rs
  • crates/tinyvoice-bus/src/live/session.rs
  • crates/tinyvoice-bus/src/live/session_tests.rs
  • crates/tinyvoice-bus/src/live/transport.rs
  • crates/tinyvoice-bus/src/live/transport_tests.rs
  • crates/tinyvoice-bus/src/names/mod.rs
  • crates/tinyvoice-bus/src/version/mod.rs
  • crates/tinyvoice-module/Cargo.toml
  • crates/tinyvoice-module/examples/verify_module.rs
  • crates/tinyvoice-module/examples/verify_module/live.rs
  • crates/tinyvoice-module/src/lib.rs
  • crates/tinyvoice-module/src/service/live/actor.rs
  • crates/tinyvoice-module/src/service/live/actor_tests.rs
  • crates/tinyvoice-module/src/service/live/callback.rs
  • crates/tinyvoice-module/src/service/live/callback_tests.rs
  • crates/tinyvoice-module/src/service/live/fixtures/start.json
  • crates/tinyvoice-module/src/service/live/mod.rs
  • crates/tinyvoice-module/src/service/live/prepare.rs
  • crates/tinyvoice-module/src/service/live/prepare_tests.rs
  • crates/tinyvoice-module/src/service/live/projection.rs
  • crates/tinyvoice-module/src/service/live/projection_tests.rs
  • crates/tinyvoice-module/src/service/live/table.rs
  • crates/tinyvoice-module/src/service/live/table_tests.rs
  • crates/tinyvoice-module/src/service/live/transport.rs
  • crates/tinyvoice-module/src/service/live/transport_tests.rs
  • crates/tinyvoice-module/src/service/mod.rs
  • crates/tinyvoice-module/src/service/mod_tests.rs
  • crates/tinyvoice-module/src/service/relay.rs
  • crates/tinyvoice/src/error/mod.rs
  • crates/tinyvoice/src/lib.rs
  • crates/tinyvoice/src/live/mod.rs
  • crates/tinyvoice/src/live/mod_tests.rs
  • docs/plans/live-module-runtime.md
  • docs/specs/live-module-runtime.md
  • docs/specs/module-capture.md
  • vendor/tinyagents
  • vendor/tinybus

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Co-authored-by: Medulla <medulla@tinyhumans.ai>
@senamakel

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@senamakel
senamakel deleted the branch shrink-core-live-voice-7414 October 11, 2026 19:23
@senamakel senamakel closed this Oct 11, 2026
@senamakel

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-11T19:31:12.783611Z bc57421 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Provided git ref bc5742142925ea3802d8ce3b8cfe9850990af929 does not exist
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant