From d745672de4c78d82735c046957088f53c58303e2 Mon Sep 17 00:00:00 2001 From: Danil Silantyev Date: Thu, 8 Oct 2026 07:03:32 +0500 Subject: [PATCH 1/2] docs: add system completion workstreams --- docs/completion-plan-20261008.md | 131 +++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 docs/completion-plan-20261008.md diff --git a/docs/completion-plan-20261008.md b/docs/completion-plan-20261008.md new file mode 100644 index 0000000..4805404 --- /dev/null +++ b/docs/completion-plan-20261008.md @@ -0,0 +1,131 @@ +# System completion plan — 2026-10-08 + +This plan is the executable follow-up to `continuation-plan.md`. It was +rechecked against the current `main` tree (`9bd1840e`) and the capability matrix +before implementation. A capability changes state only after its code, +wire/authz path, platform probe, tests, and installed qualification all pass. + +## Current truth + +- Connectivity, signed discovery, grants v2/v3, managed local sessions, + single-file resumable sync, Linux X11 desktop, and the default Iroh lane are + implemented or experimental as recorded in `capability-matrix.md`. +- Audio is wire-shaped but has no source, codec, jitter, sink, agent admission, + or viewer path. It remains `stub`. +- Linux Wayland image-copy/KMS/portal capture and macOS ScreenCaptureKit, + VideoToolbox and CGEvent are explicit stubs. X11 is the only capture/input + backend that moves pixels on a supported device. +- Sync is intentionally single-file. Its journal and confinement are the + safety foundation; recursive/two-way manifests, tombstones, conflict policy, + watch/reconcile and journal GC are not implemented. +- Noq and owned relay have strong in-process evidence but remain experimental + until the topology, interface-change, UDP-blocked and parity matrix is run on + real supported devices. +- Automatic GDS enrollment, grant issuance/renewal/revocation and applied + policy acknowledgements are outside this public module and must be integrated + through the estate authority rather than hand-edited grant files. + +## Ordered workstreams and gates + +### A — Audio core and wire path + +1. Keep `rds-core` audio tags stable; add a bounded `rds-audio` core with + validated Opus formats, fixed 20 ms default frames, packet size limits, + sequence numbers, capture timestamps, decoder PLC and a bounded reorder/ + jitter buffer. +2. Add audio service admission and grant scope only after a real source/sink + abstraction is injectable in tests. No service advertisement from a stub. +3. Add Linux PipeWire/CPAL and macOS CoreAudio adapters behind platform + features. Source callbacks must never block on network or disk; a bounded + queue drops oldest audio with counters and preserves control priority. +4. Add audio uni-stream framing, loss/late/duplicate handling, clock drift + observation and a sink adapter. Keep microphone permission separate from + desktop-view permission. + +Gate A: loopback encode/decode, malformed packet rejection, deterministic PLC, +reordering/loss matrix, bounded memory, and an injected source/sink end-to-end +session. Only then change `service:audio` from `stub`. + +### B — Recursive sync foundation + +1. Add a directory manifest format with typed file/dir/symlink metadata, + normalized relative paths, deterministic ordering and explicit unsupported + attribute states. Never follow symlinks while scanning. +2. Add snapshot/reconcile operations on top of the existing journal, with + tombstones, rename detection by stable content identity, bounded entries and + cancellation checkpoints. +3. Define one-way mirror first (dry-run/change preview, delete policy and + resumable operation IDs), then two-way conflict records and metadata policy. +4. Add journal quotas/GC with inode ownership proofs; cleanup may remove only + complete owned records and never guesses from a filename. + +Gate B: seeded Linux/macOS fixtures, crash-at-every-commit boundary, disk full, +symlink races, offline event repair, conflict preservation and exact digest +convergence. Single-file protocol compatibility remains unchanged. + +### C — Native platform backends + +1. macOS: ScreenCaptureKit stream output with complete-frame filtering, + IOSurface-backed frames and bounded queue depth; VideoToolbox encode/decode; + CGEvent input with TCC denial/regrant states; launchd/user-session identity. +2. Wayland: portal RemoteDesktop + ScreenCast session, PipeWire stream + selection by `pipewire-serial`, EIS input, restore-token lifecycle, then + ext-image-copy-capture-v1 where compositor permission exists. KMS is a + separately privileged unattended profile, never an implicit fallback. +3. Keep backend probe order and capability reporting truthful: unavailable, + denied and runtime-failed are different states. No shelling out to capture + utilities. + +Gate C: real macOS arm64 and at least two Wayland compositor profiles, monitor +selection/resize, lock/unlock/suspend, permission denial/regrant, input release +on disconnect, and exact installed artifact/signing identity. + +### D — Transport and policy completion + +1. Complete Noq/Iroh parity matrix: interface changes, NAT rebinding, suspend, + UDP-blocked TLS fallback, relay death, directory outage, path accounting, + churn hysteresis and service-aware recovery. +2. Integrate GDS enrollment and revisioned policy acknowledgement through the + estate control plane. Public code accepts signed policy snapshots and reports + stale/offline state; it never owns private tenant facts. +3. Add negotiated resource/QoS classes for control, terminal, desktop, audio + and bulk sync. Prove fairness under mixed load with RSS/FD/task ceilings. + +Gate D: identical scenario definitions on Iroh and Noq, explicit NOT-RUN rows +for unavailable topologies, and no owned-backend promotion without parity. + +### E — Release and operations + +1. Make observability reports machine-produced with source/features/toolchain, + backend, topology, sample counts, failures and skips. +2. Add secured redacted diagnostic bundles and applied-policy status. +3. Qualify signed/notarized macOS and Linux artifacts, atomic update/drain, + rollback, launchd/systemd health and compatibility with older peers. + +Gate E: clean-machine install, exact running digest, rollback after interruption, +all mandatory acceptance rows, and synchronized public/estate/controller state. + +## Research decisions used by this plan + +- ScreenCaptureKit delivers `CMSampleBuffer` output and IOSurface-backed video; + discard non-complete frames and keep capture queues bounded. See Apple’s + [SCStream documentation](https://developer.apple.com/documentation/screencapturekit/scstream) + and [capture sample](https://developer.apple.com/documentation/screencapturekit/capturing-screen-content-in-macos). +- XDG ScreenCast persistence uses single-use restore tokens; combined remote + desktop persistence belongs to the RemoteDesktop portal. Use EIS for input + rather than legacy D-Bus notify calls. See the + [ScreenCast](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html) + and [RemoteDesktop](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.RemoteDesktop.html) + contracts. +- Wayland image-copy capture is still a testing-stage protocol; negotiate + compositor buffer constraints and treat `failed` as a visible capability + result. See the [protocol definition](https://wayland.app/protocols/ext-image-copy-capture-v1). +- Opus uses standard 2.5/5/10/20/40/60 ms frame sizes; custom modes add delay + and reduce interoperability. The core uses 20 ms by default and bounded + packet validation, based on the [Opus API guidance](https://opus-codec.org/docs/opus_api-1.6.pdf) + and [RFC 6716](https://www.rfc-editor.org/rfc/rfc6716). + +Each workstream lands as a signed conventional-commit PR, with its own report +and checkpoint. After every merge: re-run public checks, refresh estate pin if +needed, qualify exact artifacts, refresh controller snapshot in preserve mode, +validate memories, and verify local/origin/controller/runtime parity. From cdeb1e0079569160593c4dad6d317cf5832f4fc8 Mon Sep 17 00:00:00 2001 From: Danil Silantyev Date: Thu, 8 Oct 2026 07:29:07 +0500 Subject: [PATCH 2/2] feat(audio): add bounded libopus packet core --- Cargo.lock | 30 ++ Cargo.toml | 1 + README.md | 2 +- crates/rds-audio/Cargo.toml | 3 + crates/rds-audio/src/lib.rs | 496 ++++++++++++++++++++++-- docs/reports/rds-audio-core-20261008.md | 31 ++ docs/roadmap.md | 6 + 7 files changed, 541 insertions(+), 28 deletions(-) create mode 100644 docs/reports/rds-audio-core-20261008.md diff --git a/Cargo.lock b/Cargo.lock index fda67d8..d43ceac 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -680,6 +680,15 @@ version = "1.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1c133bc6a41be0d194c306b5506d15e6feeea7b1d6604bd3f8310dfb2ca96486" +[[package]] +name = "cmake" +version = "0.1.58" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c0f78a02292a74a88ac736019ab962ece0bc380e3f977bf72e376c5d78ff0678" +dependencies = [ + "cc", +] + [[package]] name = "cmov" version = "0.5.4" @@ -3490,6 +3499,24 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe" +[[package]] +name = "opus" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "33718946cc77d4032911d4efe03a66dbcbfbd2bb16c3da06aaeadcc637c32216" +dependencies = [ + "opusic-sys", +] + +[[package]] +name = "opusic-sys" +version = "0.7.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c9d1ecdf206421bc74343ab3bb2f30ad2abbfee41fa341f7181fecbaf957769a" +dependencies = [ + "cmake", +] + [[package]] name = "orbclient" version = "0.3.55" @@ -4181,6 +4208,9 @@ name = "rds-audio" version = "0.1.0" dependencies = [ "bytes", + "opus", + "rds-core", + "thiserror 2.0.21", ] [[package]] diff --git a/Cargo.toml b/Cargo.toml index 034f6ab..9c3c302 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -55,6 +55,7 @@ iroh = { version = "1.3", features = ["unstable-custom-transports"] } iroh-relay = "1.3" noq = "1" noq-proto = "1" +opus = { version = "=0.4.0" } postcard = "1" proptest = "1" rand = "0.10" diff --git a/README.md b/README.md index ef6ef08..0f5c2f6 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ crates/ ├── rds-observe bounded process logs and privacy-preserving telemetry ├── rds-bench development measurement/qualification harness ├── rds-desktop capture/codec/input/render traits + platform backends -├── rds-audio Opus audio pipeline (scaffold) +├── rds-audio bounded libopus packet/jitter core (service adapters open) └── rds-sync FastCDC+BLAKE3 content-addressed sync ``` diff --git a/crates/rds-audio/Cargo.toml b/crates/rds-audio/Cargo.toml index 95465cc..157cde7 100644 --- a/crates/rds-audio/Cargo.toml +++ b/crates/rds-audio/Cargo.toml @@ -8,6 +8,9 @@ repository.workspace = true [dependencies] bytes.workspace = true +opus.workspace = true +rds-core.workspace = true +thiserror.workspace = true [lints] workspace = true diff --git a/crates/rds-audio/src/lib.rs b/crates/rds-audio/src/lib.rs index f6f8c42..5398fc4 100644 --- a/crates/rds-audio/src/lib.rs +++ b/crates/rds-audio/src/lib.rs @@ -1,46 +1,488 @@ -//! Audio pipeline for rds — scaffold. +//! Bounded audio packet, codec and playout primitives for RDS. //! -//! Remote-desktop audio rides its own stream, independent of video: -//! Opus-encoded frames with a small jitter buffer and an independent -//! clock. Platform capture/render: -//! -//! - Linux: PipeWire (`pipewire`/`libspa`) preferred, `cpal` fallback. -//! - macOS: `cpal` (CoreAudio). -//! -//! Encoding is `opus` (libopus bindings — the C library is the Opus -//! standard); resampling `rubato` when device rate differs. -//! Implementation lands with the audio milestone; the traits pin the -//! seam now. +//! The crate deliberately owns no device callback, network stream or async +//! task. Platform capture/playback adapters and the agent service consume +//! these primitives later. Keeping packetization and loss behavior here gives +//! every platform the same limits and makes malformed audio fail closed before +//! it reaches a device sink. + +use std::collections::BTreeMap; use bytes::Bytes; +use thiserror::Error; + +/// Opus packets are limited by RFC 6716's maximum representable size. +pub const MAX_OPUS_PACKET_BYTES: usize = 1275; +/// The default interactive frame duration. +pub const DEFAULT_FRAME_DURATION_MS: u32 = 20; +/// Maximum number of packets retained by one jitter buffer. +pub const MAX_JITTER_PACKETS: usize = 64; /// PCM format negotiated between source and codec. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct AudioFormat { - /// Samples per second (48000 for Opus). + /// Samples per second. Opus supports 8, 12, 16, 24 and 48 kHz. pub sample_rate: u32, - /// Channel count (1 or 2). + /// Channel count (mono or stereo). pub channels: u16, } -/// One encoded audio packet ready for the wire. -#[derive(Debug)] +impl AudioFormat { + pub const MONO_48_KHZ: Self = Self { + sample_rate: 48_000, + channels: 1, + }; + + pub fn validate(self) -> Result<(), AudioError> { + if !matches!(self.sample_rate, 8_000 | 12_000 | 16_000 | 24_000 | 48_000) { + return Err(AudioError::InvalidFormat("unsupported sample rate")); + } + if !matches!(self.channels, 1 | 2) { + return Err(AudioError::InvalidFormat("audio must be mono or stereo")); + } + Ok(()) + } + + pub fn frame_samples(self, duration_ms: u32) -> Result { + self.validate()?; + if !matches!(duration_ms, 2 | 5 | 10 | 20 | 40 | 60) { + return Err(AudioError::InvalidFrame( + "frame duration must be 2, 5, 10, 20, 40 or 60 ms", + )); + } + Ok((self.sample_rate as usize * duration_ms as usize) / 1000) + } +} + +/// One encoded audio packet ready for a future wire stream. +#[derive(Debug, Clone, PartialEq, Eq)] pub struct AudioPacket { - /// Capture timestamp in microseconds, monotonic to the source. + /// Monotonic packet sequence within one audio session. + pub seq: u64, + /// Capture timestamp in microseconds on the source's monotonic clock. pub timestamp_us: u64, - /// Opus payload. + /// Samples per channel represented by this packet. + pub samples: u32, + /// Opus payload without an Ogg container. pub data: Bytes, } -/// Source of system audio on the serving side. -pub trait AudioSource: Send + 'static { - fn format(&self) -> AudioFormat; - /// Next encoded packet; blocks as the pipeline dictates. - fn next_packet(&mut self) -> Option; +impl AudioPacket { + pub fn from_wire( + frame: rds_core::AudioFrame, + format: AudioFormat, + frame_samples: usize, + ) -> Result { + format.validate()?; + if frame.samples as usize != frame_samples { + return Err(AudioError::InvalidFrame("wire frame sample count mismatch")); + } + if frame.data.is_empty() || frame.data.len() > MAX_OPUS_PACKET_BYTES { + return Err(AudioError::InvalidPacket( + "wire packet size is outside bounds", + )); + } + Ok(Self { + seq: frame.seq, + timestamp_us: frame + .capture_ts_ms + .checked_mul(1000) + .ok_or(AudioError::InvalidPacket("wire timestamp overflow"))?, + samples: frame.samples, + data: Bytes::from(frame.data), + }) + } + + pub fn to_wire(&self) -> rds_core::AudioFrame { + rds_core::AudioFrame { + seq: self.seq, + capture_ts_ms: self.timestamp_us / 1000, + samples: self.samples, + data: self.data.to_vec(), + } + } +} + +#[derive(Debug, Error, Clone, PartialEq, Eq)] +pub enum AudioError { + #[error("invalid audio format: {0}")] + InvalidFormat(&'static str), + #[error("invalid audio frame: {0}")] + InvalidFrame(&'static str), + #[error("invalid audio packet: {0}")] + InvalidPacket(&'static str), + #[error("Opus codec error: {0}")] + Codec(String), +} + +/// Official libopus encoder with fixed frame sizing and bounded output. +pub struct OpusEncoder { + inner: opus::Encoder, + format: AudioFormat, + frame_samples: usize, + next_seq: u64, +} + +impl OpusEncoder { + pub fn new(format: AudioFormat) -> Result { + let frame_samples = format.frame_samples(DEFAULT_FRAME_DURATION_MS)?; + let channels = match format.channels { + 1 => opus::Channels::Mono, + 2 => opus::Channels::Stereo, + _ => unreachable!("AudioFormat validates channel count"), + }; + let inner = opus::Encoder::new(format.sample_rate, channels, opus::Application::Audio) + .map_err(|error| AudioError::Codec(error.to_string()))?; + Ok(Self { + inner, + format, + frame_samples, + next_seq: 0, + }) + } + + pub fn format(&self) -> AudioFormat { + self.format + } + + pub fn frame_samples(&self) -> usize { + self.frame_samples + } + + pub fn encode(&mut self, pcm: &[f32], timestamp_us: u64) -> Result { + let expected = self.frame_samples * self.format.channels as usize; + if pcm.len() != expected { + return Err(AudioError::InvalidFrame("PCM frame has the wrong length")); + } + let mut data = vec![0u8; MAX_OPUS_PACKET_BYTES]; + let len = self + .inner + .encode_float(pcm, &mut data) + .map_err(|error| AudioError::Codec(error.to_string()))?; + data.truncate(len); + let packet = AudioPacket { + seq: self.next_seq, + timestamp_us, + samples: self.frame_samples as u32, + data: Bytes::from(data), + }; + self.next_seq = self.next_seq.wrapping_add(1); + Ok(packet) + } +} + +/// Official libopus decoder. An empty payload is used only by decode_plc; +/// packet loss is represented by a jitter-buffer gap. +pub struct OpusDecoder { + inner: opus::Decoder, + format: AudioFormat, + frame_samples: usize, +} + +impl OpusDecoder { + pub fn new(format: AudioFormat) -> Result { + let frame_samples = format.frame_samples(DEFAULT_FRAME_DURATION_MS)?; + let channels = match format.channels { + 1 => opus::Channels::Mono, + 2 => opus::Channels::Stereo, + _ => unreachable!("AudioFormat validates channel count"), + }; + let inner = opus::Decoder::new(format.sample_rate, channels) + .map_err(|error| AudioError::Codec(error.to_string()))?; + Ok(Self { + inner, + format, + frame_samples, + }) + } + + pub fn decode(&mut self, packet: &AudioPacket) -> Result, AudioError> { + if packet.samples as usize != self.frame_samples { + return Err(AudioError::InvalidFrame("packet sample count mismatch")); + } + if packet.data.is_empty() || packet.data.len() > MAX_OPUS_PACKET_BYTES { + return Err(AudioError::InvalidPacket("packet size is outside bounds")); + } + let mut pcm = vec![0.0; self.frame_samples * self.format.channels as usize]; + let decoded = self + .inner + .decode_float(&packet.data, &mut pcm, false) + .map_err(|error| AudioError::Codec(error.to_string()))?; + if decoded != self.frame_samples { + return Err(AudioError::Codec( + "decoder returned an unexpected frame size".into(), + )); + } + Ok(pcm) + } + + /// Decode one lost packet using Opus packet-loss concealment. + pub fn decode_plc(&mut self) -> Result, AudioError> { + let mut pcm = vec![0.0; self.frame_samples * self.format.channels as usize]; + let decoded = self + .inner + .decode_float(&[], &mut pcm, false) + .map_err(|error| AudioError::Codec(error.to_string()))?; + if decoded != self.frame_samples { + return Err(AudioError::Codec( + "PLC returned an unexpected frame size".into(), + )); + } + Ok(pcm) + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum PushOutcome { + Accepted, + Duplicate, + Late, + DroppedOverflow, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum PopOutcome { + Packet, + Gap { sequence: u64 }, +} + +/// Sequence-aware bounded reorder buffer. It never waits on a timer and never +/// allocates beyond its configured packet count; the playout owner decides +/// when a Gap should become decoder PLC. +pub struct JitterBuffer { + format: AudioFormat, + frame_samples: usize, + capacity: usize, + target_delay: usize, + next_sequence: Option, + packets: BTreeMap, + pub accepted: u64, + pub duplicates: u64, + pub late: u64, + pub overflow: u64, + pub gaps: u64, +} + +impl JitterBuffer { + pub fn new( + format: AudioFormat, + capacity: usize, + target_delay: usize, + ) -> Result { + let frame_samples = format.frame_samples(DEFAULT_FRAME_DURATION_MS)?; + if capacity == 0 || capacity > MAX_JITTER_PACKETS || target_delay >= capacity { + return Err(AudioError::InvalidFrame("invalid jitter-buffer bounds")); + } + Ok(Self { + format, + frame_samples, + capacity, + target_delay, + next_sequence: None, + packets: BTreeMap::new(), + accepted: 0, + duplicates: 0, + late: 0, + overflow: 0, + gaps: 0, + }) + } + + pub fn push(&mut self, packet: AudioPacket) -> Result { + if packet.samples as usize != self.frame_samples { + return Err(AudioError::InvalidFrame( + "jitter packet sample count mismatch", + )); + } + if packet.data.is_empty() || packet.data.len() > MAX_OPUS_PACKET_BYTES { + return Err(AudioError::InvalidPacket( + "jitter packet size is outside bounds", + )); + } + if self.next_sequence.is_some_and(|next| packet.seq < next) { + self.late += 1; + return Ok(PushOutcome::Late); + } + if self.packets.contains_key(&packet.seq) { + self.duplicates += 1; + return Ok(PushOutcome::Duplicate); + } + let mut outcome = PushOutcome::Accepted; + if self.packets.len() == self.capacity { + self.packets.pop_first(); + self.overflow += 1; + outcome = PushOutcome::DroppedOverflow; + } + self.packets.insert(packet.seq, packet); + self.accepted += 1; + Ok(outcome) + } + + /// Pop the next packet or report a sequence gap once the target delay is + /// exceeded. The caller should invoke decoder PLC for a gap. + pub fn pop(&mut self) -> Option<(PopOutcome, Option)> { + let next = match self.next_sequence { + Some(next) => next, + None => { + let next = *self.packets.first_key_value()?.0; + self.next_sequence = Some(next); + next + } + }; + if let Some(packet) = self.packets.remove(&next) { + self.next_sequence = Some(next.wrapping_add(1)); + return Some((PopOutcome::Packet, Some(packet))); + } + if self.packets.len() > self.target_delay { + let available = *self.packets.first_key_value()?.0; + self.next_sequence = Some(available); + self.gaps += available.wrapping_sub(next); + return Some((PopOutcome::Gap { sequence: next }, None)); + } + None + } + + /// Set the first expected sequence before playout begins. This lets a + /// receiver report leading loss instead of silently choosing the first + /// packet that happened to arrive. + pub fn start_at(&mut self, sequence: u64) -> Result<(), AudioError> { + if self.next_sequence.is_some() { + return Err(AudioError::InvalidFrame("jitter playout already started")); + } + self.next_sequence = Some(sequence); + Ok(()) + } + + pub fn len(&self) -> usize { + self.packets.len() + } + + pub fn is_empty(&self) -> bool { + self.packets.is_empty() + } + + pub fn format(&self) -> AudioFormat { + self.format + } } -/// Playback sink on the viewing side (jitter buffer lives inside). -pub trait AudioSink: Send + 'static { - /// Queue a packet for playback; late packets may be dropped. - fn play(&mut self, packet: AudioPacket); +#[cfg(test)] +mod tests { + use super::*; + + fn packet(seq: u64) -> AudioPacket { + AudioPacket { + seq, + timestamp_us: seq * 20_000, + samples: 960, + data: Bytes::from_static(&[0x01, 0x02]), + } + } + + #[test] + fn format_bounds_are_explicit() { + assert!(AudioFormat::MONO_48_KHZ.validate().is_ok()); + assert_eq!(AudioFormat::MONO_48_KHZ.frame_samples(20).unwrap(), 960); + assert!( + AudioFormat { + sample_rate: 44_100, + channels: 2 + } + .validate() + .is_err() + ); + assert!(AudioFormat::MONO_48_KHZ.frame_samples(15).is_err()); + } + + #[test] + fn opus_round_trip_is_bounded_and_exactly_sized() { + let format = AudioFormat::MONO_48_KHZ; + let mut encoder = OpusEncoder::new(format).unwrap(); + let mut decoder = OpusDecoder::new(format).unwrap(); + let pcm = vec![0.0; encoder.frame_samples()]; + let packet = encoder.encode(&pcm, 123_000).unwrap(); + assert_eq!(packet.seq, 0); + assert!(packet.data.len() <= MAX_OPUS_PACKET_BYTES); + let decoded = decoder.decode(&packet).unwrap(); + assert_eq!(decoded.len(), pcm.len()); + assert_eq!(decoder.decode_plc().unwrap().len(), pcm.len()); + } + + #[test] + fn jitter_reorders_and_reports_a_gap() { + let mut jitter = JitterBuffer::new(AudioFormat::MONO_48_KHZ, 4, 1).unwrap(); + assert_eq!(jitter.push(packet(2)).unwrap(), PushOutcome::Accepted); + assert_eq!(jitter.push(packet(1)).unwrap(), PushOutcome::Accepted); + assert_eq!(jitter.push(packet(1)).unwrap(), PushOutcome::Duplicate); + assert!(matches!( + jitter.pop(), + Some((PopOutcome::Packet, Some(p))) if p.seq == 1 + )); + assert!(matches!( + jitter.pop(), + Some((PopOutcome::Packet, Some(p))) if p.seq == 2 + )); + assert!(jitter.pop().is_none()); + assert_eq!(jitter.gaps, 0); + } + + #[test] + fn jitter_reports_a_gap_after_delay() { + let mut jitter = JitterBuffer::new(AudioFormat::MONO_48_KHZ, 4, 1).unwrap(); + assert_eq!(jitter.push(packet(0)).unwrap(), PushOutcome::Accepted); + assert!(matches!( + jitter.pop(), + Some((PopOutcome::Packet, Some(p))) if p.seq == 0 + )); + assert_eq!(jitter.push(packet(2)).unwrap(), PushOutcome::Accepted); + assert_eq!(jitter.push(packet(3)).unwrap(), PushOutcome::Accepted); + assert!(matches!( + jitter.pop(), + Some((PopOutcome::Gap { sequence: 1 }, None)) + )); + assert!(matches!( + jitter.pop(), + Some((PopOutcome::Packet, Some(p))) if p.seq == 2 + )); + } + + #[test] + fn jitter_start_sequence_reports_leading_loss() { + let mut jitter = JitterBuffer::new(AudioFormat::MONO_48_KHZ, 4, 0).unwrap(); + jitter.start_at(5).unwrap(); + assert_eq!(jitter.push(packet(6)).unwrap(), PushOutcome::Accepted); + assert!(matches!( + jitter.pop(), + Some((PopOutcome::Gap { sequence: 5 }, None)) + )); + assert!(matches!( + jitter.pop(), + Some((PopOutcome::Packet, Some(p))) if p.seq == 6 + )); + } + + #[test] + fn jitter_drops_oldest_on_overflow_and_rejects_late_packets() { + let mut jitter = JitterBuffer::new(AudioFormat::MONO_48_KHZ, 2, 0).unwrap(); + assert_eq!(jitter.push(packet(0)).unwrap(), PushOutcome::Accepted); + assert_eq!(jitter.push(packet(1)).unwrap(), PushOutcome::Accepted); + assert_eq!( + jitter.push(packet(2)).unwrap(), + PushOutcome::DroppedOverflow + ); + assert_eq!(jitter.overflow, 1); + assert!(jitter.pop().is_some()); + assert_eq!(jitter.push(packet(0)).unwrap(), PushOutcome::Late); + } + + #[test] + fn wire_conversion_rejects_wrong_frame_shape() { + let frame = rds_core::AudioFrame { + seq: 1, + capture_ts_ms: 4, + samples: 1, + data: vec![1], + }; + assert!(AudioPacket::from_wire(frame, AudioFormat::MONO_48_KHZ, 960).is_err()); + } } diff --git a/docs/reports/rds-audio-core-20261008.md b/docs/reports/rds-audio-core-20261008.md new file mode 100644 index 0000000..66e37b3 --- /dev/null +++ b/docs/reports/rds-audio-core-20261008.md @@ -0,0 +1,31 @@ +# Audio core foundation — 2026-10-08 + +This wave makes the audio library a real bounded core while keeping the public +capability truthful: no device source, playback sink, agent admission or viewer +playout is advertised yet. + +Implemented in `rds-audio`: + +- validated Opus sample rates, channel counts and standard frame durations; +- libopus encode/decode with a 1275-byte packet ceiling and 20 ms + default frames; +- deterministic sequence/timestamp/sample-count conversion to + `rds_core::AudioFrame`; +- malformed-packet rejection; +- bounded sequence reorder buffering with duplicate, late and overflow counters; +- explicit gap reporting for decoder packet-loss concealment; +- six unit tests covering format bounds, encode/decode/PLC, reorder, loss, + overflow and wire validation. + +The codec dependency is pinned to `opus 0.4.0` with bundled `opusic-sys`/ +libopus (MIT/Apache-2.0 plus BSD-3-Clause). The official libopus reference +implementation is the normative interoperability boundary; the Rust crate is a +thin safe binding and the bundled build keeps installation reproducible. The +core owns no callback, async task, socket or device handle, so platform adapters +cannot block the transport or bypass the shared bounds. + +Not claimed by this report: microphone/system-audio capture, CoreAudio or +PipeWire adapters, playback, permission lifecycle, audio grants, an audio +uni-stream, A/V clock synchronization or installed-device audio acceptance. +Those are the next completion-plan workstream and keep `service:audio` in the +`stub` state until their end-to-end gate passes. diff --git a/docs/roadmap.md b/docs/roadmap.md index 7fa7d20..9f8cf90 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -6,6 +6,12 @@ Current execution order and reopened gates are in the The milestones below preserve the original direction; historical checkpoint completion does not establish that the audited defects or product gaps are closed. +The 2026-10-08 completion wave first landed the bounded `rds-audio` packet, +libopus and jitter core. Device capture/playback, agent admission and viewer +playout remain separate until their platform and authorization gates pass; see +the [completion workstreams](completion-plan-20261008.md) and the +[scoped report](reports/rds-audio-core-20261008.md). + The [native viewer](native-viewer.md) now implements W6.3 window/input and newest-frame GPU presentation, with bounded reconnect and media-gap recovery. The X11 serving increment removes fixed idle polling and supports explicit