Data streams v2 by 1egoman · Pull Request #1192 · livekit/rust-sdks

1egoman · 2026-06-26T20:26:48Z

Overview

This change introduces data streams v2, a set of major updates to data streams to add three things:

Single packet data streams for short (< 15kb), finite length payloads
DEFLATE compression, when both the sender and all receivers support compression/decompression
A maximum 15kb size for data stream attributes to close this as a DOS vector

For information about important decisions made in data streams v2, see the spec. In particular, parts 1-5 provide a good high level overview of data streams v1 + v2, what these protocol improvements are aiming to acheive, and why certain choices were made when designing the protocol.

Also maybe of interest is the corresponding web sdk pull request.

These updates are fully backwards compatible - the sdk will fall back to data streams v1 whenever an old participant is in the room.

There is also a significant amount of new test coverage to data streams (both v1 and v2). There were some small bugs I found along the way which I fixed as well.

github-actions · 2026-06-26T20:27:17Z

Changeset ✓

This PR includes a changeset covering all affected packages:

Package	Bump
`livekit`	`patch`
`livekit-api`	`patch`
`livekit-datatrack`	`patch`
`livekit-ffi`	`patch`
`livekit-protocol`	`patch`
`livekit-uniffi`	`patch`

1egoman · 2026-06-26T20:29:54Z

 bytes = "1.10.1"
 bmrng = "0.5.2"
+flate2 = "1"
 base64 = "0.22"


Note that flate2 is now a required dependency of livekit. Previously it was an optional dependency of livekit-api but that's it.

Not sure if this is is worth doing yet, but it might be worth adding a "compression" feature to this crate to conditionally include this dep. It would be enabled by default, but would provide users who do not want compression to disable it at the feature level.

I lean towards not doing it and adding it later if people wanted to opt out. But, I could be convinced otherwise if there's a really good use case.

My concern would be more how it affects compile time and binary size; probably insignificant but it might be worth checking with cargo build --timings and cargo bloat --release --crates (crate) respectively.

I ran cargo bloat --release --crates on rust-dev-client loading in a local copy of rust-sdks checked out to this branch.

Full output

File .text Size Crate 20.7% 36.0% 7.6MiB webrtc_sys 7.3% 12.8% 2.7MiB [Unknown] 4.9% 8.6% 1.8MiB std 2.4% 4.1% 891.6KiB naga 1.6% 2.8% 613.3KiB egui 1.6% 2.8% 600.6KiB livekit 1.3% 2.3% 501.2KiB tokio 1.3% 2.3% 491.4KiB wgpu_core 0.9% 1.6% 345.6KiB harfrust 0.9% 1.5% 320.6KiB rustls 0.8% 1.3% 288.6KiB image 0.7% 1.2% 258.5KiB skrifa 0.7% 1.2% 254.6KiB regex_automata 0.6% 1.1% 226.6KiB rust_dev_client 0.5% 0.9% 195.8KiB prost 0.5% 0.9% 193.4KiB zune_jpeg 0.5% 0.9% 190.2KiB libwebrtc 0.5% 0.8% 180.5KiB livekit_api 0.4% 0.8% 165.4KiB exr 0.4% 0.8% 162.8KiB read_fonts 0.4% 0.8% 162.2KiB livekit_protocol 0.4% 0.7% 157.7KiB image_webp 0.4% 0.6% 135.6KiB epaint 0.4% 0.6% 135.2KiB ring 0.3% 0.6% 125.4KiB wgpu_hal 0.3% 0.5% 118.4KiB regex_syntax 0.3% 0.5% 104.5KiB vello_cpu 0.3% 0.5% 104.3KiB tiff 0.3% 0.5% 101.3KiB reqwest 0.2% 0.4% 89.5KiB wgpu 0.2% 0.4% 88.8KiB aho_corasick 0.2% 0.4% 80.4KiB accesskit_consumer 0.2% 0.4% 75.4KiB livekit_datatrack 0.2% 0.3% 75.0KiB winit 0.2% 0.3% 68.6KiB accesskit_macos 0.2% 0.3% 66.9KiB egui_wgpu 0.2% 0.3% 64.0KiB eframe 0.2% 0.3% 61.0KiB gimli 0.2% 0.3% 59.4KiB tungstenite 0.2% 0.3% 59.3KiB hyper_util 0.2% 0.3% 58.6KiB http 0.1% 0.2% 53.8KiB webpki 0.1% 0.2% 52.8KiB egui_winit 0.1% 0.2% 50.1KiB url 0.1% 0.2% 48.0KiB serde_json 0.1% 0.2% 47.4KiB hashbrown 0.1% 0.2% 41.7KiB hyper 0.1% 0.2% 41.1KiB rustc_demangle 0.1% 0.2% 40.0KiB png 0.1% 0.2% 39.6KiB backtrace 0.1% 0.2% 38.0KiB livekit_data_stream 0.1% 0.2% 33.4KiB futures_util 0.1% 0.2% 33.4KiB smallvec 0.1% 0.1% 30.0KiB miniz_oxide 0.1% 0.1% 29.6KiB wgpu_types 0.1% 0.1% 28.6KiB autocfg 0.1% 0.1% 28.3KiB vello_common 0.1% 0.1% 28.0KiB tokio_tungstenite 0.1% 0.1% 26.6KiB fixedbitset 0.1% 0.1% 25.6KiB pollster 0.1% 0.1% 23.8KiB kurbo 0.1% 0.1% 23.2KiB addr2line 0.1% 0.1% 20.1KiB rustls_pki_types 0.1% 0.1% 19.8KiB weezl 0.1% 0.1% 19.2KiB codespan_reporting 0.0% 0.1% 18.6KiB tokio_rustls 0.0% 0.1% 18.3KiB idna 0.0% 0.1% 17.6KiB sha2 0.0% 0.1% 16.5KiB cc 0.0% 0.1% 15.4KiB proc_macro2 0.0% 0.1% 14.0KiB fdeflate 0.0% 0.1% 14.0KiB rayon_core 0.0% 0.1% 13.3KiB objc2 0.0% 0.1% 13.3KiB serde_core 0.0% 0.1% 12.7KiB serde 0.0% 0.1% 12.2KiB parking_lot_core 0.0% 0.1% 11.8KiB color 0.0% 0.1% 10.8KiB parking_lot 0.0% 0.0% 10.3KiB indexmap 0.0% 0.0% 9.9KiB anyhow 0.0% 0.0% 9.8KiB fearless_simd 0.0% 0.0% 9.7KiB rustls_native_certs 0.0% 0.0% 9.6KiB hyper_rustls 0.0% 0.0% 9.5KiB zune_inflate 0.0% 0.0% 9.4KiB icu_normalizer 0.0% 0.0% 8.8KiB zune_core 0.0% 0.0% 8.8KiB objc2_foundation 0.0% 0.0% 8.6KiB dispatch 0.0% 0.0% 8.4KiB memchr 0.0% 0.0% 8.4KiB objc2_encode 0.0% 0.0% 8.1KiB bytes 0.0% 0.0% 8.0KiB env_logger 0.0% 0.0% 7.3KiB object 0.0% 0.0% 6.8KiB http_body_util 0.0% 0.0% 6.5KiB petgraph 0.0% 0.0% 5.9KiB rfd 0.0% 0.0% 5.8KiB tower 0.0% 0.0% 5.5KiB flate2 0.0% 0.0% 5.2KiB os_info 0.0% 0.0% 5.0KiB semver 0.0% 0.0% 4.9KiB regex 0.0% 0.0% 4.5KiB base64 0.0% 0.0% 4.4KiB rand_chacha 0.0% 0.0% 4.2KiB gif 0.0% 0.0% 4.1KiB icu_provider 0.0% 0.0% 4.1KiB fax 0.0% 0.0% 4.1KiB httparse 0.0% 0.0% 4.0KiB sha1 0.0% 0.0% 3.4KiB prost_build 0.0% 0.0% 3.4KiB cxx 0.0% 0.0% 3.4KiB raw_window_handle 0.0% 0.0% 3.3KiB ipnet 0.0% 0.0% 3.3KiB objc2_app_kit 0.0% 0.0% 3.2KiB objc2_metal 0.0% 0.0% 3.2KiB compression_codecs 0.0% 0.0% 3.1KiB raw_window_metal 0.0% 0.0% 3.1KiB jsonwebtoken 0.0% 0.0% 3.0KiB arrayvec 0.0% 0.0% 2.9KiB syn 0.0% 0.0% 2.9KiB crossbeam_epoch 0.0% 0.0% 2.9KiB env_filter 0.0% 0.0% 2.7KiB jobserver 0.0% 0.0% 2.7KiB tempfile 0.0% 0.0% 2.6KiB jiff 0.0% 0.0% 2.6KiB anstream 0.0% 0.0% 2.2KiB ahash 0.0% 0.0% 2.2KiB log 0.0% 0.0% 2.0KiB livekit_runtime 0.0% 0.0% 2.0KiB futures_channel 0.0% 0.0% 2.0KiB crossbeam_deque 0.0% 0.0% 1.9KiB security_framework 0.0% 0.0% 1.7KiB qoi 0.0% 0.0% 1.7KiB anstyle 0.0% 0.0% 1.7KiB unicode_segmentation 0.0% 0.0% 1.7KiB potential_utf 0.0% 0.0% 1.7KiB async_compression 0.0% 0.0% 1.6KiB prost_types 0.0% 0.0% 1.5KiB rustix 0.0% 0.0% 1.5KiB crc32fast 0.0% 0.0% 1.5KiB mio 0.0% 0.0% 1.4KiB zmij 0.0% 0.0% 1.3KiB icu_collections 0.0% 0.0% 1.3KiB once_cell 0.0% 0.0% 1.3KiB foldhash 0.0% 0.0% 1.3KiB ecolor 0.0% 0.0% 1.2KiB emath 0.0% 0.0% 1.2KiB itoa 0.0% 0.0% 1.1KiB hmac 0.0% 0.0% 984B data_encoding 0.0% 0.0% 932B bit_set 0.0% 0.0% 920B hexf_parse 0.0% 0.0% 916B core_foundation 0.0% 0.0% 856B smol_str 0.0% 0.0% 804B form_urlencoded 0.0% 0.0% 780B accesskit 0.0% 0.0% 772B tower_http 0.0% 0.0% 768B version_check 0.0% 0.0% 752B signal_hook_registry 0.0% 0.0% 740B tokio_stream 0.0% 0.0% 728B simd_adler32 0.0% 0.0% 692B synstructure 0.0% 0.0% 676B percent_encoding 0.0% 0.0% 648B litrs 0.0% 0.0% 600B wgpu_naga_bridge 0.0% 0.0% 572B font_types 0.0% 0.0% 552B chrono 0.0% 0.0% 540B tracing_core 0.0% 0.0% 536B peniko 0.0% 0.0% 528B rand_core 0.0% 0.0% 496B uuid 0.0% 0.0% 488B av_scenechange 0.0% 0.0% 488B block2 0.0% 0.0% 416B libm 0.0% 0.0% 412B fastrand 0.0% 0.0% 396B livekit_common 0.0% 0.0% 392B futures_task 0.0% 0.0% 384B __rustc 0.0% 0.0% 356B errno 0.0% 0.0% 344B utf8_iter 0.0% 0.0% 332B want 0.0% 0.0% 332B unicode_width 0.0% 0.0% 248B signature 0.0% 0.0% 232B getrandom 0.0% 0.0% 224B arboard 0.0% 0.0% 224B shlex 0.0% 0.0% 200B num_bigint 0.0% 0.0% 196B find_msvc_tools 0.0% 0.0% 172B lock_api 0.0% 0.0% 152B built 0.0% 0.0% 128B cpufeatures 0.0% 0.0% 116B accesskit_winit 0.0% 0.0% 108B dispatch2 0.0% 0.0% 68B digest 0.0% 0.0% 64B bytemuck 0.0% 0.0% 52B half 0.0% 0.0% 48B untrusted 0.0% 0.0% 44B nix 0.0% 0.0% 32B zerovec 0.0% 0.0% 32B rand 57.4% 100.0% 21.0MiB .text section size, the file size is 36.7MiB Note: numbers above are a result of guesswork. They are not 100% correct and never will be.

Here are the relevant lines from the full output:

0.0% 0.0% 5.5KiB flate2 0.0% 0.0% 3.2KiB compression_codecs 0.0% 0.0% 1.7KiB async_compression

So it looks like ~10kb of size increase due to adding in async-compression and all dependencies. If you think it's worth adding the feature, let me know, but I'm inclined to leave it as is for now.

Breaks out a subset of the changes from #1192 into a pull request which exclusively moves the existing data streams implementation into its own crate, `livekit-data-streams`, and adds a new `livekit-common` crate (kept fairly minimal for now) for common utilities which `livekit-data-streams` and `livekit` both need. This is largely being scoped to a "lift and shift" type effort, more extensive refactoring of the `livekit-data-streams` crate will be done in #1192. ### Breaking changes There should not be any breaking changes from the perspective of `livekit`'s public api. If so, please surface them. ### Testing This pull request adds the relevant v1 data streams tests from #1192. More test coverage will be introduced as a follow up in the data streams v2 change. --------- Co-authored-by: Jacob Gelman <3182119+ladvoc@users.noreply.github.com>

…s and remove derive Default

…er compression mechanism was added in the future This should never happen but adding this extra check makes it just that much more bulletproof.

Test mode checks the contents of header.inline_content via is_inline, so .take() will clear this out and cause that field to be wrong on the receive end. However, this isn't currently being used in tests at all.

This is required for the python / node sdks to adopt the standardized data stream implementation, since it uses this feature for (I think) avatars

This allows configuring the data stream max payload size value when constructing a room.

The node and python sdks today have their own data streams implementations. If this branch were merged today, then it would mean that the new client_protocol=2 would be advertised, and these pre-existing implementations would receive data stream v2 messages, not v1 (ie, single packet data streams, compression, etc). So for now, add a flag which can be set in livekit-ffi to disable advertising the new client protocol so this can be merged. Once merged, node and python can be updated to pass this flag. Then, once data streams v2 is done on both node + python and both are updated, this flag can be removed by reverting this commit.

devin-ai-integration

Devin Review found 3 new potential issues.

View 3 additional findings in Devin Review.

devin-ai-integration · 2026-07-23T17:49:38Z

+        // --- Uncompressed (v1) stream: contiguous chunks, content delivered as-is. ---
+        if descriptor.progress.chunk_index != chunk.chunk_index {
+            inner.close_stream_with_error(&id, StreamError::MissedChunk);
+            return;
+        }
+
+        descriptor.progress.chunk_index += 1;
+        descriptor.progress.bytes_processed += chunk.content.len() as u64;
+
+        if match descriptor.progress.bytes_total {
+            Some(total) => descriptor.progress.bytes_processed > total,
+            None => false,
+        } {
+            inner.close_stream_with_error(&id, StreamError::LengthExceeded);
+            return;
+        }
+        inner.yield_chunk(&id, Bytes::from(chunk.content));
+        inner.publish_progress(&id);


🟡 Maximum payload size limit is ignored for uncompressed data streams

The receiver's configured maximum payload size is only enforced while decompressing compressed streams; an incoming uncompressed multi-packet stream (livekit-data-stream/src/incoming/manager.rs:444-461) is never measured against that limit, so its documented guarantee does not hold for uncompressed traffic.
Impact: A remote peer can push an uncompressed stream far larger than the receiver's configured cap without the promised error, defeating the size safeguard and risking excessive memory use.

Where the cap is applied vs. skipped

The cap (max_payload_byte_length, documented on RoomDataStreamOptions::with_max_payload_byte_length at livekit/src/room/mod.rs:411-418 as: "If a data stream payload goes above this size, then a StreamError::PayloadTooLarge will be thrown") is threaded into the incoming Manager and consulted only by DeflateDecompressState::push (livekit-data-stream/src/incoming/manager.rs:100-104) and inflate_raw (:140-142) — i.e. only the compressed and inline-compressed paths.

The uncompressed branch of handle_chunk (livekit-data-stream/src/incoming/manager.rs:444-459) only compares bytes_processed against the sender-declared bytes_total (yielding LengthExceeded) and never against max_payload_byte_length. A stream that declares a huge total_length (or an unknown-size stream where bytes_total is None, so no check runs at all) accumulates without ever emitting PayloadTooLarge, contradicting the public contract.

Prompt for agents

The receiver-side max payload guard (max_payload_byte_length, documented on RoomDataStreamOptions::with_max_payload_byte_length in livekit/src/room/mod.rs to throw StreamError::PayloadTooLarge when a payload exceeds the size) is only enforced for compressed streams (DeflateDecompressState::push and inflate_raw). The uncompressed multi-packet path in Manager::handle_chunk (the branch after the `if let Some(decompressor)` block) only checks bytes_processed against the sender-declared bytes_total (LengthExceeded), never against self.max_payload_byte_length. Streams with an unknown size (bytes_total = None) get no bound at all. Consider adding a check in the uncompressed branch that closes the stream with StreamError::PayloadTooLarge once cumulative bytes_processed exceeds max_payload_byte_length, so the documented guarantee applies uniformly to compressed and uncompressed streams. The Manager already stores max_payload_byte_length as a field.

Was this helpful? React with 👍 or 👎 to provide feedback.

This was done on purpose, but I am starting to maybe rethink this and I might make this apply everywhere just to be consistient.

devin-ai-integration · 2026-07-23T17:49:41Z

+    async fn push(&mut self, input: &[u8]) -> StreamResult<Vec<u8>> {
+        use futures_util::io::AsyncWriteExt;
+
+        self.decoder.write_all(input).await.map_err(|_| StreamError::Decompression)?;
+
+        // Flush so all currently-decodable output lands in the inner `Vec`.
+        self.decoder.flush().await.map_err(|_| StreamError::Decompression)?;
+
+        let output_bytes = std::mem::take(self.decoder.get_mut());
+        self.output_bytes_length += output_bytes.len();
+        if self.output_bytes_length > self.max_byte_length {
+            return Err(StreamError::PayloadTooLarge);
+        }
+
+        Ok(output_bytes)
+    }


🟨 Per-chunk deflate decompression buffers full chunk output before size check

In the streaming decompressor DeflateDecompressState::push (livekit-data-stream/src/incoming/manager.rs:92-107), each compressed chunk (up to ~15 KB on the wire) is fully written and flushed into an in-memory Vec before the accumulated output length is compared against max_payload_byte_length. A single highly-compressible chunk can therefore inflate to a large buffer (a per-chunk decompression bomb) before the guard fires and rejects the stream.

Was this helpful? React with 👍 or 👎 to provide feedback.

1egoman commented Jun 26, 2026

View reviewed changes

ladvoc reviewed Jun 26, 2026

View reviewed changes

Comment thread livekit/src/prelude.rs Outdated

ladvoc reviewed Jun 26, 2026

View reviewed changes

Comment thread livekit/src/room/participant/remote_participant.rs Outdated

ladvoc reviewed Jun 26, 2026

View reviewed changes

Comment thread livekit/src/room/participant/mod.rs Outdated

1egoman mentioned this pull request Jun 29, 2026

Add participant capabilities to participant list livekit-examples/rust-dev-client#19

Draft

1 task

1egoman force-pushed the data-streams-v2 branch 3 times, most recently from 3bf1532 to d9e9bb3 Compare July 6, 2026 20:05

1egoman commented Jul 6, 2026

View reviewed changes

Comment thread livekit-data-stream/src/incoming/mod.rs Outdated

Comment thread livekit-data-stream/src/incoming/stream_reader.rs

Comment thread livekit-data-stream/src/outgoing/mod.rs Outdated

Comment thread livekit-data-stream/src/outgoing/mod.rs Outdated

1egoman marked this pull request as ready for review July 6, 2026 21:15

1egoman requested review from MaxHeimbrock, alan-george-lk, lukasIO, stephen-derosa and xianshijing-lk as code owners July 6, 2026 21:15

This comment was marked as resolved.

Sign in to view

1egoman mentioned this pull request Jul 7, 2026

Move data streams into their own crate #1227

Merged

1egoman force-pushed the data-streams-v2 branch from 15c2d7b to 9daf2a9 Compare July 8, 2026 19:21

1egoman marked this pull request as draft July 8, 2026 19:22

1egoman removed request for MaxHeimbrock, alan-george-lk, lukasIO, stephen-derosa and xianshijing-lk July 8, 2026 19:23

1egoman changed the base branch from main to data-streams-refactor July 8, 2026 19:24

This comment was marked as resolved.

Sign in to view

Base automatically changed from data-streams-refactor to main July 9, 2026 20:00

1egoman force-pushed the data-streams-v2 branch from 339361a to 2552a56 Compare July 9, 2026 20:34

1egoman added 26 commits July 23, 2026 13:32

feat: add max data stream size limit configurable on room level

7a86ee8

fix: ensure that decompress state tracks output bytes, not input bytes

a24b792

refactor: adjust StreamByteOptions / StreamTextOptions to use builder…

5762dde

…s and remove derive Default

feat: add e2e test to verify max data stream room length option works

8cb1a5b

refactor: rename on_ -> handle_

056bf93

fix: run cargo fmt

7c7d9f8

docs: add livekit-data-stream AGENTS.md file

14b07d1

feat: add data stream max size limit to inline payload path as well

2d0e855

fix: use StreamByteOptions builder in example

39979fd

feat: add enforce_header_size unit tests

8dc1fae

fix: update remaining examples to use new_with_topic

27536fb

feat: add support for trailer attributes to data streams

5d25bd6

fix: update rpc tests to not set attributes field directly

64288b4

fix: update data stream tests to use .attributes()

80a028e

feat: add "CompressionType::Unrecognized" to handle cases where a new…

adffeab

…er compression mechanism was added in the future This should never happen but adding this extra check makes it just that much more bulletproof.

refactor: add nested modules to incoming / outgoing managers

1763be7

feat: add more test coverage to incoming / outgoing managers

4e9c62a

refactor: rename collect -> as_bytes

deb2ada

fix: clone inline_content when testing

d23f1ea

Test mode checks the contents of header.inline_content via is_inline, so .take() will clear this out and cause that field to be wrong on the receive end. However, this isn't currently being used in tests at all.

fix: get rid of deflate_raw_i32 / none_i32

a69fa52

feat: add ability to override data stream sender identity

05402a0

This is required for the python / node sdks to adopt the standardized data stream implementation, since it uses this feature for (I think) avatars

fix: ensure that passed file name is respected

03ad4bb

feat: add RoomDataStreamOptions to livekit-ffi

e811a8d

This allows configuring the data stream max payload size value when constructing a room.

feat: add support for data stream trailer attributes via livekit-ffi

88a7b30

fix: run cargo fmt

f94a5d5

1egoman force-pushed the data-streams-v2 branch from 79a8851 to f94a5d5 Compare July 23, 2026 17:40

devin-ai-integration Bot reviewed Jul 23, 2026

View reviewed changes

This was referenced Jul 23, 2026

Bump livekit-ffi to latest livekit/node-sdks#698

Draft

Bump livekit ffi livekit/python-sdks#766

Draft

Uh oh!

Conversation

1egoman commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Uh oh!

github-actions Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changeset ✓

Uh oh!

Uh oh!

Uh oh!

1egoman Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

ladvoc Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

1egoman Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ladvoc Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

1egoman Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

1egoman Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

devin-ai-integration Bot Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

1egoman commented Jun 26, 2026 •

edited

Loading

github-actions Bot commented Jun 26, 2026 •

edited

Loading

1egoman Jun 29, 2026 •

edited

Loading