fix(pipeline): link the byte-stream capsfilter after h264parse, not before - #36
Merged
Merged
Conversation
…efore
Starting an AirPlay cast on macOS failed immediately, before any network
traffic:
Selected encoder "vtenc_h264" VideoToolbox H.264 (Hardware) hw=true
ERROR AirPlay casting failed
e=GStreamer error: Failed to link elements 'vtenc_h264-1' and 'capsfilter1'
Capture and encoder probing both worked; the pipeline would not assemble.
`vtenc_h264` advertises `stream-format: avc` on its src pad as a fixed
string, not a list, so it cannot emit byte-stream at all. The AirPlay
chain demanded byte-stream directly from the encoder:
encoder -> capsfilter(byte-stream) -> h264parse -> appsink
leaving an empty caps intersection. h264parse is the element that converts
avc to byte-stream, so it has to come first. `x264enc` offers
`{ avc, byte-stream }`, which is why the same chain links on Linux and why
this went unnoticed — the Miracast pipeline already uses the correct shape
(`encoder -> h264parse -> mpegtsmux`), which is why Miracast works.
Reduced, the old order reproduces the failure exactly and the new one does
not:
$ gst-launch-1.0 videotestsrc ! vtenc_h264 \
! 'video/x-h264,profile=high,stream-format=byte-stream' ! h264parse ! fakesink
WARNING: erroneous pipeline: could not link vtenc_h264-0 to h264parse0,
vtenc_h264-0 can't handle caps video/x-h264, profile=high, stream-format=byte-stream
$ gst-launch-1.0 videotestsrc ! vtenc_h264 ! h264parse \
! 'video/x-h264,stream-format=byte-stream,alignment=au' ! fakesink
(clean)
Also drops the `profile=high` constraint. `vtenc_h264` has no profile
property and does not advertise the field, so pinning it would just move
the negotiation failure one element downstream.
`sender_pipeline.rs` (the WebRTC path) carried the same over-constraint and
is fixed alongside, though it has no callers so this is latent rather than
observed. It needs no h264parse: `rtph264pay` accepts avc and byte-stream
alike, so constraining alignment only lets the two negotiate.
snadahalli
force-pushed
the
fix/airplay-macos-encoder-link
branch
from
August 26, 2026 09:38
435530a to
557137a
Compare
Developer1010x
pushed a commit
that referenced
this pull request
Aug 26, 2026
…43) #42 was merged twelve seconds after #41, into `fix/airplay-hkp-header` — which #41 had just merged into master and left behind. The merge succeeded, so nothing looked wrong, but the commits landed on a branch nothing points at and master never received them. Master therefore has the `X-Apple-HKP` header from #41 and none of what it was a prerequisite for: the SRP proof still hashes g padded, transient pairing still runs M5/M6 and gets the connection closed, and there is no encrypted control channel. Pairing is broken on master in exactly the way #42 fixed. This restores #42's own diff — the eight `openplay-airplay` files it actually touched — on top of current master. Deliberately *not* a merge of `fix/airplay-hkp-header`. That branch was cut before #35, #36, #37, #39 and #40 merged, so a diff against it reads as deleting `openplay-discovery/src/address.rs` and reverting 244 lines of `openplay-sender/src/app.rs`. Merging it would silently undo five landed fixes. Only the range between #42 and its own parent is safe to replay, and that range touches nothing outside `openplay-airplay`. Verified after the replay: 233 tests pass, clippy and fmt clean, and the work from #35/#36/#37/#40 is still in the tree. Co-authored-by: Sandeepa Nadahalli <1698507+snadahalli@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Starting an AirPlay cast on macOS fails immediately, before any network traffic:
Capture and encoder probing both succeed. The pipeline itself will not assemble.
Why
vtenc_h264advertisesstream-format: avcon its src pad as a fixed string, not a list — it cannot emit byte-stream at all. The AirPlay chain demanded byte-stream directly from the encoder:h264parseis the element that converts avc to byte-stream, so it has to come first.x264encoffers{ avc, byte-stream }, which is why the identical chain links on Linux and why this survived — and the Miracast pipeline already uses the correct shape (encoder → h264parse → mpegtsmux), which is why Miracast works.Reduced, the old order reproduces the failure exactly and the new one does not:
The fix
Move the capsfilter after
h264parse, and dropprofile=high—vtenc_h264has no profile property and does not advertise the field, so pinning it would just move the negotiation failure one element downstream.sender_pipeline.rs(the WebRTC path) carried the same over-constraint and is fixed alongside. It is latent rather than observed there, since that path has no callers. It needs noh264parse:rtph264payaccepts avc and byte-stream alike, so constraining alignment only lets the two negotiate.Verification
fmt --checkclean,clippy --all-targets --all-features -D warningsclean,cargo test --all207 passed / 0 failed on macOS arm64, plus the reducedgst-launch-1.0cases above.Not verified end to end: completing a cast additionally needs Screen Recording permission and a receiver that accepts the connection (see #27), neither of which this PR touches. What is verified is that the pipeline now links.