Audio Session Defaults - #1019
Conversation
|
📱 TF build 2.15.0.b20260527 |
|
I think I'll postpone it till minor release with better docs/changelog. |
|
cc @MaxHeimbrock maybe good to match platform audio here? (or vice versa) |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Thanks for the cc. I am currently improving platform audio with Unity where Unity itself also tries to configure the audio session. This might have implications that are different from the ones in Swift. Once the Unity platform audio integration is more robust I will try to see if we can align it more with Swift. Btw how do you test this, do you do a lot of on device testing? |
Muting the microphone drops the recording requirement, which previously flipped the category back to .playback. A mid-session category change tears down Apple's Voice Processing I/O, so echo cancellation stayed off until the session ended. A sticky `hasRecorded` bit, cleared when the WebRTC engine stops, holds .playAndRecord across mute toggles. It is keyed to the engine's own session requirement, so an externally held requirement (a prepared SoundPlayer sound) does not pin the category after the call. Category selection moves into a pure `selectConfiguration(state:)` so it can be tested directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Letting another app share the audio device during a call is a known cause of echo, and the WebRTC ADM carries a retry loop to work around a -66637 (kAudioUnitErr_Initialized) engine-init race that only appears when the option is set (related symptom: #1011). The option entered the .playAndRecord defaults during the AVAudioEngine ADM rewrite (#536), having been added in #228 and removed again in #260. It stays on the .playback preset, where mixing is correct for a listener. This matches what client-sdk-flutter already configures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
d231636 to
23a1e6b
Compare
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
23a1e6b to
6ec10a3
Compare
|
I narrowed the scope of this PR a bit, to avoid speculative changes, focusing on:
The impact of resetting after call is hard to estimate atm + it can be implemented on the consumer side with moderate effort. |
3 participants
Reworks the iOS audio session defaults. No public API changes; observable behavior changes are listed below with rationale.
.playAndRecordis sticky for the life of the engine session. Category selection is now a pureselectConfiguration(state:)driven by the engine state plus ahasRecordedbit:.playAndRecordonce recording engages, kept until the WebRTC engine stops,.playbackotherwise. Muting the microphone no longer flips the category back to.playback— a mid-session category change tears down Apple's Voice Processing I/O and leaves echo cancellation dead for the rest of the call. The bit is keyed to the engine's own session requirement, so external requirement holders (e.g. a preparedSoundPlayersound, which lives for the lifetime of the prepared sound) don't pin.playAndRecordafter the call ends. Covered by new hermetic unit tests (AudioSessionCategorySelectionTests)..mixWithOthersdropped from the.playAndRecordpresets (kept on the listener.playbackpreset). On the in-call path it lets another app hold the audio device alongside the call — a known echo source (see e.g. Agora's echo troubleshooting) — and the WebRTC ADM needs a retry loop to work around a-66637engine-init race when it is set (related: LiveKit participant audio becomes inaudible during incoming calls or alarms #1011, Audio Recording Fails to Resume System Audio After Incoming Call Interruption in Background #886). It has a turbulent history here — added in Adjust AVAudioSession CategoryOptions (include .mixWithOthers) #228, removed in Remove.mixWithOtherswhen category.playAndRecord#260, re-entered during the AVAudioEngine ADM rewrite (AVAudioEngine version AudioDeviceModule #536); this is a clean revert. This matches the Flutter SDK'scommunicationpreset (allowBluetooth/A2DP/allowAirPlay, nomixWithOthers). Note the user-visible consequence: publishing the mic now interrupts other apps' audio for the rest of the call; muting doesn't bring it back.Docs/audio.mdgains an "Audio session category selection" section covering the rules above, what counts as "recording", and the mic-permission implications.Compared to earlier revisions of this PR:
canPublishMicrophone→.playAndRecord) is gone.canPublishdefaults to true in standard tokens, so it made.playbackunreachable — every subscriber would have activated a.playAndRecordsession (mic prompt; TCC kill for apps withoutNSMicrophoneUsageDescription). The viewer-mic-prompt bug it targeted is native — the ADM's teardown path touchedAVAudioEngine.inputNode, which alone instantiates the input unit — and is fixed by Do not instantiate AVAudioEngine.inputNode on playout-only engine teardown webrtc-sdk/webrtc#286, shipped in 150.7871.01 (Bump WebRTC to 150.7871.01 #1103)..ambientcategory reset before deactivation (volume-rocker register fix) was dropped: no other iOS SDK we checked (LiveKit Flutter/React Native, Stream Video, Agora, Twilio, upstreamRTCAudioSession) resets or restores the category on teardown, and.ambientwould break apps that manage their own category (no background audio, silent-switch muting). A restore-previous-configuration approach may follow separately..allowAirPlayis kept. QA1803 only documents implied options for.voiceChat/.videoChat(allowBluetooth,defaultToSpeaker); it doesn't back removing AirPlay, and Flutter keeps it too.