Apply pre-render object positions and head rotation from the first frame - #22
Closed
trsonic wants to merge 11 commits into
Closed
Apply pre-render object positions and head rotation from the first frame#22trsonic wants to merge 11 commits into
trsonic wants to merge 11 commits into
Conversation
Ports the fork's MODULE.bazel, BUILD.bazel, extensions.bzl and third_party/pffft.BUILD onto the public repo layout (code at the repository root instead of liboar/).
Bazel globs do not cross package or module boundaries; the vendored OBR subtree's BUILD and MODULE.bazel files would stop the top-level targets from picking up its sources.
Azimuths outside (-180, 180] (e.g. +270 for -90) skipped the front-back fold and hit the closest-speaker fallback, whose linear angle difference picks the wrong speaker whenever the true separation crosses the +/-180 seam. Normalize azimuths once at metadata ingestion so every downstream consumer sees canonical angles, and make the fallback distance circular.
Renders aliased azimuth pairs (+270 vs -90 etc.) to stereo and requires bit-identical output, and checks lateral sources collapse onto opposite speakers. Before the fix a source at +270 landed on the left speaker.
enable_testing() at the top level puts the obr unit tests and the liboar examples in one ctest registry; the examples are registered under an "oar" label (mirroring the obr subtree's "obr" label) derived from the directory's target list so new examples cannot be silently left out. The CI workflow's hardcoded example list and separate obr ctest invocation collapse into a single label-filtered ctest run.
_open stored the address of the stack-local pout in ear_renderer->out_sp_layout, which lives for the renderer's lifetime. The pointer is only dereferenced during _open itself (matrix lookups), so the bug is latent, but any future reader of out_sp_layout.sp_layout.predefined_sp after _open returns would hit a dangling stack pointer. Null it before returning, matching how the input-layout paths already reset pin/cin.
The ARM NEON matrix-render, OBR resampler/sh_hrir DSP, and LFE filter sample-rate fixes were merged upstream (AOMediaCodec/oar #14, #15 and the matrix_render include commit), so they drop off the pending list. The OLR azimuth wrapping, ctest registration, and EAR dangling-pointer fixes take their place.
An object source is registered with the ambisonic encoder at its default position (azimuth 0, elevation 0, distance 1) as soon as the audio element is added. A position update arriving before the first render call therefore only moved the ramp target, and the first processed block audibly glided the object from front-center to its configured position. Loudspeaker rendering (OLR) applies pre-render metadata immediately, so binaural output disagreed with loudspeaker output over the first block. Track whether the encoder has processed any audio yet; until it has, SetSource() snaps the current parameters to the new target instead of scheduling a ramp. There is no previously audible position to interpolate from, so metadata applied before the first render now takes effect from the first frame, matching OLR. Updates arriving after audio has been rendered still ramp across one block to avoid clicks. Reported upstream as #21.
The ambisonic rotator's current rotation starts at identity and only the target rotation is supplied per Process() call, so a head pose set before the first render call made the whole scene slerp from identity to that pose across the first processed block. Apply the same rule as for object positions: until any audio has been rendered there is no previously audible rotation to interpolate from, so the first processed block applies the target rotation in full from the first frame. Rotation changes after audio has been rendered still slerp across the block in 32-frame intervals. Blocks rendered with head tracking disabled bypass the rotator but are audible at the identity rotation, so the processing group marks them via MarkAudioRendered(); enabling head tracking with a stored pose after such blocks still slerps instead of snapping.
Collaborator
Author
|
Withdrawing for now; will resubmit later. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #21.
Problem
As analyzed in #21, OLR and OBR disagree on metadata applied before the first render call. When an audio element is added, OBR registers each object source with the ambisonic encoder at the default position (azimuth 0, elevation 0, distance 1). A position update arriving before the first
Process()call only moves the ramp target, so the first rendered block audibly glides the object from front-center to its configured position. OLR applies pre-render metadata immediately, so loudspeaker and binaural output differ over the first block. The same pattern exists in the ambisonic rotator: a head pose set before the first render slerps in from identity.Fix
Until a component has rendered any audio there is no previously audible state to interpolate from, so updates snap instead of scheduling a ramp:
AmbisonicEncodertracks whether it has processed a block; while it hasn't,SetSource()setscurrentalong withtarget. This covers every pre-render update path (object channels, loudspeaker-channel positions, gains) since they all funnel throughSetSource().AmbisonicRotatordoes the same for the target rotation on its first processed block.ProcessingGroupmarks them viaMarkAudioRendered(); enabling head tracking with a stored pose after such blocks still slerps instead of snapping.Updates arriving after audio has been rendered still ramp/slerp across one block to avoid clicks, and mid-stream update semantics are unchanged.
Tests
AmbisonicEncoderTest.PreRenderPositionUpdateTakesEffectImmediately— reproduces the Question: Initial object position OLR vs OBR #21 glide on unfixed code (frame 0 carries the front-position coefficients instead of the configured position).AmbisonicEncoderTest.PostRenderPositionUpdateRampsAcrossBlock— guards the click-avoidance ramp after audio has flowed.AmbisonicRotatorTest.FirstProcessedBlockAppliesTargetRotationInFull,RotationAfterBypassedBlocksStillSlerps,RotationChangeAfterFirstBlockStillSlerps— same contract for the rotator, including the head-tracking-disabled case.Each "first block" test fails without its fix; the existing encoder and rotator tests pass unchanged.