Playback: Zero-latency path to MIDI synth tracks (plan)

[drowaudio]

Summary

Plan for a zero-latency monitoring path for live MIDI input into synth tracks. This is an engine change (modules/tracktion_engine + modules/tracktion_graph) - no app-side work is needed for the graph plumbing, only for whatever UI eventually toggles it.

No implementation yet: the approach needs a decision from the maintainer (see Open questions), in particular whether the goal is "remove the avoidable compensation delay" or "route the monitored track straight to the output, bypassing the master bus".

Where the delay actually comes from

Live MIDI reaches the graph via LiveMidiInjectingNode (keyboard/step entry/previews, added in createNodeForAudioTrack, tracktion_EditNodeBuilder.cpp:1592) or via MidiInputDeviceNode / HostedMidiInputDeviceNode through createLiveInputsNode. Both are injected upstream of the track's plugin chain, so a note played now is heard after the full PDC delay. That delay has three distinct components:

1. Latency of plugins downstream of the injection point on the track itself - the synth's own reported latency plus any track FX. This is real processing latency and is irreducible without bypassing those plugins (which is what the existing Edit::setLowLatencyMonitoring already does, bluntly, by disabling them).
2. LatencyNodes inserted to align this track's branch with more-latent sibling branches - SummingNode::createLatencyNodes (tracktion_SummingNode.h:267) and the same logic duplicated in ConnectedNode (tracktion_ConnectedNode.h:272). This is pure compensation delay and is entirely avoidable for a live path.
3. Latency of the master plugin chain and anything post-track (createMasterPluginsNode). Avoidable only by routing around it.

The maintainer's comment on the issue proposes attacking (2) by marking the live node so compensation is skipped. That is the right first step, but it is worth being explicit that it does not address (3): a linear-phase EQ or lookahead limiter on the master bus will still delay the monitored note by its full latency.

The structural problem with "just the live path"

The issue comment asks whether the exemption can be applied to the live path only. It cannot, as the graph is currently shaped:

• LiveMidiInjectingNode wraps the whole clips node, so live MIDI and timeline MIDI are merged into one buffer before reaching the synth.
• Even if they were kept separate, they must converge at the synth - there is one plugin instance, so there is no point downstream of it at which the two streams can be treated differently.

So the exemption necessarily applies to the whole track branch. That is only correct when the track has no timeline material that must stay in sync with the rest of the Edit - which is precisely the record-armed/input-monitored case, where clips are already muted by the TrackMutingNode pair in createNodeForAudioTrack ("When recording, clips should be muted but the plugin should still be audible"). Hence the gating question below.

Option 1 - compensation-exempt branches (recommended first step)

Addresses (2). Localised, testable at the tracktion_graph level, and is the foundation for anything else.

Mechanism

• Add an explicit bool bypassLatencyCompensation to NodeProperties (tracktion_Node.h:150).

  Prefer this over encoding it as a negative latencyNumSamples. latencyNumSamples is already max'd/accumulated in about eight places, std::numeric_limits<int>::min() is already used as the "no inputs yet" sentinel in SummingNode::getNodeProperties and InsertSendNode, and subtractNoWrap exists specifically to cope with that sentinel. A second negative meaning would be silently absorbed by those std::max calls.

• Propagation rule: pass-through nodes propagate the flag; merging nodes compute latencyNumSamples as the max over non-exempt inputs only and report bypassLatencyCompensation = false, since the merge re-establishes a compensated stream. The merge sites are:

  • SummingNode::getNodeProperties / createLatencyNodes
  • ConnectedNode::getNodeProperties and its latency loop
  • CombiningNode (tracktion_CombiningNode.cpp:164)
  • ArrangerLauncherSwitchingNode (tracktion_ArrangerLauncherSwitchingNode.cpp:54)
  • RackReturnNode (tracktion_RackReturnNode.cpp:44) and RackNode (tracktion_RackNode.cpp:99)
  • ReturnNode (tracktion_TestNodes.h:568)

• SummingNode::createLatencyNodes and the ConnectedNode equivalent skip exempt inputs entirely (no LatencyNode wrapped around them).

• Set the flag in LiveMidiInjectingNode::getNodeProperties(), and in MidiInputDeviceNode / HostedMidiInputDeviceNode for hardware MIDI in, when the owning track opts in.

Invariant that must hold: the root node's latencyNumSamples must be unchanged by the presence of an exempt branch. PlayHeadPositionNode (tracktion_PlayHeadPositionNode.h:47) and EditPlaybackContext::getLatencySamples() (tracktion_EditPlaybackContext.cpp:209) both read it; if an exempt branch pulled it down, timeline sync and the host-reported latency would both break.

Complexity: medium.

Option 2 - direct monitor tap

Addresses (2) and (3). The armed track's post-plugin output is sent to a dedicated monitor bus (the existing SendNode/ReturnNode bus mechanism already used for getWaveInputDeviceBusID / sidechains fits) and summed in at the device output node, after createMasterPluginsNode, with the track's normal contribution suppressed while monitoring.

The signal cannot simply be added in parallel - the same audio would then be heard twice, once early and once delayed - so the track's normal path has to be muted for the duration, which changes what the user hears (no master-bus processing on the monitored track). That is a defensible and common "direct monitoring" behaviour, but it is a product decision, not an implementation detail.

Complexity: large.

Option 3 - hybrid

Option 1 as the default behaviour for armed tracks, Option 2 behind an explicit per-track "direct monitoring" option. Probably where this ends up, but Option 1 should land and be proven first.

Affected files

• modules/tracktion_graph/tracktion_graph/tracktion_Node.h - NodeProperties flag
• modules/tracktion_graph/tracktion_graph/nodes/tracktion_SummingNode.h - props + createLatencyNodes
• modules/tracktion_graph/tracktion_graph/nodes/tracktion_ConnectedNode.h - same, for the multi-threaded player
• modules/tracktion_graph/tracktion_graph/tracktion_TestNodes.h - ReturnNode, test summing nodes
• modules/tracktion_engine/playback/graph/tracktion_LiveMidiInjectingNode.{h,cpp} - set the flag
• modules/tracktion_engine/playback/graph/tracktion_MidiInputDeviceNode.{h,cpp}, tracktion_HostedMidiInputDeviceNode.{h,cpp} - set the flag
• modules/tracktion_engine/playback/graph/tracktion_CombiningNode.cpp, tracktion_ArrangerLauncherSwitchingNode.cpp, tracktion_RackReturnNode.cpp, tracktion_RackNode.cpp - merge rule
• modules/tracktion_engine/playback/graph/tracktion_EditNodeBuilder.cpp - gating in createNodeForAudioTrack / createLiveInputsNode
• modules/tracktion_engine/model/edit/tracktion_Edit.{h,cpp} and/or AudioTrack - whatever property gates it

Tests

New, in the existing TRACKTION_UNIT_TESTS blocks:

• tracktion_Node.test.cpp: a SummingNode with one high-latency branch and one exempt branch - assert no LatencyNode is wrapped around the exempt branch, the root latencyNumSamples is unchanged, and the exempt branch's signal arrives at block 0 while the compensated branch stays aligned.
• tracktion_ConnectedNode.test.cpp: the same case through the multi-threaded player, since ConnectedNode duplicates the compensation logic and would otherwise diverge.
• Engine-level: an Edit with a latency-inducing plugin on a second track plus a LiveMidiInjectingNode on the first - assert the injected note appears undelayed.

Must stay green: the existing latency tests in tracktion_Node.test.cpp, tracktion_NodeVisiting.test.cpp, tracktion_RackBenchmarks.test.cpp, and the disableLatencyCompensation path added in b5d5f9a.

Edge cases

• Rendering/freezing must never be exempt - params.forRendering has to force it off, or renders become sample-inaccurate.
• Aux sends and sidechains fed from an exempt branch carry the uncompensated copy into a compensated destination.
• TrackWaveInputDeviceNode derives latencyUpToThisPoint from node properties to timestamp recorded material (tracktion_TrackWaveInputDeviceNode.cpp:45); changing branch latency changes punch-in offsets.
• Racks compute latency through RackNode/RackInstanceNode, including automationAdjustmentTime derived from latency.
• Multiple live inputs on one track are summed by createLiveInputsNode before the track chain.
• MIDI output devices (MidiOutputDeviceInstanceInjectingNode) are a separate path and are not covered here.
• Interaction with the existing Edit::setLowLatencyMonitoring (which disables plugins) and Edit::setLatencyCompensationEnabled (Edit-wide off switch) needs defining - all three should not fight each other.

Open questions

1. Is Option 1 enough? It removes the inter-track alignment delay but leaves master-bus plugin latency in place. If the expectation from "without having to go through the entire graph" is that master FX are bypassed too, Option 2 is required.
2. What turns it on? Derived automatically from record-arm + monitor mode, an explicit per-track property, or an Edit-wide setting?
3. Is an exempt track's timeline material playing early acceptable? Since the exemption cannot be confined to the live path, either it is gated on the track's clips being muted (record-armed), or the user accepts that clips on that track run ahead of the rest of the Edit.
4. Explicit NodeProperties bool, or the negative latencyNumSamples sentinel from the issue comment? I'd recommend the bool for the reasons above, but it is your call.

Fixes #209

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.81%. Comparing base (aa6301a) to head (38541e7).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #431      +/-   ##
===========================================
- Coverage    59.81%   59.81%   -0.01%     
===========================================
  Files          568      568              
  Lines        79964    79964              
  Branches     12381    12379       -2     
===========================================
- Hits         47828    47827       -1     
- Misses       32136    32137       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.