streamcoreai · jason-shen · Aug 26, 2026
diff --git a/README.md b/README.md
@@ -75,7 +75,7 @@ Docker, TURN ports, and production notes: [Quick start guide](./docs/quickstart.
 | **Transport** | WebRTC audio over WHIP ([RFC 9725](https://www.rfc-editor.org/rfc/rfc9725.html)) — one HTTP POST, no signaling socket. Opus/RTP both ways |
 | **Connectivity** | Built-in Pion STUN/TURN on UDP *and* TCP 3478 — no external coturn. A network handover or NAT rebind is recovered by ICE restart on the same session, so the conversation survives it |
 | **Turn-taking** | Adaptive VAD that tracks each call's noise floor, plus a debounce that merges mid-sentence pauses into one turn |
-| **Interruption** | Barge-in that ducks agent audio, filters backchannels ("mm-hm"), and cancels in-flight LLM and TTS on a confirmed interrupt |
+| **Interruption** | Barge-in that ducks agent audio, filters backchannels ("mm-hm"), and cancels in-flight LLM and TTS on a confirmed interrupt. On paths with no echo cancellation, such as telephony, the threshold is bounded by what the agent just sent so it never interrupts itself |
 | **Streaming** | Streaming STT → streaming LLM → chunk-streaming TTS, so audio starts before synthesis finishes |
 | **Sessions & events** | Server-generated session IDs, multi-peer sessions, DataChannel events for transcript, response, state, and per-turn latency |
 | **Reach** | Browser, mobile, backend, CLI, [SIP telephony](https://github.com/streamcoreai/sip-server), and [ESP32](https://github.com/streamcoreai/esp32) endpoints |

diff --git a/README.zh-CN.md b/README.zh-CN.md
@@ -75,7 +75,7 @@ Docker、TURN 端口与生产部署注意事项见[快速开始指南](./docs/qu
 | **传输** | 基于 WHIP（[RFC 9725](https://www.rfc-editor.org/rfc/rfc9725.html)）的 WebRTC 音频 —— 一次 HTTP POST，无需常驻信令连接。双向 Opus/RTP |
 | **连通性** | 内置 Pion STUN/TURN，同时监听 UDP 与 TCP 3478 —— 无需额外的 coturn。网络切换或 NAT 重新绑定可在同一会话上通过 ICE restart 恢复，对话不会因此中断 |
 | **轮次控制** | 自适应 VAD 跟踪每通电话的噪声基线，并用去抖把句中停顿合并为同一轮 |
-| **插话打断** | Barge-in 先压低智能体音量，过滤 "嗯嗯" 这类回应词，确认打断后取消进行中的 LLM 与 TTS |
+| **插话打断** | Barge-in 先压低智能体音量，过滤 "嗯嗯" 这类回应词，确认打断后取消进行中的 LLM 与 TTS。在没有回声消除的链路（如电话）上，打断阈值会以智能体刚发出的音频为下限，因此不会被自己的声音打断 |
 | **流式链路** | 流式 STT → 流式 LLM → 分块流式 TTS，合成尚未结束音频就已开始播放 |
 | **会话与事件** | 服务端生成会话 ID、多 peer 会话，DataChannel 推送转写、回复、状态与每轮延迟 |
 | **接入范围** | 浏览器、移动端、后端服务、CLI、[SIP 电话](https://github.com/streamcoreai/sip-server) 与 [ESP32](https://github.com/streamcoreai/esp32) 设备 |

diff --git a/config.toml.example b/config.toml.example
@@ -29,6 +29,16 @@ turn_merge_ms = 350          # Debounce window for merging finals into one turn.
 # rag_prefetch = false                    # Start retrieval during the merge window instead of after it
 # readback_bargein_guard_enabled = false  # Ignore weak barge-ins while the agent reads values back
 
+# Echo reference for barge-in, decided per session. On a path with no AEC
+# (SIP/PCMU telephony) the agent's own voice comes back loud enough to look
+# like an interruption; a browser cancels it in getUserMedia and needs no
+# bound. "auto" follows what each client declares on the WHIP URL (aec=none),
+# which sip-server already sends, so one server can host both.
+# echo_guard = "auto"           # auto | always | off. "always" is for raw-path clients that send no hint
+# echo_guard_gain = 0.6         # Echo cannot exceed this fraction of what produced it
+# echo_guard_margin = 1.8       # How far inbound must clear the echo bound to count as the caller
+# echo_guard_window_ms = 400    # How long sent audio stays in the reference window
+
 # Provider selection
 
 # Speech-to-speech mode. Setting a provider here replaces [stt], [llm], and

diff --git a/docs/capabilities.md b/docs/capabilities.md
@@ -80,6 +80,7 @@ StreamCore can run a complete speech-to-agent-to-speech pipeline, but that is on
 - Opus decode → PCM → pipeline → PCM → Opus encode → RTP
 - Energy-based VAD with configurable onset/offset frame counts, adapting to each call's noise floor so a quiet caller on a clean line and a caller beside a road both register
 - Barge-in on a faster VAD profile: agent audio ducks while the caller talks over it and recovers if the interruption turns out to be a backchannel
+- Echo reference for paths with no AEC, such as SIP/PCMU telephony: the barge-in threshold is bounded by the RMS the server just sent, so the agent stops interrupting itself on its own returning voice. Applied per session from a client hint, so one instance serves browsers and phone calls without either compromising the other
 - Turn debounce that merges consecutive final transcripts, so "I want to… um… book a table" is answered once, not twice
 - Sentence-boundary chunking so TTS starts before the LLM finishes, and chunk-level streaming so audio plays before a sentence is fully synthesized
 - Optional per-utterance delivery tags — the model may prefix a sentence with `[warm]`, `[empathetic]`, `[calm]`, or `[excited]`, which map to provider voice controls and are never spoken aloud

diff --git a/docs/capabilities.zh-CN.md b/docs/capabilities.zh-CN.md
@@ -80,6 +80,7 @@ StreamCore 可以跑通一条完整的「语音 → 智能体 → 语音」链
 - Opus 解码 → PCM → 处理流水线 → PCM → Opus 编码 → RTP
 - 基于能量的 VAD，起止帧数可配置，并自适应每通电话的噪声基线，因此安静线路上的轻声用户与路边嘈杂环境中的用户都能被正确识别
 - 使用更快 VAD 配置的 barge-in：用户抢话时智能体音量随即压低，若判定只是回应词则恢复
+- 面向没有 AEC 的链路（如 SIP/PCMU 电话）的回声参考：用服务器刚发出音频的 RMS 给打断阈值加下限，智能体不会再被自己绕回来的声音打断。该判定按会话依据客户端提示生效，因此同一个实例可以同时服务浏览器和电话，互不影响
 - 轮次去抖，把连续的 final 转写合并，让「我想…嗯…订个位」只被回答一次，而不是两次
 - 按句边界分块，使 TTS 在 LLM 生成结束前就开始；再按 chunk 级流式播放，使一句话尚未合成完就已出声
 - 可选的逐句表达标签 —— 模型可以在句首加上 `[warm]`、`[empathetic]`、`[calm]` 或 `[excited]`，它们会映射到服务商的音色控制参数，并且永远不会被读出来

diff --git a/docs/configuration.md b/docs/configuration.md
@@ -32,6 +32,10 @@ user_speech_quiet_ms = 600           # Quiet period after the caller stops befor
 turn_merge_ms = 350                  # Debounce window for merging finals into one turn
 # rag_prefetch = false               # Start retrieval during the merge window instead of after it
 # readback_bargein_guard_enabled = false  # Ignore weak barge-ins while the agent reads values back
+# echo_guard = "auto"                # auto | always | off. Auto follows each client's aec hint
+# echo_guard_gain = 0.6              # Echo cannot exceed this fraction of what produced it
+# echo_guard_margin = 1.8            # How far inbound must clear the echo bound to count as the caller
+# echo_guard_window_ms = 400         # How long sent audio stays in the reference window
 
 # Speech-to-speech. When set, replaces [stt], [llm], and [tts] entirely.
 [realtime]
@@ -160,6 +164,18 @@ Notes:
 - `pipeline.user_speech_quiet_ms` is how long the caller must be quiet before the agent starts speaking.
 - `pipeline.rag_prefetch` overlaps retrieval with the turn-merge window. Off by default; it issues a speculative embedding + search that is discarded if the turn text changes.
 - `pipeline.readback_bargein_guard_enabled` keeps weak corrections and backchannels from cutting off a confirmation readback. Only explicit commands (stop, cancel, hang up) interrupt. Off by default.
+- `pipeline.echo_guard` stops the agent barging in on its own voice. A browser runs AEC before audio reaches the server, so the VAD never sees the agent's output come back; over a carrier there is no AEC anywhere in the path, the returning audio is attenuated but structurally identical to speech, and an energy test cannot tell it from a caller. The guard keeps a rolling window of the RMS the server actually sent and requires inbound to clear `sent_rms x echo_guard_gain x echo_guard_margin` before it counts as an interruption. While the agent is silent that bound is zero and the ordinary adaptive threshold governs, so a quiet caller on a clean line is unaffected.
+
+  The decision is per session, because one instance usually serves browsers and SIP calls at once and the two need opposite answers. A client declares a raw path by adding `aec=none` to the WHIP URL, which `sip-server` sends on every call; browsers send nothing and are read as already cancelled.
+
+  | `echo_guard` | Effect |
+  | --- | --- |
+  | `"auto"` (default) | On for peers that sent `aec=none`, off for everyone else |
+  | `"always"` | On for every peer. For a raw-path client you cannot change to send the hint |
+  | `"off"` | Never on |
+
+  Leave it on `"auto"` unless you have a client on a path with no AEC that you cannot modify. Setting `"always"` on a server that also hosts browsers makes genuine browser barge-ins clear the agent's own output level first, which is the regression the per-session default exists to avoid.
+- `pipeline.echo_guard_gain`, `pipeline.echo_guard_margin`, and `pipeline.echo_guard_window_ms` tune that bound, for the sessions it applies to. The gain is how loud echo can be relative to the audio that produced it, the margin is what separates double-talk from echo, and the window should cover the round trip of the carrier's echo. The defaults (0.6, 1.8, 400ms) were measured on an 8kHz mu-law path; retune only against recordings of your own. The bound follows the barge-in duck on its own, since it is sampled from what goes on the wire after attenuation.
 - `deepgram.endpointing` and `deepgram.utterance_end_ms` tune when a turn is considered finished upstream; the turn-merge debounce runs on top of them.
 - `deepgram.tts_model` picks the Aura voice; STT (`model`) and TTS (`tts_model`) share the one API key. Voices are named `[family]-[voice]-[language]` — see [Deepgram's voice list](https://developers.deepgram.com/docs/tts-models).
 - `cartesia.max_concurrency` should match your plan's TTS concurrency limit — Cartesia counts active generations, not calls, and returns 429 past the limit.

diff --git a/docs/configuration.zh-CN.md b/docs/configuration.zh-CN.md
@@ -26,6 +26,10 @@ user_speech_quiet_ms = 600           # Quiet period after the caller stops befor
 turn_merge_ms = 350                  # Debounce window for merging finals into one turn
 # rag_prefetch = false               # Start retrieval during the merge window instead of after it
 # readback_bargein_guard_enabled = false  # Ignore weak barge-ins while the agent reads values back
+# echo_guard = "auto"                # auto | always | off. Auto follows each client's aec hint
+# echo_guard_gain = 0.6              # Echo cannot exceed this fraction of what produced it
+# echo_guard_margin = 1.8            # How far inbound must clear the echo bound to count as the caller
+# echo_guard_window_ms = 400         # How long sent audio stays in the reference window
 
 # Speech-to-speech. When set, replaces [stt], [llm], and [tts] entirely.
 [realtime]
@@ -152,6 +156,18 @@ voice = "en-Emma_woman"
 - `pipeline.user_speech_quiet_ms` 是用户需要安静多久，智能体才开始说话。
 - `pipeline.rag_prefetch` 让检索与轮次合并窗口重叠。默认关闭；它会发出一次推测性的 embedding + 检索，若该轮文本发生变化则丢弃。
 - `pipeline.readback_bargein_guard_enabled` 可避免弱纠正与回应词打断智能体的确认复述。只有明确的命令（stop、cancel、hang up）才会打断。默认关闭。
+- `pipeline.echo_guard` 防止智能体被自己的声音打断。浏览器会在音频到达服务器之前先做 AEC，因此 VAD 根本看不到智能体自己的输出绕回来；而在电话线路上整条链路没有任何 AEC，回声虽然衰减了，但结构上和语音完全一样，单靠能量判据无法与真人区分。该开关会维护一个滚动窗口，记录服务器实际发出音频的 RMS，只有当上行音频超过 `已发送 RMS x echo_guard_gain x echo_guard_margin` 时才算作打断。智能体沉默时该下限为零，判定重新交回自适应阈值，因此干净线路上说话轻的来电者不受影响。
+
+  这个判断是按会话而不是按服务器做的：同一个实例通常同时服务浏览器和 SIP 通话，而两者需要相反的答案。客户端通过在 WHIP URL 上加 `aec=none` 来声明自己处在没有 AEC 的链路上，`sip-server` 每通电话都会带上它；浏览器什么都不发，会被视为已经做过回声消除。
+
+  | `echo_guard` | 行为 |
+  | --- | --- |
+  | `"auto"`（默认） | 对发送了 `aec=none` 的会话开启，其余关闭 |
+  | `"always"` | 对所有会话开启。用于无法改动、又发不出该提示的裸链路客户端 |
+  | `"off"` | 始终关闭 |
+
+  除非你有一个无法修改、又跑在无 AEC 链路上的客户端，否则请保持 `"auto"`。在同时服务浏览器的服务器上设为 `"always"`，会让浏览器端真实的插话必须先盖过智能体自身的输出电平，而按会话判断的默认值正是为了避免这种回退。
+- `pipeline.echo_guard_gain`、`pipeline.echo_guard_margin` 和 `pipeline.echo_guard_window_ms` 用于调节这个下限（仅对开启了该判定的会话生效）。gain 是回声相对于产生它的音频最多能有多响，margin 用于区分双讲与回声，window 应覆盖线路回声的往返时间。默认值（0.6、1.8、400ms）是在 8kHz µ-law 链路上实测得到的；只有拿到你自己链路的录音再去重新调参。该下限会自动跟随打断时的音量压低，因为它采样的是经过衰减后真正发到线路上的信号。
 - `deepgram.endpointing` 与 `deepgram.utterance_end_ms` 调节上游认定一轮结束的时机；轮次合并去抖运行在它们之上。
 - `deepgram.tts_model` 选择 Aura 音色；STT（`model`）与 TTS（`tts_model`）共用同一个 API key。音色命名规则为 `[family]-[voice]-[language]` —— 见 [Deepgram 音色列表](https://developers.deepgram.com/docs/tts-models)。
 - `cartesia.max_concurrency` 应与你套餐的 TTS 并发上限一致 —— Cartesia 统计的是进行中的生成数而不是通话数，超限会返回 429。

diff --git a/docs/protocol.md b/docs/protocol.md
@@ -20,6 +20,16 @@ The client creates an SDP offer, gathers ICE candidates, and `POST`s it to `/whi
 
 This implementation aligns with the core WHIP flow: `POST` with `application/sdp`, `201 Created` with the answer, `Location` for the session URL, `ETag` for the ICE session, `PATCH` for ICE restart, `DELETE` for teardown, `OPTIONS` with `Accept-Post`, and full ICE gathering on both sides. Audio is `sendrecv`, with a DataChannel for bidirectional events.
 
+### Optional query parameters on `POST /whip`
+
+| Parameter | Values | Meaning |
+|---|---|---|
+| `resume` | a resume token | Reattach this offer to an existing conversation — see [Session resume](#session-resume) |
+| `direction` | `inbound`, `outbound` | Which way a telephony call was placed. Selects `pipeline.greeting_outgoing` for outbound calls |
+| `aec` | `none` | Nothing upstream of the server cancels echo on this path, so the agent's own voice comes back to it. Turns on the barge-in echo bound for this session under the default `pipeline.echo_guard = "auto"` |
+
+All are optional and unknown values are ignored. Omitting `aec` means echo cancellation ran upstream, which is true of every browser, so a client only sends it when running on a raw path — `sip-server` sends it on every call.
+
 ### ICE restart
 
 A transient network event — a phone moving between Wi-Fi and cellular, a laptop changing networks, a NAT rebinding after an idle gap — breaks connectivity without ending the call. Recovering by `POST`ing a fresh offer would allocate a new session, a new pipeline, and a new LLM client, so the conversation history and the rolling summary would be gone and the greeting would replay. `PATCH` recovers the *same* connection instead: new ICE credentials and candidates, but the same `PeerConnection`, the same DTLS association, the same tracks, and the same running pipeline.

diff --git a/docs/protocol.zh-CN.md b/docs/protocol.zh-CN.md
@@ -20,6 +20,16 @@
 
 本实现与 WHIP 的核心流程一致：以 `application/sdp` 发起 `POST`，用 `201 Created` 返回 answer，用 `Location` 给出会话 URL，用 `ETag` 标识 ICE 会话，用 `PATCH` 做 ICE 重启，用 `DELETE` 销毁，用 `OPTIONS` 返回 `Accept-Post`，并在双端做完整 ICE 收集。音频为 `sendrecv`，并带一个用于双向事件的 DataChannel。
 
+### `POST /whip` 的可选查询参数
+
+| 参数 | 取值 | 含义 |
+|---|---|---|
+| `resume` | 恢复令牌 | 把本次 offer 重新挂到既有会话上，见[会话恢复](#会话恢复session-resume) |
+| `direction` | `inbound`、`outbound` | 电话呼叫的方向。呼出通话会据此选用 `pipeline.greeting_outgoing` |
+| `aec` | `none` | 服务器上游没有任何环节做回声消除，智能体自己的声音会绕回来。在默认的 `pipeline.echo_guard = "auto"` 下，该会话会启用打断回声下限 |
+
+以上都是可选的，无法识别的取值会被忽略。不带 `aec` 表示上游已经做过回声消除，浏览器都属于这种情况，因此只有跑在裸链路上的客户端才需要发送它 —— `sip-server` 每通电话都会带上。
+
 ### ICE 重启
 
 短暂的网络事件 —— 手机在 Wi-Fi 与蜂窝之间切换、笔记本更换网络、空闲后 NAT 重新绑定 —— 会中断连通性，但通话本身并未结束。若用重新 `POST` offer 的方式恢复，会分配新的会话、新的流水线和新的 LLM 客户端，对话历史与滚动摘要随之丢失，开场白也会重播。`PATCH` 恢复的是*同一条*连接：ICE 凭据与候选是新的，但 `PeerConnection`、DTLS 关联、媒体轨道以及正在运行的流水线都保持不变。