Skip to content

Polish Bestie voice controls and optional transcripts - #82

Open
mahanti wants to merge 4 commits into
feat/bestie-realtime-voicefrom
am-voice-polish
Open

mahanti wants to merge 4 commits into
feat/bestie-realtime-voicefrom
am-voice-polish

Conversation

@mahanti

@mahanti mahanti commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Bestie’s call panel now centers the companion and keeps call controls in a stable footer. Thinking and tool approval move into the header’s ellipsis menu; transcripts are hidden by default and can be toggled during a call.

  • Reuse Messages rows for transcripts, with an even avatar/transcript split, independent scrolling, and a soft blur/fade at the top.
  • Use real microphone/playback analysers for a monochrome mirrored waveform and restrained speech gestures, including reduced-motion behavior and lifecycle cleanup.
  • Match header buttons through the design system and preserve keyboard focus through call startup/cancellation.
  • Keep approval prompts visible outside the transcript scroller. No provider, credential, tool-execution, or protocol changes.

Stacked on #37 and targeting feat/bestie-realtime-voice, so this PR contains only the polish changes.

Screenshots

Real UI with synthetic audio, isolated identities, and fixture text. All nine states, including connecting, speaking, muted, dark mode, and the compact layout.

Idle In call Transcript
Idle In call Transcript

Validation

  • All hosted checks passed at e3ecb47: CI run, including all four browser shards, measurements, JavaScript, Rust/tool integration, and CI required. DCO and both security checks also passed.

  • All 1,119 unit tests and TypeScript passed in pre-push at e3ecb47.

  • All 18 Bestie browser cases and all 30 autocomplete cases passed across Chromium and WebKit. All 16 layout/completion-layout cases and 14 design-system browser cases also passed.

  • Screenshot review found and fixed avatar overlap in short panels; regression coverage checks that the image stays above the transcript.

  • Local batch checks passed: 35 integration tests, frontend builds, lint/design checks, Rust formatting, and workspace Clippy. The broad browser run had one stale companion-copy assertion, repaired and verified by rerunning the complete affected files.

  • Fixed two test-ordering races found during validation: establish each asynchronous signer before releasing its gate, and wait for autocomplete retry options to render before keyboard navigation. No runtime changes, retries, skipped tests, or timeout increases.

  • Local native limitation: 8 of 9 native tests passed on this Mac. The real-login-shell probe is blocked by a managed security acknowledgement prompt. No shell policy or test timeout was changed. Hosted Rust and tool integration validates the native tests on Linux.

Model/hardware feel remains a manual check; fixture screenshots are not inference benchmarks.

mahanti and others added 3 commits September 16, 2026 14:16
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Arjun Mahanti <arjun@squareup.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Arjun Mahanti <arjun@squareup.com>
Wait for each signing gate so real crypto completion cannot reorder the requests under test. Assert the waiting request is rejected instead of dispatched after cooldown.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Arjun Mahanti <arjun@squareup.com>
Assert the fixture publication is accepted and the options are rendered before Arrow Up. The CI trace showed the key moving the textarea caret while React still had a pending completion render.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Arjun Mahanti <arjun@squareup.com>
@mahanti
mahanti marked this pull request as ready for review September 16, 2026 19:07
@mahanti
mahanti requested a review from wesbillman as a code owner September 16, 2026 19:07

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Changes requested

Reviewed e3ecb47706be008960a9a904868dfe77a177dd42 against stacked base a3eda453e713bd2ef0dbf79b8c5fcb1d6310545c (feat/bestie-realtime-voice, not main). Two introduced accessibility regressions need correction:

  1. [P2] Preserve focused End when a call terminates asynchronously. Bestie.tsx:38–48, with the outgoing branch at line 113. Keyboard startup focuses End and consumes restoreFocus; only explicit footer activation sets it again. If the microphone then disconnects while End remains focused, voice.mjs:482–489 calls fail(), and call.ts:302–306 publishes error. That makes busy false and End inert/aria-hidden, without moving focus to the enabled Call replacement. The base retained one start/end button, so its focus survived this transition. Restore focus when the outgoing End actually owns it, without stealing focus from elsewhere. Add a regression that starts by keyboard, leaves End focused, triggers microphone ended, and expects Call to receive focus. This is a source trace, not a claim about a measured browser-specific fallback focus destination.

  2. [P2] Include the visible “Call” text in the button’s accessible name. Bestie.tsx:156–169. The new button visibly says Call, but aria-label="Start Bestie voice conversation" overrides that text and contains no “Call”. The shared Button forwards this label unchanged. This violates Label in Name and prevents speech-input targeting from relying on the displayed wording. Use a name such as Call Bestie, updating the focus selector at line 45 and exact-name test expectations. The base control was icon-only, so this mismatch is introduced here.

Scope and validation

Audio/analyzer ownership and cleanup, call lifecycle, controls/menus, transcript rendering, approval placement, panel integration, and changed test contracts were reviewed on pinned objects. The independent compact-approval concern did not establish keyboard inaccessibility from source; keep short-panel keyboard approval scrolling as an acceptance check, not a proven security defect.

All 11 existing exact-head hosted checks succeeded; GitHub reports clean mergeability against the stacked base. Source-only review on Blox: no checkout, builds, tests, PR-code execution, or CI reruns. Physical-device/model feel and native acceptance were not exercised. This is not approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants