videoeditor-record: web-based narration recorder (videoeditor record) - #9
Merged
Conversation
…rder New crate videoeditor-record and a `record` subcommand: `videoeditor record <episode>` serves a one-page teleprompter + mic recorder on 127.0.0.1 and writes kept takes straight into the episode, ready for the normal render/assemble pipeline. - Capture in the browser (getUserMedia/MediaRecorder): device picker, permission UX, and live level metering are already solved there — cross-platform native capture is not. localhost is a secure context, so the mic works without TLS. Raw voice: echoCancellation / noiseSuppression / autoGainControl all disabled. - UI: clip sidebar with per-take fit status, big teleprompter text, 3-2-1 countdown, elapsed-vs-window timer that flags overruns live, level meter (WebAudio), review player, keyboard-driven flow (space/enter/r/arrows). - Server (tiny_http, sync — no async runtime added): uploads (webm/opus or mp4/aac) transcode via ffmpeg to the exact mp3 format the TTS path produces; every kept take lands in audio/takes/<id>/ and the replaced clip is archived, so no take is ever lost; audio/clips.json is rebuilt from disk (self-healing) and each save returns the episode fit-check. - Clip ids off the URL are allowlist-sanitized (no path traversal). - guide.md pipeline/director-loop + CLAUDE.md layout updated. Live-tested end to end against a scratch episode: webm upload → transcode → archive (take_001/take_002/replaced_003) → manifest → fit-check response; traversal attempts rejected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… before you keep a take
POST /api/review/<id> analyzes a pending take without keeping it, and the
UI shows a COACH panel while you listen back:
- always (no API needed): duration vs scene window, mean/max dBFS via
ffmpeg volumedetect, clipping and too-quiet flags
- with ELEVENLABS_API_KEY: Scribe transcription → script-vs-spoken word
diff (LCS; dropped words + ad-libs), words/sec pacing, dead-air gaps
from word timestamps, tagged background audio events
- coaching lines synthesize it ("dropped from the script: boomer",
"dead air at 3.0s", "clipping — back off the mic", or "clean take —
ship it"); degrades gracefully to local-only metrics without a key
Tests: LCS diff (missing/added words), clean-take praise, multi-fault
coaching (overrun + clipping + drops + racing + dead air + cough).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- capture and transcode stereo (interface sends a stereo mix; assembly normalizes all narration to stereo anyway), 192k mp3 archive quality - L/R channel meters in the UI instead of a single downmixed bar - every take is archived to audio/takes/<id>/take_NNN.mp3 the moment it is reviewed — retakes lose nothing; keep promotes the archived file via /api/keep/<id>/<n> without re-uploading Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- GET /api/takes/<id> lists every archived take (take_* and replaced_*) with duration and approval state; GET /audio/takes/<id>/<file> serves them for audition - POST /api/approve/<id>/<file> promotes an archived take to the clip and records the pick in audio/takes/approvals.json (in-episode, so approvals survive restarts and travel with the repo) - keep after recording is now just an approval of the freshly archived take; A/B switching approved takes doesn't pile up duplicate archives - UI: takes panel under the prompter with ▶ audition and approve buttons, approved take highlighted Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- review saves take_NNN.json next to take_NNN.mp3 — the archive holds the analysis, not just the audio - /api/takes/<id> returns each take's saved review; the takes panel shows comparison stats per row (script %, pace, peak, clipped, long, pauses) with full coaching notes on hover - takes panel refreshes as soon as a new take is analyzed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- three-column layout: clips | takes rail | stage — takes and coach move out of the teleprompter area for good - state-aware stage via body class: recording dims the rails so only the script competes for attention; review shrinks (never hides) the script and shows one review card: coach stats, notes, transcript, player, keep/retake - takes rail newest-first (list_takes sorts descending) - uber index.html split into web/: index.html + style.css + 6 plain ES modules (state/mic/record/review/takes/main) talking over an event bus — no framework, no build step, still embedded via include_str! Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tap a take row to unfold its saved coach report — coaching notes + transcript — right in the rail (chevron rotates, height animates via the grid 0fr→1fr trick, one row open at a time). Replaces the hidden hover tooltip. Play/approve stop propagation so they don't toggle. Expansion is keyed clip/file so re-renders keep it open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The recorder UI is now Rust end to end: crates/videoeditor-record-ui is a Leptos 0.8 (CSR) app compiled to wasm with trunk. videoeditor-record embeds the COMMITTED dist (ui/, regenerate with `just ui`) so the CLI stays one self-contained binary and both crates publish to crates.io with no wasm toolchain required by consumers. - same REST API, same stylesheet contract, same studio layout: clips | takes rail (newest first, inline disclosure of the saved coach report) | teleprompter stage that shrinks-but-never-hides the script during review; body-class state machine unchanged - audio interop (getUserMedia, stereo ChannelSplitter analysers, MediaRecorder) via web-sys; wasm-bindgen pinned =0.2.121 to match nixpkgs' wasm-bindgen-cli (bump together) - nix devshell now carries the whole toolchain: rust w/ wasm32 target (rust-overlay), trunk, wasm-bindgen-cli, node, playwright + pinned browsers (PLAYWRIGHT_BROWSERS_PATH); nix build needs no wasm because the dist is committed - e2e/: 9-test playwright journey against the real binary + real Chromium with a fake mic — boot, record→coach→keep, newest-first, disclosure, approve-from-rail, audition, retake, clip nav, and the script-never-hidden guarantee; `just e2e`, wired into CI (ubuntu) - vanilla JS front end deleted Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…st, drift guard - videoeditor-record-ui is publish = false (+ release-plz release = false): it doesn't stand alone; its trunk build ships inside videoeditor-record's committed ui/ dist, which publishes to crates.io as part of that crate - no manual steps: `just e2e` depends on `just ui` (dist always rebuilt from source before the suite), and `just check` — which CI runs on every push, no wasm toolchain required — verifies ui/.source-hash against the UI source, so a stale committed dist cannot merge - dist regenerated with the nix toolchain so bytes stay stable across machines; docs updated to match Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
videoeditor record <episode>spawns a local web recorder so the director can perform the narration instead of (or alongside) TTS — the "videocall-style" recorder: teleprompter, live level meter, per-clip takes.Design
Browser capture, Rust server. Native cross-platform mic capture (device pickers, permission prompts, monitoring) is a swamp;
getUserMedia/MediaRecordersolve all of it, and localhost is a secure context so the mic works without TLS. Leptos/WASM was considered for the page and rejected: it would add a wasm toolchain to a crates.io-published workspace for a single page, when the repo's house style is already hand-authored HTML+JS (all scene templates). The page is one embeddedindex.htmlserved bytiny_http(sync — no async runtime enters the workspace).Flow
videoeditor record <dir>→ binds 127.0.0.1:4747, opens the browser.[CLIP:]with take status + duration vs its scene window; teleprompter shows the narration text big.space→ 3-2-1 countdown → record with live level meter and an elapsed-vs-window timer that flags overruns while you speak;spaceto stop; review player;enterkeeps,rretakes.audio/takes/<id>/(replaced clips too — no take is ever lost), promoted toaudio/clips/<id>.mp3, manifest rebuilt from disk, fit-check returned and shown in the UI.Raw voice: echoCancellation/noiseSuppression/autoGainControl disabled (call-style processing ruins VO takes); the meter is there to set levels.
Testing
just checkclean; unit tests for URL-id sanitization (path traversal) and take numbering.GET /api/episode, webm/opus upload → transcode →take_001/take_002/replaced_003archive chain →clips.jsonrebuild → fit-check response verified; traversal attempt rejected.🤖 Generated with Claude Code