Fix cut boundaries: timeline drift, audio-guided edges, phrase cutter

.agents/skills/cut-mistakes/SKILL.md

@@ -53,6 +53,14 @@ node .agents/skills/cut-mistakes/scripts/apply-cuts.mjs \
 
 Writes the EDL, a re-timed `<stem>.mistakes-transcript.json` (feeds the motion-graphics agent + `validate-beat-sync.mjs`), a decisions log, and — with `--apply` — the cut video (ffmpeg trim+concat, A/V in sync). Drop `--apply` for a dry run.
 
+With `--video`, every cut edge is **audio-snapped** before frame snapping (see below); pass `--no-snap-audio` to cut at the exact approved times instead.
+
+## Audio-guided edges (why cuts at word times clip the next word)
+
+ASR word timestamps are 50–150ms coarse, and a speaker usually starts the next word inside that window. A cut placed at the approved `end` (which is normally a word's `end`) therefore lands on the first syllable of the following word, and a cut at a word's `start` can leave the tail of the previous one — heard as "the transition is halfway through a word". `apply-cuts.mjs` now moves each delete edge to the quietest 30ms nearby (searching up to 150ms outward, 30ms inward, on a 16 kHz mono extract of the source), and never past the neighbouring kept words. The EDL records both `requested_*` and snapped times. The helper lives in `scripts/lib/audio-edges.mjs` and is shared with `scripts/cut-phrases.mjs`.
+
+Measured on a 41-cut set: last-80ms energy before each cut went from speech level (≈-30 dB) to -31…-78 dB; the few that stay near -31 dB are true no-gap boundaries where the cut sits inside the last word's own decay rather than on the next word's onset.
+
 ## Reviewing the result
 
 Use the shared review tool to eyeball the cuts and spot-check boundaries:
@@ -65,8 +73,32 @@ node scripts/build-edl-review.mjs <stem>.mistakes-edl.json \
 npx serve . -p 8080 -n
 ```
 
+## Retimed-transcript precision (read before reusing timestamps downstream)
+
+`<stem>.mistakes-transcript.json` only lines up with the rendered cut if
+every cut lands on a real frame. `apply-cuts.mjs` now snaps each cut edge
+to the nearest source frame (via `ffprobe`'s `r_frame_rate`, only when
+`--video` is given) *and* writes the filtergraph times at microsecond
+precision. The second half matters: at millisecond precision a 30fps
+boundary (33.333…ms) rounds past the frame's real timestamp, ffmpeg drops
+the boundary frame, video runs a frame short of its sample-exact audio, and
+`concat` shifts every later segment a few ms in the same direction. Snapping
+alone, with ms-precision output, measured **no improvement** over the
+original (0.31s vs 0.27s drift at the end of an identical 45-cut clip);
+snap + µs precision measured 0.01s. See the same section in the
+`cut-silences` skill for the full comparison.
+
+Because this stage runs on `cut-silences`' output, its input transcript
+carries whatever drift that stage left behind — both scripts must be on the
+fixed version for the final transcript to be trustworthy. And still
+re-transcribe the final render and diff it against the intended text before
+calling any downstream cut done: transcription timing has ~10–100ms jitter,
+variable-frame-rate sources are not handled, and a locked-off single-camera
+shot looks identical a second early in a frame strip.
+
 ## Notes
 
 - A very clean delivery may yield few or zero real cuts — that's a valid outcome; don't cut natural speech to hit a quota.
-- Cuts between two spoken words are hard joins. They're usually clean for stutters/false starts; for tighter audio a 20-30ms fade can be added later.
+- Cuts between two spoken words are hard joins. With audio snapping on they land in the gap, not on a word; for tighter audio a 20-30ms fade can be added later.
+- Cutting *pieces* out of a long recording (short-form ads, cutdowns)? Use `scripts/cut-phrases.mjs` instead of hand-picking timestamps: name the first and last words of each phrase, transcribe short windows of the source itself for ground truth, and let it snap, cut, assemble, and re-transcribe-and-diff.
 - Hand the `mistakes-transcript.json` + the clean video to **Agent 3 (motion graphics / tiered cards)**.

.agents/skills/cut-mistakes/scripts/apply-cuts.mjs

@@ -27,17 +27,18 @@ function die(m, c = 1) { console.error(m); exit(c); }
 
 const args = argv.slice(2);
 if (args.length === 0 || args.includes("--help")) {
-  console.log("Usage: node apply-cuts.mjs <transcript.json> --cuts <approved.json> [--video in.mp4] [--output out.mp4] [--out-dir dir] [--apply]");
+  console.log("Usage: node apply-cuts.mjs <transcript.json> --cuts <approved.json> [--video in.mp4] [--output out.mp4] [--out-dir dir] [--apply] [--no-snap-audio]");
   exit(args.length === 0 ? 1 : 0);
 }
-const opts = { transcript: null, cuts: null, video: null, output: null, outDir: null, apply: false };
+const opts = { transcript: null, cuts: null, video: null, output: null, outDir: null, apply: false, snapAudio: true };
 for (let i = 0; i < args.length; i++) {
   const a = args[i];
   if (a === "--cuts") opts.cuts = args[++i];
   else if (a === "--video") opts.video = args[++i];
   else if (a === "--output" || a === "-o") opts.output = args[++i];
   else if (a === "--out-dir") opts.outDir = args[++i];
   else if (a === "--apply") opts.apply = true;
+  else if (a === "--no-snap-audio") opts.snapAudio = false;
   else if (a.startsWith("--")) die(`unknown option: ${a}`);
   else if (!opts.transcript) opts.transcript = a;
 }
@@ -69,6 +70,31 @@ if (!Number.isFinite(duration)) {
 if (!Number.isFinite(duration) || duration <= 0) die('Invalid source duration');
 if (approved.some(c => c.start < 0 || c.end > duration)) die('Cut range is outside source duration bounds');
 
+// ---- frame rate (for boundary snapping, video renders only) ----
+// ffmpeg's trim/atrim filters cut on real source frame boundaries, not exact
+// float seconds. If the retimed transcript is computed from un-snapped
+// floats, each cut's actual rendered position silently differs from the math
+// by a fraction of a frame, and that error compounds linearly with cut count.
+// Snapping every delete-range edge to the nearest real frame time BEFORE
+// computing anything downstream keeps the retimed transcript closer to what
+// ffmpeg actually renders. Only meaningful (and only applied) when --video is
+// given -- with no video there is no rendered file to drift from, and
+// snapping would just add pointless rounding to an otherwise-exact edit.
+let fps = null;
+if (opts.video && existsSync(resolve(opts.video))) {
+  fps = 30;
+  const probe = spawnSync("ffprobe", [
+    "-v", "error", "-select_streams", "v:0", "-show_entries", "stream=r_frame_rate",
+    "-of", "default=noprint_wrappers=1:nokey=1", resolve(opts.video),
+  ]);
+  const raw = probe.stdout?.toString().trim();
+  if (raw && raw.includes("/")) {
+    const [n, d] = raw.split("/").map(Number);
+    if (Number.isFinite(n) && Number.isFinite(d) && d > 0) fps = n / d;
+  } else if (Number.isFinite(Number(raw))) fps = Number(raw);
+}
+const snap = (t) => (fps ? Math.round(t * fps) / fps : t);
+
 function mergeRanges(ranges) {
   const sorted = ranges.map((r) => ({ start: Number(r.start), end: Number(r.end), reasons: r.reason ? [r.reason] : [] }))
     .sort((a, b) => a.start - b.start);
@@ -80,7 +106,26 @@ function mergeRanges(ranges) {
   }
   return merged;
 }
-const mergedDeletes = mergeRanges(approved);
+// ---- audio-guided edges (video renders only, default on) ----
+// A cut placed exactly at an ASR word edge usually clips the onset of the
+// next word or leaves the tail of the last one, because word timestamps are
+// 50-150ms coarse. Before frame snapping, move each delete edge to the
+// quietest 30ms nearby, never past the neighbouring kept words.
+let audioSnapped = false;
+let rangesForRender = mergeRanges(approved);
+if (fps && opts.snapAudio) {
+  const { extractWav, loadWav, snapRange } = await import(new URL("../../../../scripts/lib/audio-edges.mjs", import.meta.url));
+  const outDirEarly = opts.outDir ? resolve(opts.outDir) : dirname(transcriptPath);
+  mkdirSync(outDirEarly, { recursive: true });
+  const wav = loadWav(extractWav(resolve(opts.video), join(outDirEarly, `${basename(resolve(opts.video), extname(opts.video))}.16k.wav`)));
+  rangesForRender = rangesForRender.map((r) => {
+    const prevKept = words.filter((w) => w.end <= r.start).at(-1);
+    const nextKept = words.find((w) => w.start >= r.end);
+    return snapRange(wav, r, { prevEnd: prevKept ? prevKept.end : 0, nextStart: nextKept ? nextKept.start : duration });
+  });
+  audioSnapped = true;
+}
+const mergedDeletes = rangesForRender.map((r) => ({ ...r, start: snap(r.start), end: snap(r.end) })).filter((r) => r.end > r.start);
 
 const keepRanges = [];
 let cursor = 0;
@@ -121,8 +166,8 @@ function buildFilterScript(ranges) {
   const lines = [];
   for (let i = 0; i < ranges.length; i++) {
     const r = ranges[i];
-    lines.push(`[0:v]trim=start=${r.start.toFixed(3)}:end=${r.end.toFixed(3)},setpts=PTS-STARTPTS[v${i}];`);
-    lines.push(`[0:a]atrim=start=${r.start.toFixed(3)}:end=${r.end.toFixed(3)},asetpts=PTS-STARTPTS[a${i}];`);
+    lines.push(`[0:v]trim=start=${r.start.toFixed(6)}:end=${r.end.toFixed(6)},setpts=PTS-STARTPTS[v${i}];`);
+    lines.push(`[0:a]atrim=start=${r.start.toFixed(6)}:end=${r.end.toFixed(6)},asetpts=PTS-STARTPTS[a${i}];`);
   }
   lines.push(`${ranges.map((_, i) => `[v${i}][a${i}]`).join("")}concat=n=${ranges.length}:v=1:a=1[v][a]`);
   return lines.join("\n");
@@ -149,7 +194,7 @@ writeFileSync(join(outDir, `${stem}.mistakes-decisions.md`), [
 ].join("\n"));
 
 const summary = {
-  inputDuration: Number(duration.toFixed(3)), editedDuration,
+  inputDuration: Number(duration.toFixed(3)), fps, audioSnapped, editedDuration,
   removed: Number(removedTotal.toFixed(3)), removedPct: Number(((removedTotal / duration) * 100).toFixed(1)),
   cuts: mergedDeletes.length, editedWords: editedWords.length, outDir,
 };

.agents/skills/cut-silences/SKILL.md

@@ -80,4 +80,30 @@ The JSON summary printed to stdout includes `removed`, `removedPct`, range count
 
 ## Hand-off to the next agent
 
-Pass `<stem>.silence-transcript.json` (and the `silenced.mp4` if rendered) to the **cut-mistakes** agent. Because timestamps are already on the edited timeline, downstream beat timing and `scripts/validate-beat-sync.mjs` work without further adjustment.
+Pass `<stem>.silence-transcript.json` (and the `silenced.mp4` if rendered) to the **cut-mistakes** agent.
+
+## Retimed-transcript precision
+
+`<stem>.silence-transcript.json` only lines up with `silenced.mp4` if every
+cut lands on a real frame. Two things have to hold, and originally neither
+did:
+
+1. Delete-range edges are snapped to the nearest source frame (via
+   `ffprobe`'s `r_frame_rate`, only when `--video` is given) before any
+   retiming math.
+2. The filtergraph writes those times at microsecond precision. At
+   millisecond precision a 30fps boundary (33.333…ms) rounds past the frame's
+   real timestamp, ffmpeg drops the boundary frame, video runs one frame
+   short of its sample-exact audio, and `concat` shifts every later segment
+   by a few ms — always in the same direction. Across a few hundred cuts that
+   compounded to 1–3s.
+
+Measured on one identical 45-cut clip: pre-fix drift 0.27s at the end of the
+file; snap-only (ms precision) 0.31s — no improvement; snap + µs precision
+0.01s, with the rendered duration matching the transcript's claim exactly.
+
+Still re-transcribe the final render and diff it against the intended text
+before calling any downstream cut done. Transcription timing itself has
+~10–100ms of jitter, variable-frame-rate sources are not handled, and a
+locked-off single-camera shot looks identical a second early in a frame
+strip — that check is the only thing that catches a uniform time offset.

.agents/skills/cut-silences/scripts/cut-silences.mjs

@@ -103,6 +103,32 @@ if (!Number.isFinite(duration)) {
 }
 if (!Number.isFinite(duration)) duration = words.at(-1).end + opts.tailPad;
 
+// ---- frame rate (for boundary snapping, video renders only) ----
+// ffmpeg's trim/atrim filters cut on real source frame boundaries, not exact
+// float seconds. If the retimed transcript is computed from un-snapped floats,
+// each cut's actual rendered position silently differs from the math by a
+// fraction of a frame, and that error compounds linearly with cut count (a
+// typical silence pass makes 100-300+ cuts). Snapping every delete-range edge
+// to the nearest real frame time BEFORE computing anything downstream keeps
+// the retimed transcript closer to what ffmpeg actually renders. Only
+// meaningful (and only applied) when --video is given -- with no video there
+// is no rendered file to drift from, and snapping would just add pointless
+// rounding to an otherwise-exact edit.
+let fps = null;
+if (opts.video && existsSync(resolve(opts.video))) {
+  fps = 30;
+  const probe = spawnSync("ffprobe", [
+    "-v", "error", "-select_streams", "v:0", "-show_entries", "stream=r_frame_rate",
+    "-of", "default=noprint_wrappers=1:nokey=1", resolve(opts.video),
+  ]);
+  const raw = probe.stdout?.toString().trim();
+  if (raw && raw.includes("/")) {
+    const [n, d] = raw.split("/").map(Number);
+    if (Number.isFinite(n) && Number.isFinite(d) && d > 0) fps = n / d;
+  } else if (Number.isFinite(Number(raw))) fps = Number(raw);
+}
+const snap = (t) => (fps ? Math.round(t * fps) / fps : t);
+
 // ---- build the delete ranges (silence only) ----
 function mergeRanges(ranges) {
   const sorted = ranges
@@ -147,7 +173,7 @@ for (let i = 1; i < words.length; i++) {
   }
 }
 
-const mergedDeletes = mergeRanges(deleteRanges);
+const mergedDeletes = mergeRanges(deleteRanges).map((r) => ({ ...r, start: snap(r.start), end: snap(r.end) })).filter((r) => r.end > r.start);
 
 // ---- keep ranges = the complement of the deletes ----
 const keepRanges = [];
@@ -193,8 +219,8 @@ function buildFilterScript(ranges) {
   const lines = [];
   for (let i = 0; i < ranges.length; i++) {
     const r = ranges[i];
-    lines.push(`[0:v]trim=start=${r.start.toFixed(3)}:end=${r.end.toFixed(3)},setpts=PTS-STARTPTS[v${i}];`);
-    lines.push(`[0:a]atrim=start=${r.start.toFixed(3)}:end=${r.end.toFixed(3)},asetpts=PTS-STARTPTS[a${i}];`);
+    lines.push(`[0:v]trim=start=${r.start.toFixed(6)}:end=${r.end.toFixed(6)},setpts=PTS-STARTPTS[v${i}];`);
+    lines.push(`[0:a]atrim=start=${r.start.toFixed(6)}:end=${r.end.toFixed(6)},asetpts=PTS-STARTPTS[a${i}];`);
   }
   const inputs = ranges.map((_, i) => `[v${i}][a${i}]`).join("");
   lines.push(`${inputs}concat=n=${ranges.length}:v=1:a=1[v][a]`);
@@ -255,6 +281,7 @@ writeFileSync(decPath, decisions);
 
 const summary = {
   sourceDuration: Number(duration.toFixed(3)),
+  fps,
   editedDuration,
   removed: Number(removedTotal.toFixed(3)),
   removedPct: Number(((removedTotal / duration) * 100).toFixed(1)),

.agents/skills/short-form-edit/SKILL.md

@@ -50,6 +50,14 @@ this test. Treat the answer as editorial judgment until audience data exists.
    Render the clean cut once and derive word timing from the actual frame-aligned
    EDL. Lock the cut after the rough animatic, before polished graphics. Every later timing change rebuilds dependent
    captions, graphics, and sound from the same map.
+   When pulling phrases out of a long recording, use `scripts/cut-phrases.mjs`:
+   name each phrase's first and last words, transcribe short windows of the
+   source file itself for ground truth (never a retimed transcript), and let it
+   snap each cut to the local audio minimum. Cutting at ASR word times plus a
+   fixed pad lands on the next word's first syllable. Cut pieces with a keyframe
+   every second (`-g 30`) or the renderer warns about seek failures. The script
+   re-transcribes the assembly and diffs it against the intended text; do the
+   same on the final render.
 
 User authorization persists. An instruction to make a finished video and iterate
 authorizes ordinary local editing and rendering. An explicit request for Kie
@@ -274,7 +282,9 @@ human reactions where they support the story.
    sounds. Make meaningful contacts perceptible, vary texture and density, and
    retain quieter conviction beats. Do not substitute louder music or a whoosh
    on every cut. Inspect the actual animation landing before setting each cue.
-6. Run lint and inspect the live preview before rendering. Browser automation is
+6. Run lint and `hyperframes validate` before every render, and render with
+   `--strict` so a lint error (a missing font, logo, or GSAP file) blocks the
+   render instead of producing a silently broken video. Browser automation is
    headless with Pointer Lock and pointer capture disabled in every context.
    Local automated review is permitted when the user authorized autonomous
    iteration. Render a draft, inspect it, revise, then render final quality.