Skip to content

fix(knowledge): fix seven video-digest watch-pipeline papercuts #6048

Description

@kyle-sexton

Problem

Small defects hit during one complete /knowledge:video-digest watch of
https://www.youtube.com/watch?v=MN9dGgmLyso (knowledge 0.17.0; the slice reached
status: complete). Each was re-checked against origin/main at 6f5909646 (knowledge 0.18.0).
Item P3 of the source item is left out (see Constraints); the remaining labels keep the source
numbering.

  • P1. rebuild-visual-frames.js crashes before any promotion. watch/rebuild-visual-frames.js:59-61
    calls fs.readdirSync on key-frames/frames with no existence check, so running it before
    vision-gated-promote.js throws an uncaught ENOENT: no such file or directory, scandir '...\key-frames\frames'. The pipeline doc lists Pass 2 detail reads of rows in
    key-frames/visual-frames.md before promotion, so that order is natural. The same file already
    treats a missing promotion-map.json as {} (:47).
  • P2. The checklist leaves {{FLOOR_SHEET_TRIAGE_PCT}} unrendered. templates/watch-checklist.md:65
    (item 4.9) uses {{FLOOR_SHEET_TRIAGE_PCT}}, but the replacements map in
    watch/init-watch-checklist.js:139-152 has no entry for it, so every rendered
    run-state/watch-checklist.md shows the raw token, including after --force once the vision plan
    sets the floors.
  • P4. No failure pattern for HTTP 403 from an aging yt-dlp. yt-dlp 2026.07.04 (above the
    2026.6 floor) failed the video download with
    ERROR: [download] Got error: HTTP Error 403: Forbidden. Giving up after 3 retries and
    fragment 1 not found, after its own warning that it was "older than 90 days". Upgrading to
    2026.08.19 fixed it. reference/sources/youtube.md:77-85 "Failure patterns" lists bot/sign-in,
    removed, not-a-video, unsupported host and 429/503/timeouts, not this.
  • P5. run-watch.js is silent for many minutes. Run in the background with stdout and stderr
    to a log, the log stayed empty through the whole download (about 6.5 min,
    videoPassMs: 397380) and the ASR transcription; output arrived only at exit. On main,
    watch/run-watch.js writes stderr only on errors and usage (:70-148); no stage emits progress.
  • P6. mark-phase <slice> vision records empty metrics, which makes vision-metrics-honesty
    vacuous.
    runMarkPhase (watch/watch-state.js:374-398) calls markPhaseComplete(state, phase)
    with no metrics, so after a full vision pass (52 cells triaged, 2 promoted) watch.json holds
    phases.vision: { completedAt, metrics: {} }. extraction/evals/check-watch-outcomes.js:599-608
    pushes the vision-metrics-honesty check only when metrics.contactSheetsTriaged is a number,
    so with empty metrics the check is silently skipped, not passed or failed.
  • P7. The action router has no row for a bare URL. SKILL.md:64-75 "Action router" has rows
    for (empty), queue, transcript <url>, watch, watch <n>, watch <url> and resume;
    /knowledge:video-digest <url> with no action word matches none. The run treated it as
    watch <url>.
  • P8. Temp sessions are never cleaned (low priority). Nothing on main removes a slice's
    tempSession directories after close; extraction/lib/temp-session-paths.js and
    watch/watch-state.js only normalize and serialize those paths. Windows does not clean %TEMP%
    on its own.

Evidence

Verified this pass by reading origin/main at 6f5909646: every path:line above, including
that the honesty check is skipped (not merely passed) when metrics are empty, and that no
rmSync/cleanup of temp-session dirs exists under extraction/watch/ or
extraction/lib/temp-session-paths.js.

Reported by the producing run and not re-run here: the P1 stack trace, the P4 403 errors and the
fix by upgrading yt-dlp, the P5 empty log and videoPassMs: 397380, the P6 vision-pass counts,
and the P8 measurement: on 2026-10-03 %TEMP% held 101 video-extraction-*, video-frames-* and
video-sheets-* directories; 98 of them (33 extraction dirs dated 2026-08-16, plus frames and
sheets dirs) total about 2 MB and look like test-run leftovers, while the current watch's three
directories are 754 MiB and are needed for resume.

Not a duplicate: #5803 (closed) fixed mark-phase synthesis ordering (only close writes
status: complete), not the empty vision metrics in P6. Searches for video-digest in:title,
youtube-digest, watch-checklist found nothing else covering these.

Proposed approach

  • P1: in rebuild-visual-frames.js, treat a missing frames/ directory as empty (write a
    visual-frames.md with no rows), matching the promotion-map.json handling at :47.
    Alternative: exit 1 with a one-line message naming the required order; rejected because the
    pipeline doc invites the early read.
  • P2: add {{FLOOR_SHEET_TRIAGE_PCT}} to the replacements map in init-watch-checklist.js,
    taken from the same floor computation buildFloorsLine uses, with a deferred placeholder text
    when vision-plan.md is absent (as FLOORS_LINE does). Add a test that a rendered checklist
    contains no {{ token.
  • P4: add a row to reference/sources/youtube.md "Failure patterns": "HTTP 403 on media
    fragments: update yt-dlp first". Optionally surface yt-dlp's own "older than 90 days" warning in
    the preflight.
  • P5: one progress line per stage on stderr from run-watch.js (acquire start/end, transcript
    start/end with strategy, watching start/end), using the existing writeStderr.
  • P6: have mark-phase <slice> vision compute contactSheetsTriaged, cells triaged and promoted
    count from the triage manifest and key-frames/promotion-map.json, and pass them to
    markPhaseComplete. Separately, consider making vision-metrics-honesty report a failing or
    warning row when the vision phase is marked but metrics are missing, so the check cannot be
    skipped silently.
  • P7: add one router row: a bare URL means watch <url>.
  • P8: have close (or a new clean action) remove the slice's tempSession directories after a
    successful close, and find which test created the 2026-08-16 video-extraction-* dirs so the
    suite's tmp-cleanup covers it.

Files: extraction/watch/rebuild-visual-frames.js, extraction/watch/init-watch-checklist.js,
templates/watch-checklist.md (only if the token name changes), reference/sources/youtube.md,
extraction/watch/run-watch.js, extraction/watch/watch-state.js,
extraction/evals/check-watch-outcomes.js, SKILL.md, and the matching *.test.js files, all
under plugins/knowledge/skills/video-digest/.

Acceptance criteria

  • rebuild-visual-frames.js <slice> on a slice with no key-frames/frames/ exits 0 without a
    stack trace (test).
  • A checklist rendered by init-watch-checklist.js, with and without vision-plan.md,
    contains no {{...}} token (test).
  • reference/sources/youtube.md "Failure patterns" has an HTTP 403 row that says to update
    yt-dlp.
  • run-watch.js writes at least one stderr line at the start of acquisition, transcription and
    watching (test with stubbed stages, or a captured-stderr assertion).
  • After mark-phase <slice> vision on a fixture slice with a triage manifest,
    watch.json phases.vision.metrics.contactSheetsTriaged is a number, and
    vision-metrics-honesty appears in the outcome report (test).
  • SKILL.md "Action router" has a row mapping a bare URL to watch <url>.
  • After a successful close, the slice's tempSession directories no longer exist (test on a
    temp fixture), or the decision to keep them is documented.
  • The video-digest extraction suite passes.

Constraints and gotchas

Context

Source: local handoff item 20261003-051227-knowledge-video-digest-pipeline-papercuts.md (retired
into this issue). Related: #5982 (open), draft PR #6004, #5803 (closed), and the sibling issue for
the caption-less ASR path from the same run
(#6047).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.priority: mediumReal value, no hard deadline; normal backlog flow.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions