Skip to content

feat(transcript): noise tagging + llm review-relevance grading#495

Open
plind-junior wants to merge 1 commit into
testfrom
feat/transcript-noise-grading
Open

feat(transcript): noise tagging + llm review-relevance grading#495
plind-junior wants to merge 1 commit into
testfrom
feat/transcript-noise-grading

Conversation

@plind-junior

Copy link
Copy Markdown
Member

makes the review console's session transcript skimmable — a long agent session
buries the few exchanges a reviewer needs to judge whether the session's
proposals are trustworthy. two presentation-only layers over the normalized
transcript; the raw session file stays the evidence of record.

what changed

  • deterministic noise tagging (transcript.py): classify_noise tags
    injected scaffolding (system reminders, hook payloads, command output, meta)
    by prefix, and a message whose blocks are all noise rolls up to a single
    collapsed stub. no llm.
  • llm review-relevance grading (transcript_grading.py, new): the
    deployment's compile.llm_cmd grades the surviving dialog messages key /
    low (ungraded = normal) so a reviewer skims to what matters. every reply is
    mechanically validated — out-of-range indices, unknown grades, non-dialog
    targets are dropped; an unusable reply degrades to an error note on the
    transcript, never an exception. results cache in state.db keyed by the raw
    file's content hash (index_db.get/put_transcript_grades).
  • surface: vouch session transcript --grade (--regrade bypasses the
    cache) and the grade/regrade params on kb.session_transcript across mcp
    • jsonl. the review console renders the noise stubs and key/low badges + notes
      (TranscriptView.tsx, MessageBlock.tsx, transcript.ts).
  • capture.load_config gains a VOUCH_CAPTURE_DISABLE guard so the grading llm
    subprocess does not capture itself back into the kb.

verification

full gate green: pytest (whole suite), mypy, ruff; webapp vitest (153 tests) and
tsc && vite build. new tests: test_transcript_grading.py,
test_session_transcript.py (+84), TranscriptView.test.tsx (+158). cli flags
confirmed wired. a live llm-grading drive needs a configured compile.llm_cmd
(not exercised here); the grading pipeline itself is covered by unit tests.

make the review console's session transcript skimmable. two layers over the
normalized transcript, both presentation-only — the raw session file stays the
evidence of record.

deterministic noise tagging (transcript.py): classify_noise tags injected
scaffolding (system reminders, hook payloads, command output, meta) by prefix,
and a message whose blocks are all noise rolls up to a single collapsed stub.
no llm.

llm review-relevance grading (transcript_grading.py): the deployment's
compile.llm_cmd grades the surviving dialog messages key/low (ungraded =
normal) so a reviewer skims to the exchanges that matter. every reply is
mechanically validated — out-of-range indices, unknown grades, and non-dialog
targets are dropped; an unusable reply degrades to an error note on the
transcript, never an exception. results cache in state.db keyed by the raw
file's content hash (index_db.get/put_transcript_grades).

surface: `vouch session transcript --grade` (--regrade bypasses the cache) and
the grade/regrade params on kb.session_transcript across mcp + jsonl. the review
console renders the noise stubs and key/low badges with notes. capture's
load_config gains a VOUCH_CAPTURE_DISABLE guard so the grading llm subprocess
does not capture itself back into the kb.
@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 40b979ee-a358-41cb-a949-b75776dac558

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/transcript-noise-grading

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added cli command line interface mcp mcp, jsonl, and http surfaces retrieval context, search, synthesis, and evaluation tests tests and fixtures size: XL 1000 or more changed non-doc lines labels Jul 15, 2026
@plind-junior plind-junior self-assigned this Jul 16, 2026
@plind-junior plind-junior added the auto-pr auto-pr orchestration label Jul 16, 2026
@plind-junior plind-junior enabled auto-merge July 16, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-pr auto-pr orchestration cli command line interface mcp mcp, jsonl, and http surfaces retrieval context, search, synthesis, and evaluation size: XL 1000 or more changed non-doc lines tests tests and fixtures

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant