fix(extract): stop fragment claims and stopword-matched injection

[plind-junior]

the receipt pipeline's segmenter split candidate spans on every period, so dotted tokens — "1.2.0", "cli.py:2550", markdown links — sheared into mid-token shards, and the auto-approve gate happily durably approved them because "verbatim in source" was the only bar. real KBs built by ingest_source ended up full of claims like "0`, and let release." — receipt-verified garbage.

this changes the boundary rule to punctuation-followed-by-whitespace (a bare dot inside a token is never a sentence end) and adds a claim-worthiness gate: spans with dangling backticks, unbalanced brackets, punctuation-led starts, or fewer than three words are dropped rather than filed. the gate lives in segment_source, so every caller inherits it, and the verbatim-substring invariant that receipts depend on is untouched — spans are still contiguous byte runs of the source.

on the recall side, the prompt hook passed the raw prompt to retrieval, and since fts ORs every token, a conversational prompt like "which one is better?" matched claims on "one" and injected noise on every turn. the hook now searches only on informative tokens and injects nothing at all when a prompt has none — zero tokens of context for prompts that can't be answered from the KB anyway.

context packs also now carry the full approved claim text for claim hits instead of the fts5 «»-highlighted 16-token snippet window. the elided snippet is a search-ui affordance; in model context it read as mangled fragments. claims are length-capped and the existing max_chars budget still applies, so pack size stays bounded.

test-first throughout: six new tests reproduce the exact observed failures (dotted-token shearing, unbalanced-markup fragments, stopword-only injection, snippet windows in context) and pin the fixed behaviour. one existing test was adjusted to keep exercising the exception-swallow path now that stopword-only prompts short-circuit before retrieval. full gate is green locally: pytest (minus embeddings), mypy src, ruff check src tests.

Summary by CodeRabbit

• New Features

  • Improved retrieval by focusing searches on meaningful prompt terms.
  • Context now includes complete claim text instead of shortened search snippets.

• Bug Fixes

  • Improved sentence extraction for versions, file paths, and links.
  • Filters out incomplete, malformed, or overly short claim fragments.
  • Prevents irrelevant context from being injected for prompts without meaningful terms.

[coderabbitai]

Walkthrough

Claim segmentation now preserves dotted tokens and filters malformed fragments. Prompt retrieval removes stopwords and skips empty queries. Claim context rendering now exposes full claim text instead of FTS5 highlight snippets.

Changes

Context quality improvements

Layer / File(s)	Summary
Sentence-boundary claim segmentation src/vouch/extract.py, tests/test_extract.py	Sentence spans are split at whitespace-delimited punctuation boundaries, while dotted tokens, malformed markup, punctuation-led fragments, and single-word spans are filtered.
Full claim context summaries src/vouch/context.py	Claim context items now use complete claim.text values while preserving retraction filtering and evidence extraction.
Informative prompt retrieval src/vouch/hooks.py, tests/test_hooks.py	Prompt queries remove stopwords, one-character tokens, digits, and duplicates; prompts without informative tokens inject no context, with tests covering query filtering and full claim rendering.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ClaudePromptHook
  participant InformativeTokens
  participant BuildContextPack
  participant ContextRenderer
  ClaudePromptHook->>InformativeTokens: Filter prompt tokens
  InformativeTokens-->>ClaudePromptHook: Ordered informative tokens
  ClaudePromptHook->>BuildContextPack: Query with joined tokens
  BuildContextPack-->>ContextRenderer: Context items with full claim text
  ContextRenderer-->>ClaudePromptHook: Rendered knowledge context

Loading

Possibly related PRs

• vouchdev/vouch#43: Also changes build_context_pack claim-summary enrichment and return behavior.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main changes: cleaner claim extraction and stopword-filtered retrieval injection.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/extract-fragment-claims

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/vouch/extract.py`:
- Around line 41-42: Update the opening-character gate used by _is_claimworthy
to recognize common Markdown prefix characters, including list bullets and
blockquotes such as -, *, and >. Add these characters to _OPENING_CHARS while
preserving the existing bracket-pair handling and claim evaluation behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: beb576d1-78c3-41be-ba98-ce8d946905a6

📥 Commits

Reviewing files that changed from the base of the PR and between 74abc8e and 06b2f28.

📒 Files selected for processing (5)

• src/vouch/context.py
• src/vouch/extract.py
• src/vouch/hooks.py
• tests/test_extract.py
• tests/test_hooks.py

[coderabbitai]

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

allow markdown list bullets and blockquotes.

this gate drops valid propositions if they start with markdown formatting characters like -, *, or >, as they are not included in _opening_chars. this will silently discard valid list items, blockquotes, and headers when _is_claimworthy checks the first character. consider adding common markdown prefix characters to _opening_chars to preserve these structures.

🐛 proposed fix to preserve markdown lists

-_OPENING_CHARS = "\"'`([{"
+_OPENING_CHARS = "\"'`([{*-+>#"

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	_OPENING_CHARS = "\"'`([{"
	_BRACKET_PAIRS = (("(", ")"), ("[", "]"), ("{", "}"))
	_OPENING_CHARS = "\"'`([{*-+>#"
	_BRACKET_PAIRS = (("(", ")"), ("[", "]"), ("{", "}"))

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/vouch/extract.py` around lines 41 - 42, Update the opening-character gate
used by _is_claimworthy to recognize common Markdown prefix characters,
including list bullets and blockquotes such as -, *, and >. Add these characters
to _OPENING_CHARS while preserving the existing bracket-pair handling and claim
evaluation behavior.