feat: content-aware re-rank pass (Phase 2.3)

[hallelx2]

Summary

Adds a content-aware re-rank pass that runs after the retrieval Strategy.Select. Takes the strategy's selected section IDs + their loaded content and issues one LLM call to score each section (0-100) against the query. Sections are reordered descending by score; retrieval.rerank.top_k optionally truncates the survivors.

This rescues the failure mode where the strategy reasoned over title + summary + HyDE candidate questions alone and got fooled by surface-level lexical matches — reading the actual content closes the gap. One extra LLM call per query, ~3-5k input tokens, ~$0.0003 on gemini-2.5-flash for a 5-candidate × 2k-char call.

Design

• pkg/retrieval/rerank.go — ReRanker with the same defensive shape as the planner: tolerant JSON parsing, retry on parse failure, graceful degradation to input order when retries are exhausted. Every input ID surfaces in the output exactly once — re-rank can reorder but never drops candidates.
• pkg/config — new retrieval.rerank block (enabled, model, max_content_chars, top_k) plus VLE_RETRIEVAL_RERANK_* env overrides following the same pattern as the existing planning + agentic + answer_span env handling.
• Server wiring — per-request enable_rerank body field on QueryRequest and AnswerRequest. When the pass runs, sections (in /v1/query) and citations (in /v1/answer) carry a score field. For /v1/answer the re-rank usage is added to the response's usage block.

Failure envelope

• Re-rank is opt-in: defaults to enabled=false. Existing callers see zero behaviour change.
• Transport / parse / hallucination failures NEVER drop sections. At worst the strategy's original order is preserved.
• Per-request enable_rerank overrides the config so callers can experiment without a server restart.

Opt-in instructions

retrieval:
  rerank:
    enabled: true
    model: \"gemini-2.5-flash\"      # or your cheap/fast model
    max_content_chars: 2000
    top_k: 5                       # 0 = keep all candidates

Or per-request:

{
  \"document_id\": \"...\",
  \"query\": \"What was Apple's fiscal 2023 revenue?\",
  \"enable_rerank\": true
}

Test plan

• go build ./... clean.
• go vet ./... clean.
• go test ./... — all green (existing tests still pass).
• 16 new ReRanker table-driven tests: happy path / empty input / LLM transport failure / parse-retry exhaustion / bad-JSON-then-success / hallucinated IDs / missing IDs / duplicate IDs / negative-score clamp / prompt content + truncation / nil LLM no-op / nil receiver no-op / parser code-fence + leading-prose + empty input.
• 4 new config tests: defaults, env override happy path, env disable of a YAML-set true, env rejects garbage numbers, validate rejects negatives.

[sourcery-ai]

Sorry @hallelx2, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[coderabbitai]

Warning

Review limit reached

@hallelx2, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 41 minutes and 44 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9842ec9b-de4d-4b41-b05e-9367990b64b3

📥 Commits

Reviewing files that changed from the base of the PR and between 54ae0d5 and 478e578.

📒 Files selected for processing (8)

• cmd/engine/main.go
• config.example.yaml
• internal/api/server.go
• openapi.yaml
• pkg/config/config.go
• pkg/config/config_test.go
• pkg/retrieval/rerank.go
• pkg/retrieval/rerank_test.go

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/rerank-with-content

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}