fix(storage): prevent full-text scores from contributing as vector scores - #260
Conversation
aptend
left a comment
aptend
left a comment
There was a problem hiding this comment.
Deep review of exact head a936812 against base/merge-base 9962999. The change only clears retrieval_score when a full-text result is appended as a full-text-only candidate; the score is first retained in ft_map, and overlapping candidates keep their vector-origin score. I traced the downstream hybrid scorer, which reads retrieval_score solely as vec_score and ft_map as kw_score, and checked deduplication/order and error paths. The focused regression and all 49 runnable memoria-storage library tests pass; one DB-dependent test is intentionally ignored, and database-backed integration was not rerun locally. CI checks are green and git diff --check is clean. No blocking issue found.
Merge Queue Status
This pull request spent 9 seconds in the queue, including 2 seconds running CI. Required conditions to merge
|
What type of PR is this?
Which issue(s) this PR fixes
Fixes #259
What this PR does / why we need it
Full-text-only candidates retained their raw full-text score in
retrieval_score, which the hybrid scorer then interpreted as a vector score. This gave those candidates an unintended additional ranking contribution.Preserve full-text scores in the existing keyword-score map and clear the vector-facing score only when appending full-text-only candidates. Candidates returned by vector search, including overlapping candidates, keep their vector scores. The merge is extracted into a private helper so its score provenance can be tested without a database.
Validation
cargo test --manifest-path memoria/Cargo.toml -p memoria-storage hybrid_merge_keeps_fulltext_scores_out_of_vector_score --lib(1 test). This calls the production merge helper with mocked vector-only, overlapping, and full-text-only memories and checks candidate order, deduplication, and score provenance.memory.retrieval_score = Nonemakes the same test fail (F: Some(2.0)instead ofNone); restoring it makes the test pass.git diff --checkpasses. Package formatting check reports pre-existing formatting differences outside this change; those were left untouched.Database-backed integration tests have not been run.