grounded search over the docs: Hugging Face embeddings into a Joltrin vector store, with citations and I don't know

[gerardrecinto]

examples/hf_grounded_search chunks five docs by heading, embeds each chunk with sentence-transformers/all-MiniLM-L6-v2, stores the vectors in a Joltrin vector store through the Python bindings (one transaction), and answers a question with the nearest passages and their file#heading citations. If the best match scores under a threshold it says it does not know.

The embedding is written with transformers and torch directly: tokenize, run the model, average the token vectors while ignoring padding, normalize to length 1. The model is pinned to one Hub commit so the results cannot move.

Retrieval only. Nothing here writes new text, so it cannot invent a sentence. Handing the passages to a language model is a separate step this does not take.

test_search.py runs real embeddings against a real store and asserts: the store's score equals the cosine similarity (0.4572 both ways on the check), all 8 questions the docs answer find their source file in the top 3, and all 6 questions about other topics are refused. The highest out-of-topic score was 0.17 and the lowest in-topic best score was 0.30, so the 0.25 threshold sits in a gap of 0.13. That is 14 questions, a starting point and not a benchmark. It also prints three near-topic probes, and two of them (a PostgreSQL replication question and a Kubernetes liveness probe question) were answered, because a score threshold cannot tell close to the topic from answered by the docs. I left that visible on purpose.

Run on an Apple silicon Mac with Python 3.12 and torch 2.14.1. A new hf-search-example workflow builds the Linux native library, installs CPU torch and runs the same test when the example, the Python bindings or the indexed docs change. It is not a required check, and I have not seen it run in CI yet.

Thanks, Gerard Recinto

[github-actions]

Gemini PR Review

Reviewed commit: e1fa9b2b05e2df46f8acbb874817fe39678383c6
Verdict: FAIL

• Correctness / Leak: The search.py script and test_search.py both create temporary directories using tempfile.mkdtemp (in Index initialization) but do not explicitly clean them up when the scripts exit. This can lead to an accumulation of unused directories and files on the filesystem, consuming disk space. A try...finally block with shutil.rmtree should be used to ensure these temporary resources are released.
  • File: examples/hf_grounded_search/search.py
  • File: examples/hf_grounded_search/test_search.py

This result blocks merge. Push a fix and comment /gemini review. To contest a finding, see the Gemini dispute process in CONTRIBUTING.md.

[github-actions]

Gemini PR Review

Reviewed commit: e95f371efbd954b8755af7a302a4ba7080336056
Verdict: FAIL

• Correctness Bug / Resource Leak in examples/hf_grounded_search/search.py and examples/hf_grounded_search/test_search.py: The temporary directory created by tempfile.mkdtemp to house the Joltrin vector store is not explicitly deleted after the program or test execution. This leaves behind temporary files and directories on the filesystem, leading to a resource leak.
• Correctness / Robustness in examples/hf_grounded_search/requirements.txt: The transformers and huggingface-hub dependencies are not pinned to a specific version. This can lead to non-reproducible environments, where future installs might pull in newer versions that could introduce breaking changes or alter behavior, potentially causing the example or its tests to fail unexpectedly, despite the model being pinned by revision.

This result blocks merge. Push a fix and comment /gemini review. To contest a finding, see the Gemini dispute process in CONTRIBUTING.md.

[gerardrecinto]

/gemini review

[github-actions]

Gemini PR Review

Reviewed commit: e95f371efbd954b8755af7a302a4ba7080336056
Verdict: FAIL

• examples/hf_grounded_search/requirements.txt: The transformers and huggingface-hub dependencies are not pinned to specific versions. This can lead to non-reproducible builds or unexpected behavior/failures if future versions introduce breaking changes or security vulnerabilities, which is an unhandled edge case for the stability of the example and its associated CI workflow.

This result blocks merge. Push a fix and comment /gemini review. To contest a finding, see the Gemini dispute process in CONTRIBUTING.md.

[gerardrecinto]

/gemini review

[github-actions]

Gemini PR Review

Reviewed commit: 106a5b5ff64a8347fb0e36d368e4bd713232dc65
Verdict: FAIL

• Correctness/Leaks: examples/hf_grounded_search/search.py and examples/hf_grounded_search/test_search.py create temporary directories using tempfile.mkdtemp but do not explicitly delete them. While these are short-lived scripts and the operating system will eventually clean up temporary files, it is generally good practice to explicitly clean up resources when they are no longer needed to prevent minor resource leaks and ensure predictable behavior in all environments.

This result blocks merge. Push a fix and comment /gemini review. To contest a finding, see the Gemini dispute process in CONTRIBUTING.md.

[github-actions]

Gemini PR Review

Reviewed commit: 106a5b5ff64a8347fb0e36d368e4bd713232dc65
Verdict: FAIL

• Correctness Bug / Resource Leak: In examples/hf_grounded_search/search.py (main function) and examples/hf_grounded_search/test_search.py (test setup), tempfile.mkdtemp is used to create temporary directories for the vector store, but these directories are not explicitly cleaned up after the script or test run completes. This leads to leftover files on the filesystem. While not a critical memory leak for short-lived scripts/tests, it's a resource management oversight; using tempfile.TemporaryDirectory as a context manager would ensure automatic cleanup.

This result blocks merge. Push a fix and comment /gemini review. To contest a finding, see the Gemini dispute process in CONTRIBUTING.md.

[gerardrecinto]

/gemini review

[github-actions]

Gemini PR Review

Reviewed commit: 25f21d184994651e6cc39fdb1cb2e160d2e2d3ae
Verdict: PASS

[gerardrecinto]

/gemini review

[github-actions]

Gemini PR Review

Reviewed commit: 25f21d184994651e6cc39fdb1cb2e160d2e2d3ae
Verdict: PASS

• No actionable findings.