Raw natural language search poc - #3597
Open
nsantacruz wants to merge 5 commits into
Open
Conversation
Disables Elasticsearch keyword search on the /search page and instead renders results from the semantic KNN search pipeline: up to 40 direct semantic matches followed by up to 10 link-origin matches, each tagged with a chip. Filter sidebar and sort dropdown are hidden since they aren't useful for this few, non-facetable results. Sheet search is unaffected (KNN index only covers texts). /api/knn-search is bearer-token gated for external/tool callers, so this adds a public same-origin proxy (api/search-wrapper/semantic) that reuses the same core search logic (KnnSearch.run_search) without shipping the shared secret to browser JS. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
SemanticSearchQuerier set isQueryRunning but left the previous query's hits in state, so SearchResultList kept rendering the old results alongside the loading spinner until the new response landed -- every search appeared to be one query behind. Reset hits on requery, matching ElasticSearchQuerier's existing behavior. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
📊 Code Quality Score: 30/100
Was this score accurate? 👍 Yes · 👎 No Scored by GitVelocity · How are scores calculated? |
The KNN search wrapper endpoint (/api/search-wrapper/semantic) did the Gemini embedding call and pgvector KNN search synchronously in the web request, which can be slow. It now enqueues the search as a Celery task (semantic_search.tasks.knn_search_task) and returns a task_id; the frontend polls the existing generic /api/async/<task_id> endpoint until the task completes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
autodiscover_tasks only scanned a fixed list of packages, so the worker never imported semantic_search/tasks.py and rejected semantic_search.knn_search as unregistered.
KnnSearch.run_search now executes inside a Celery task, but the tasks Rollout template never injected GEMINI_API_KEY like the web pod does, so semantic search 503'd with "Semantic search is not configured" on any environment relying on the secret (works locally only because local_settings.py hardcodes a key). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A brief description of the PR
Code Changes
The following changes were made to the files below
Notes
Any additional notes go here