test: search reflects publish/unpublish (DataSpaceBackend#193) - #104
Merged
Merged
Conversation
It was readonly-only, so it ran on prod deploys but never on dev deploys or in the PR gate. It is the check that caught DataSpaceBackend#193 (stale search totals) on a prod deploy; with smoke it catches the same regression on dev first.
Covers the publish/unpublish -> search path from DataSpaceBackend#193. Filters search on the automated-test tag because title search is a fuzzy term query against 4-gram tokens and cannot match a long unique title.
Regression for DataSpaceBackend#193: the query is cached before publishing, then read immediately after publish and after unpublish with no wait. Stale cache invalidation or an unrefreshed index write fails it.
saqibmanan
force-pushed
the
test-sync/DataSpaceBackend-pr193
branch
from
September 18, 2026 09:09
9a07430 to
8ce3679
Compare
saqibmanan
changed the title
saqibmanan
marked this pull request as ready for review
September 18, 2026 11:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Covers CivicDataLab/DataSpaceBackend#193 (merge
89cb9e4). Base:CI, the branch whose workflows run.What changed (from the diff): search results are cached for an hour under a version key that publishing bumps. Before #193 the bump ran before the Elasticsearch write, and the write wasn't refreshed, so a search right after publish or unpublish re-cached the old results for the full hour. #193 moved invalidation after a refreshed write, and stopped the version key from expiring.
Tests, one per marker:
smoketest_sitemap_datasets_count_matches_backend(existing, read-only) now also markedsmokefunctionaltest_published_dataset_is_searchable: publish → findable, unpublish → gone (polls up to 20s)regressiontest_search_reflects_publish_without_waiting: query cached first, then read immediately after publish and after unpublish, no waitThe sitemap check is what caught #193 on a prod deploy, but it was
readonly-only, so it never ran on dev or in the PR gate. The two new tests write (they publish), so they live intests/api/functionaland never run on prod.Search membership is checked by filtering on the
automated-testtag, not a text query: title search runs afuzzyterm query against 4-gram tokens, so a long unique title never matches (andKHOJfinds 0 whilekhojfinds 2).Proof against dev, retries off:
2 passed(functional + regression); sitemap1 passed2 failed; sitemap window flipped:1 failedNot covered: the version key's no-expiry fix. Proving it needs a >300s wait against a shared cache, which doesn't belong in CI.