You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 1c4ffa5
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: apps/sim/lib/knowledge/__integration__/knowledge-projection.integration.ts
+41-2Lines changed: 41 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -24,6 +24,7 @@ import {
24
24
credentialGroupEnrollment,
25
25
document,
26
26
embedding,
27
+
embeddingKeywordSearch,
27
28
embeddingKeywordTin,
28
29
embeddingSearch,
29
30
knowledgeBase,
@@ -383,7 +384,6 @@ beforeAll(async () => {
383
384
sql.raw(`CREATE SCHEMA tin;
384
385
CREATE FUNCTION tin.full_score(tid) RETURNS double precision LANGUAGE sql IMMUTABLE AS 'SELECT 1.0::float8';
385
386
CREATE FUNCTION knowledge_tin_base_token(text) RETURNS text LANGUAGE sql IMMUTABLE AS $$SELECT 'kb'$$;
386
-
CREATE FUNCTION knowledge_tin_membership_key(text) RETURNS bigint LANGUAGE sql IMMUTABLE AS $$SELECT hashtextextended('embedding_keyword_tin:' || $1, 0)$$;
387
387
CREATE FUNCTION knowledge_tin_stream(vector tsvector) RETURNS text LANGUAGE sql IMMUTABLE AS $$
388
388
SELECT coalesce(string_agg(entry.lexeme, ' ' ORDER BY position), '')
389
389
FROM unnest(vector) AS entry(lexeme, positions, weights), unnest(entry.positions) AS position
@@ -400,7 +400,6 @@ afterAll(async () => {
400
400
sql.raw(`DROP OPERATOR IF EXISTS ==> (text, text);
401
401
DROP FUNCTION IF EXISTS tin_fixture_match(text, text);
402
402
DROP FUNCTION IF EXISTS knowledge_tin_base_token(text);
403
-
DROP FUNCTION IF EXISTS knowledge_tin_membership_key(text);
404
403
DROP FUNCTION IF EXISTS knowledge_tin_stream(tsvector);
Copy file name to clipboardExpand all lines: apps/sim/lib/sim-search/indexed/README.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,7 +14,7 @@ While the gate is off:
14
14
- Indexed-only surfaces refuse with `SearchIndexDormantError` (a `409`): the Stats report and connecting a source that crawls into a search index. The indexed document page is not found.
15
15
- A knowledge search that names a search-index knowledge base (the Knowledge block, v1, v2, Sim's knowledge tool) still answers from the documents it already holds, decided on each document exactly as a workspace knowledge base is.
16
16
- Nothing crawls into search indexes: content syncs, member syncs, and processing recovery skip them (`lib/knowledge/connectors/indexing-policy.ts`).
17
-
- The projector owes search-index documents nothing: their marks are released with the rest, and it writes no Tin keyword rows.
17
+
- The projector owes search-index documents nothing: their marks are released with the rest, and it writes no Tin keyword rows. The GIN keyword projection follows `is_search_index` alone, so it keeps its search-index rows either way.
18
18
19
19
## Layout
20
20
@@ -28,7 +28,7 @@ The dormant UI sits in `indexed/` folders next to the component that picks it fr
28
28
## Re-enabling
29
29
30
30
1. Set `SIM_SEARCH_LIVE=false` in both the app and the Trigger.dev environment, and deploy. The container entrypoint (`apps/sim/bootstrap.ts`) mirrors it to `NEXT_PUBLIC_SIM_SEARCH_LIVE` for the client; crawling, processing, and projection read it in whichever process runs them.
31
-
2. Confirm the Tin objects exist (`0019_tin_keyword_projection`, `0024_knowledge_projection_async`), backfill `embedding_keyword_tin`for every search-index knowledge base, and build its index.
31
+
2. Confirm the keyword projection objects exist (`0019_tin_keyword_projection`, `0024_knowledge_projection_async`, `0025_scope_keyword_projections`). Both keyword projections, `embedding_keyword_search` and `embedding_keyword_tin`, hold only search-index rows, written by the chunk triggers and by the trigger on `knowledge_base.is_search_index`. Backfill both for every search-index knowledge base whose rows were removed while dormant, and build the Tin index.
32
32
3. Resume and fully resync the connectors of search-index knowledge bases, so content that went stale while dormant is indexed again.
33
33
34
34
Projection rows written before projections carried their document's source and ACL are decided on their document until they are rewritten.
0 commit comments