Add Neon skills for database project access

.agents/skills/neon-postgres/references/full-text-search.md

@@ -0,0 +1,99 @@
+# Full-Text Search with BM25 Ranking
+
+Use `lakebase_text` for BM25 relevance ranking with PostgreSQL's standard `tsvector` type. The `lakebase_bm25` index adds corpus-aware ranking and top-K pushdown.
+
+Lakebase Search requires Postgres 16 or later. Enable the extension before creating the index:
+
+```sql
+CREATE EXTENSION IF NOT EXISTS lakebase_text;
+```
+
+`lakebase_text` has no extension dependency. It relies on a preloaded library that Neon enables by default; if the project customized its preloaded-library list, confirm the library remains enabled.
+
+## Prepare and Index Text
+
+Prefer a stored generated `tsvector` when search text comes from stable table columns:
+
+```sql
+CREATE TABLE documents (
+  id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+  title text NOT NULL,
+  body text NOT NULL,
+  body_tsv tsvector GENERATED ALWAYS AS
+    (to_tsvector('english', body)) STORED
+);
+```
+
+Create the index after the initial corpus has been inserted so build-time corpus statistics are meaningful. BM25 scoring is tuned by two storage parameters set at index-build time:
+
+- `k1` controls term-frequency saturation (default `1.2`, range `1.2`–`2.0`): higher values let repeated terms keep adding relevance.
+
+- `b` controls document-length normalization (default `0.75`, range `0.0`–`1.0`): higher values penalize longer documents more.
+
+Both can only be set in the `WITH` clause, and updating them rebuilds the index:
+
+```sql
+CREATE INDEX documents_body_bm25 ON documents
+  USING lakebase_bm25 (body_tsv)
+  WITH (k1 = 1.2, b = 0.75);
+```
+
+After a large bulk load, run `VACUUM` to refresh the statistics used by BM25 scoring.
+
+## Query and Interpret Scores
+
+`to_bm25query` binds the query `tsvector` to the BM25 index whose corpus statistics should be used. The `<@>` operator returns a negative BM25 score, so lower (more negative) values are more relevant and must sort ascending:
+
+```sql
+SELECT
+  id,
+  title,
+  body_tsv <@> to_bm25query(
+    to_tsvector('english', $1),
+    'documents_body_bm25'::regclass
+  ) AS score
+FROM documents
+ORDER BY score
+LIMIT $2;
+```
+
+Use the same text-search configuration for document and query vectors. Select a language-specific or custom configuration that matches the corpus.
+
+## Set the Candidate Limit
+
+`lakebase_bm25.default_limit` controls how many rows the index returns before PostgreSQL applies the SQL `LIMIT`. Its default is `1000`; setting it close to the requested top-K avoids unnecessary scoring.
+
+## Use Prefilter Selectively
+
+Enable prefilter when a `WHERE` condition is strict or unpredictable and cheap to evaluate. It lets the index prune rows before BM25 scoring. A loose or expensive filter can be slower with prefilter enabled.
+
+```sql
+BEGIN;
+SET LOCAL lakebase_bm25.default_limit = 20;
+SET LOCAL lakebase_bm25.prefilter = on;
+
+SELECT
+  id,
+  title,
+  body_tsv <@> to_bm25query(
+    to_tsvector('english', $1),
+    'documents_body_bm25'::regclass
+  ) AS score
+FROM documents
+WHERE id % 1000 = 0
+ORDER BY score
+LIMIT $2;
+COMMIT;
+```
+
+## Set Parameters at Build Time or Per Query
+
+Several BM25 parameters can be set in two places. As an index storage parameter in the `CREATE INDEX` `WITH` clause, a value is fed into the index as its build-time default. As a session GUC via `SET` (or `SET LOCAL`), it applies per query and takes precedence over the stored default when both are present.
+
+- `default_limit` and `prefilter` exist in both forms: set an index default that fits the common case, then override it per query with a GUC without rebuilding.
+- `k1` (default `1.2`) and `b` (default `0.75`) are storage parameters only. There is no GUC for them.
+- `enable_scan` (default `on`) is a GUC only.
+
+The examples use `SET LOCAL` so each override is scoped to its own transaction, which is required behind a connection pool or stateless driver.
+
+Source: [`lakebase_text` documentation](https://neon.com/docs/extensions/lakebase-text).

.agents/skills/neon-postgres/references/hybrid-search.md

@@ -0,0 +1,90 @@
+# Hybrid Search
+
+Use hybrid search when either semantic similarity or exact vocabulary can identify a relevant document. Lakebase Search does not provide a built-in hybrid function: run vector and BM25 retrieval separately, then combine their results with a fusion strategy suited to the workload.
+
+Lakebase Search requires Postgres 16 or later. Hybrid search uses both extensions:
+
+```sql
+CREATE EXTENSION IF NOT EXISTS lakebase_vector CASCADE;
+CREATE EXTENSION IF NOT EXISTS lakebase_text;
+```
+
+`lakebase_vector` installs `pgvector` through `CASCADE`; `lakebase_text` has no extension dependency. Both rely on preloaded libraries that Neon enables by default. If the project customized its preloaded-library list, confirm both libraries remain enabled.
+
+Prepare a table with both vector and text-search columns:
+
+```sql
+CREATE TABLE documents (
+  id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+  title text NOT NULL,
+  body text NOT NULL,
+  embedding vector(1536),
+  body_tsv tsvector GENERATED ALWAYS AS
+    (to_tsvector('english', body)) STORED
+);
+```
+
+Replace `1536` with the embedding model's dimension. Use the same model and preprocessing for stored-document and query embeddings, and choose a PostgreSQL text-search configuration appropriate for the corpus.
+
+Create and validate each retriever independently before combining them. Follow [Vector search](vector-search.md) and [Full-text search](full-text-search.md) for their indexes, query operators, and tuning.
+
+Reciprocal Rank Fusion (RRF) is the approach in the Lakebase Search get-started guide and a useful default because it combines ranks instead of incomparable raw distances and scores. It is not the only option: weighted rank fusion, normalized score fusion, or a reranker may fit applications with different relevance signals.
+
+## RRF Example
+
+For rank `r` and constant `k`, each retriever contributes `1 / (k + r)`. The documented starting point uses 40 candidates per retriever and `k = 60`; tune both for the corpus and workload.
+
+Bind the query embedding as `$1`, query text as `$2`, and final result count as `$3`:
+
+```sql
+WITH vector_ranked AS (
+  SELECT id, RANK() OVER (ORDER BY distance) AS rank
+  FROM (
+    SELECT id, embedding <=> $1::vector AS distance
+    FROM documents
+    ORDER BY distance
+    FETCH FIRST 40 ROWS WITH TIES
+  ) AS vector_candidates
+),
+keyword_ranked AS (
+  SELECT id, RANK() OVER (ORDER BY score) AS rank
+  FROM (
+    SELECT
+      id,
+      body_tsv <@> to_bm25query(
+        to_tsvector('english', $2),
+        'documents_body_bm25'::regclass
+      ) AS score
+    FROM documents
+    ORDER BY score
+    FETCH FIRST 40 ROWS WITH TIES
+  ) AS keyword_candidates
+)
+SELECT
+  d.id,
+  d.title,
+  COALESCE(1.0 / (60 + v.rank), 0) +
+    COALESCE(1.0 / (60 + k.rank), 0) AS rrf_score
+FROM documents AS d
+LEFT JOIN vector_ranked AS v ON v.id = d.id
+LEFT JOIN keyword_ranked AS k ON k.id = d.id
+WHERE v.id IS NOT NULL OR k.id IS NOT NULL
+ORDER BY rrf_score DESC, d.id
+LIMIT $3;
+```
+
+`RANK()` gives tied retrieval scores the same rank. Sort by `rrf_score` descending and use the stable ID as a final tie-breaker.
+
+`FETCH FIRST ... ROWS WITH TIES` keeps every candidate tied at the cutoff, so `RANK()` receives the complete boundary tie group. The candidate set can therefore exceed 40 rows. `lakebase_bm25.default_limit` defaults to `1000`; increase it only when the BM25 candidate set needs to exceed that value.
+
+## Adapt the Hybrid Search
+
+- Retrieve more candidates from each source than the final result count; otherwise one retriever can dominate before fusion has enough overlap. Keep `lakebase_bm25.default_limit` above the BM25 candidate target and allow room for boundary ties.
+- Keep each retriever's operator and index configuration correct independently before tuning RRF.
+- Tune candidate counts and the RRF constant with judged or behavioral relevance data, plus latency measurements.
+- Add weights only when product evidence shows one retriever should contribute more. Weight the reciprocal-rank contributions, not the raw vector distance and negative BM25 score.
+- Apply the same access-control and tenant filters to both candidate CTEs. If BM25 filters are strict and cheap, evaluate whether `lakebase_bm25.prefilter` improves the filtered query.
+
+Source: [Lakebase Search get-started guide][lakebase-search-guide].
+
+[lakebase-search-guide]: https://neon.com/docs/ai/lakebase-search-get-started#combine-results-with-hybrid-search

.agents/skills/neon-postgres/references/vector-search.md

@@ -0,0 +1,137 @@
+# Semantic Vector Search
+
+Use `lakebase_vector` for approximate nearest-neighbor retrieval over embeddings. It retains pgvector's vector types, distance operators, and query syntax; the index access method is `lakebase_ann`.
+
+## Contents
+
+- [Create the extension](#create-the-extension) — enable `lakebase_vector` and its `pgvector` dependency
+- [Prepare embeddings](#prepare-embeddings) — define the vector column and keep embedding dimensions consistent
+- [Build the index](#build-the-index) — match the distance metric, operator class, and query operator
+- [Tune the index](#tune-the-index) — configure index-build options and concurrent index management
+- [Query](#query) — rank by vector distance or filter by a similarity radius
+- [Tune search](#tune-search) — inspect the index and tune recall against latency
+  - [Use prefilter selectively](#use-prefilter-selectively) — apply selective filters before ANN scoring
+
+## Create the Extension
+
+Lakebase Search requires Postgres 16 or later. Enable the extension before creating vector columns or indexes:
+
+```sql
+CREATE EXTENSION IF NOT EXISTS lakebase_vector CASCADE;
+```
+
+`lakebase_vector` installs `pgvector` through `CASCADE`. It relies on a preloaded library that Neon enables by default; if the project customized its preloaded-library list, confirm the library remains enabled.
+
+## Prepare Embeddings
+
+Use any embedding provider whose vector dimensions and distance metric match the schema and index:
+
+```sql
+CREATE TABLE documents (
+  id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+  title text NOT NULL,
+  body text NOT NULL,
+  embedding vector(1536)
+);
+```
+
+Replace `1536` with the embedding model's dimension. Generate stored-document and query embeddings with the same model and preprocessing. Keep embedding generation outside SQL unless the architecture already provides an in-database embedding function.
+
+## Build the Index
+
+Choose the operator class and query operator as a matched pair:
+
+| Metric | Common use | Operator class | Distance operator |
+| --- | --- | --- | --- |
+| Cosine | Most text embeddings | `vector_cosine_ops` | `<=>` |
+| L2 / Euclidean | Absolute distance matters; vectors do not need normalization | `vector_l2_ops` | `<->` |
+| Inner product | Unit-normalized vectors; matches cosine for unit vectors | `vector_ip_ops` | `<#>` |
+
+```sql
+CREATE INDEX documents_embedding_ann ON documents
+  USING lakebase_ann (embedding vector_cosine_ops);
+```
+
+## Tune the Index
+
+The default index options suit most workloads:
+
+- `build_mode = 'standard'` balances recall and index build time. Use `quality` for better recall when a longer build is acceptable.
+- `lists = 'auto'` chooses the IVF partition layout from the number of indexed vectors. Choose between `auto` and a manual value case by case: test both on the target dataset and use the value that better meets recall and performance targets.
+
+To prioritize recall over index build time:
+
+```sql
+CREATE INDEX documents_embedding_ann_quality ON documents
+  USING lakebase_ann (embedding vector_cosine_ops)
+  WITH (build_mode = 'quality');
+```
+
+To override the automatic partition layout instead:
+
+```sql
+CREATE INDEX documents_embedding_ann_lists ON documents
+  USING lakebase_ann (embedding vector_cosine_ops)
+  WITH (lists = '1024');
+```
+
+For a large table, use `CREATE INDEX CONCURRENTLY` to avoid locking out writes while creating the index. For a frequently changing table, periodically use `REINDEX INDEX CONCURRENTLY` to rebuild the index with minimal write locking.
+
+## Query
+
+Generate the query embedding with the same model and preprocessing used for stored documents, then bind it as a parameter:
+
+```sql
+SELECT id, title, embedding <=> $1::vector AS distance
+FROM documents
+ORDER BY distance
+LIMIT $2;
+```
+
+Distance sorts ascending: a smaller value is a closer match. Keep the query operator consistent with the index operator class.
+
+To filter by a similarity radius, use the matching boolean range operator in `WHERE` and the distance operator in `ORDER BY`:
+
+```sql
+SELECT id, title
+FROM documents
+WHERE embedding <<=>> sphere($1::vector, 0.5)
+ORDER BY embedding <=> $1::vector
+LIMIT $2;
+```
+
+The cosine range operator `<<=>>` returns a boolean; do not use it as the ranking expression.
+
+## Tune Search
+
+Inspect the index before overriding defaults:
+
+```sql
+SELECT lakebase_ann_index_info('documents_embedding_ann');
+```
+
+This reports `lists`, `default_probes`, and `default_epsilon`. Small datasets use exact flat search before IVF lists are built. In that state, `lists` and `default_probes` are empty. Leave `lakebase_ann.probes` set to its default of `'auto'`; `lakebase_ann.epsilon` still controls full-precision reranking during flat search.
+
+For an IVF index, `lakebase_ann.probes` controls how many partitions are searched at each level. Higher values generally improve recall at the cost of speed. Its default is `'auto'`. When `lists` is not empty, the shape of `probes` must match the shape of `lists`: use one value for a one-level index or two comma-separated values for a two-level index. At each level, the `probes` value must be no larger than the corresponding `lists` value. A mismatched or out-of-range value causes an error.
+
+`lakebase_ann.epsilon` controls how many candidates are reranked using full-precision distances. Higher values rerank more candidates and take longer. Its default is `'auto'`, which works well for most workloads.
+
+### Use Prefilter Selectively
+
+By default, PostgreSQL applies non-vector filters after the ANN index returns candidate rows. Enable prefilter when a filter is cheap to evaluate and removes most rows. Leave it off for loose or expensive filters.
+
+```sql
+BEGIN;
+SET LOCAL lakebase_ann.prefilter = on;
+
+SELECT id, title
+FROM documents
+WHERE id % 100 = 0
+ORDER BY embedding <=> $1::vector
+LIMIT $2;
+COMMIT;
+```
+
+Start with `probes` and `epsilon` set to `'auto'`. Benchmark manual probe values against representative query embeddings and choose the smallest values that satisfy recall and tail-latency targets. Keep session settings and the query in the same transaction when using a connection pool or stateless driver.
+
+Source: [`lakebase_vector` documentation](https://neon.com/docs/extensions/lakebase-vector).