chore: clarify sorted table and index creation constraints

Author: hotdata-automation[bot]

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- chore: clarify sorted table and index creation constraints
 - feat(indexes): add nlist field to index responses
 - chore(uploads): clarify finalized upload retention period
 - chore(loads): clarify idempotency key conflict behavior

docs/AddManagedTableDecl.md

@@ -10,7 +10,7 @@ Name | Type | Description | Notes
 **key** | **List[str]** | Columns that uniquely identify a row, enabling the key-based load modes (`delete`, `update`, `upsert`) on this table: those loads match rows by these columns' values. Omit (the default) to declare no key; the table can still be loaded with `replace` and `append`, but key-based modes are then rejected. | [optional] 
 **name** | **str** |  | 
 **partition_by** | [**List[TablePartitionKey]**](TablePartitionKey.md) | Partition keys for this table, applied in order. Omit for no partitioning. Declared when the table is created and fixed thereafter. | [optional] 
-**sorted_by** | [**List[TableSortKey]**](TableSortKey.md) | Sort keys for this table, applied in order. Omit for no sort order. Declared when the table is created and fixed thereafter. | [optional] 
+**sorted_by** | [**List[TableSortKey]**](TableSortKey.md) | Sort keys for this table, applied in order. Omit for no sort order. Declared when the table is created and fixed thereafter. A table with a sort order cannot be indexed: creating any index on it is refused, so leave this empty on a table you plan to search. | [optional] 
 
 ## Example
 

docs/AddManagedTableRequest.md

@@ -10,7 +10,7 @@ Name | Type | Description | Notes
 **key** | **List[str]** | Columns that uniquely identify a row, enabling the key-based load modes (`delete`, `update`, `upsert`) on this table: those loads match rows by these columns' values. Omit (the default) to declare no key; the table can still be loaded with `replace` and `append`, but key-based modes are then rejected. | [optional] 
 **name** | **str** |  | 
 **partition_by** | [**List[TablePartitionKey]**](TablePartitionKey.md) | Partition keys for this table, applied in order. Omit for no partitioning. Declared when the table is created and fixed thereafter. | [optional] 
-**sorted_by** | [**List[TableSortKey]**](TableSortKey.md) | Sort keys for this table, applied in order. Omit for no sort order. Declared when the table is created and fixed thereafter. | [optional] 
+**sorted_by** | [**List[TableSortKey]**](TableSortKey.md) | Sort keys for this table, applied in order. Omit for no sort order. Declared when the table is created and fixed thereafter. A table with a sort order cannot be indexed: creating any index on it is refused, so leave this empty on a table you plan to search. | [optional] 
 
 ## Example
 

docs/CreateIndexRequest.md

@@ -1,6 +1,6 @@
 # CreateIndexRequest
 
-Request body for POST .../indexes  Two constraints span the whole table rather than this request alone.  A table holds at most one vector index, whichever algorithm it uses. A second vector index is refused until the first is dropped.  And a vector index that generates its own embeddings — that is, one created with `embedding_provider_id` — has to be the only index on its table. So a table that already carries any index (sorted, full-text, or vector) will not accept an embedding-backed vector index, and a table that already carries an embedding-backed vector index will not accept any further index of any type. To move between the two arrangements, drop what is there first. Plan for it when designing a table: combining full-text search with generated embeddings on one table is not possible, so use a separate table for the second index, or supply the embeddings yourself.  A vector index over a column that already holds vectors — no `embedding_provider_id` — is not affected by that second rule and coexists with sorted and full-text indexes normally.  Embedding generation also rewrites the table to add its generated column, and that rewrite cannot preserve a declared partition or sort order. An embedding-backed vector index is therefore refused on a table declaring either.
+Request body for POST .../indexes  Two constraints span the whole table rather than this request alone.  A table holds at most one vector index, whichever algorithm it uses. A second vector index is refused until the first is dropped.  And a vector index that generates its own embeddings — that is, one created with `embedding_provider_id` — has to be the only index on its table. So a table that already carries any index (sorted, full-text, or vector) will not accept an embedding-backed vector index, and a table that already carries an embedding-backed vector index will not accept any further index of any type. To move between the two arrangements, drop what is there first. Plan for it when designing a table: combining full-text search with generated embeddings on one table is not possible, so use a separate table for the second index, or supply the embeddings yourself.  A vector index over a column that already holds vectors — no `embedding_provider_id` — is not affected by that second rule and coexists with sorted and full-text indexes normally.  Embedding generation also rewrites the table to add its generated column, and that rewrite cannot preserve a declared partition. An embedding-backed vector index is therefore refused on a partitioned table.  No index of any type can be created on a table that declares a sort order (`sorted_by`). Such a request is refused with 400.
 
 ## Properties
 

docs/DatabaseDefaultTableDecl.md

@@ -10,7 +10,7 @@ Name | Type | Description | Notes
 **key** | **List[str]** | Columns that uniquely identify a row, enabling the key-based load modes (`delete`, `update`, `upsert`) on this table: those loads match rows by these columns' values. Omit (the default) to declare no key; the table can still be loaded with `replace` and `append`, but key-based modes are then rejected. | [optional] 
 **name** | **str** |  | 
 **partition_by** | [**List[TablePartitionKey]**](TablePartitionKey.md) | Partition keys for this table, applied in order. Omit for no partitioning. Declared when the table is created and fixed thereafter. | [optional] 
-**sorted_by** | [**List[TableSortKey]**](TableSortKey.md) | Sort keys for this table, applied in order. Omit for no sort order. Declared when the table is created and fixed thereafter. | [optional] 
+**sorted_by** | [**List[TableSortKey]**](TableSortKey.md) | Sort keys for this table, applied in order. Omit for no sort order. Declared when the table is created and fixed thereafter. A table with a sort order cannot be indexed: creating any index on it is refused, so leave this empty on a table you plan to search. | [optional] 
 
 ## Example
 

docs/IndexesApi.md

@@ -15,7 +15,7 @@ Method | HTTP request | Description
 
 Create an index on a table
 
-Create a sorted, BM25 full-text, or vector index on a table. The table must have data loaded. A table can hold several indexes, with two limits: it holds at most one vector index, whichever algorithm it uses; and a vector index that generates its own embeddings (created with `embedding_provider_id`) must be the only index on its table, so it cannot be added alongside an existing index and no further index can be added alongside it. A request that breaks either limit is refused with 400, including when `async` is set. The request body describes the rules in full.
+Create a sorted, BM25 full-text, or vector index on a table. The table must have data loaded, and must not declare a sort order (`sorted_by`): a table with a sort order cannot be indexed. A table can hold several indexes, with two limits: it holds at most one vector index, whichever algorithm it uses; and a vector index that generates its own embeddings (created with `embedding_provider_id`) must be the only index on its table, so it cannot be added alongside an existing index and no further index can be added alongside it. A request refused for any of these reasons gets 400, including when `async` is set. The request body describes the rules in full.
 
 ### Example
 

hotdata/api/indexes_api.py

@@ -64,7 +64,7 @@ def create_index(
     ) -> IndexInfoResponse:
         """Create an index on a table
 
-        Create a sorted, BM25 full-text, or vector index on a table. The table must have data loaded. A table can hold several indexes, with two limits: it holds at most one vector index, whichever algorithm it uses; and a vector index that generates its own embeddings (created with `embedding_provider_id`) must be the only index on its table, so it cannot be added alongside an existing index and no further index can be added alongside it. A request that breaks either limit is refused with 400, including when `async` is set. The request body describes the rules in full.
+        Create a sorted, BM25 full-text, or vector index on a table. The table must have data loaded, and must not declare a sort order (`sorted_by`): a table with a sort order cannot be indexed. A table can hold several indexes, with two limits: it holds at most one vector index, whichever algorithm it uses; and a vector index that generates its own embeddings (created with `embedding_provider_id`) must be the only index on its table, so it cannot be added alongside an existing index and no further index can be added alongside it. A request refused for any of these reasons gets 400, including when `async` is set. The request body describes the rules in full.
 
         :param connection_id: The database's `default_connection_id`, as returned when the database is created or fetched (required)
         :type connection_id: str
@@ -147,7 +147,7 @@ def create_index_with_http_info(
     ) -> ApiResponse[IndexInfoResponse]:
         """Create an index on a table
 
-        Create a sorted, BM25 full-text, or vector index on a table. The table must have data loaded. A table can hold several indexes, with two limits: it holds at most one vector index, whichever algorithm it uses; and a vector index that generates its own embeddings (created with `embedding_provider_id`) must be the only index on its table, so it cannot be added alongside an existing index and no further index can be added alongside it. A request that breaks either limit is refused with 400, including when `async` is set. The request body describes the rules in full.
+        Create a sorted, BM25 full-text, or vector index on a table. The table must have data loaded, and must not declare a sort order (`sorted_by`): a table with a sort order cannot be indexed. A table can hold several indexes, with two limits: it holds at most one vector index, whichever algorithm it uses; and a vector index that generates its own embeddings (created with `embedding_provider_id`) must be the only index on its table, so it cannot be added alongside an existing index and no further index can be added alongside it. A request refused for any of these reasons gets 400, including when `async` is set. The request body describes the rules in full.
 
         :param connection_id: The database's `default_connection_id`, as returned when the database is created or fetched (required)
         :type connection_id: str
@@ -230,7 +230,7 @@ def create_index_without_preload_content(
     ) -> RESTResponseType:
         """Create an index on a table
 
-        Create a sorted, BM25 full-text, or vector index on a table. The table must have data loaded. A table can hold several indexes, with two limits: it holds at most one vector index, whichever algorithm it uses; and a vector index that generates its own embeddings (created with `embedding_provider_id`) must be the only index on its table, so it cannot be added alongside an existing index and no further index can be added alongside it. A request that breaks either limit is refused with 400, including when `async` is set. The request body describes the rules in full.
+        Create a sorted, BM25 full-text, or vector index on a table. The table must have data loaded, and must not declare a sort order (`sorted_by`): a table with a sort order cannot be indexed. A table can hold several indexes, with two limits: it holds at most one vector index, whichever algorithm it uses; and a vector index that generates its own embeddings (created with `embedding_provider_id`) must be the only index on its table, so it cannot be added alongside an existing index and no further index can be added alongside it. A request refused for any of these reasons gets 400, including when `async` is set. The request body describes the rules in full.
 
         :param connection_id: The database's `default_connection_id`, as returned when the database is created or fetched (required)
         :type connection_id: str

hotdata/models/add_managed_table_decl.py

@@ -33,7 +33,7 @@ class AddManagedTableDecl(BaseModel):
     key: Optional[List[StrictStr]] = Field(default=None, description="Columns that uniquely identify a row, enabling the key-based load modes (`delete`, `update`, `upsert`) on this table: those loads match rows by these columns' values. Omit (the default) to declare no key; the table can still be loaded with `replace` and `append`, but key-based modes are then rejected.")
     name: StrictStr
     partition_by: Optional[List[TablePartitionKey]] = Field(default=None, description="Partition keys for this table, applied in order. Omit for no partitioning. Declared when the table is created and fixed thereafter.")
-    sorted_by: Optional[List[TableSortKey]] = Field(default=None, description="Sort keys for this table, applied in order. Omit for no sort order. Declared when the table is created and fixed thereafter.")
+    sorted_by: Optional[List[TableSortKey]] = Field(default=None, description="Sort keys for this table, applied in order. Omit for no sort order. Declared when the table is created and fixed thereafter. A table with a sort order cannot be indexed: creating any index on it is refused, so leave this empty on a table you plan to search.")
     __properties: ClassVar[List[str]] = ["constant_per_key", "key", "name", "partition_by", "sorted_by"]
 
     model_config = ConfigDict(

hotdata/models/add_managed_table_request.py

@@ -33,7 +33,7 @@ class AddManagedTableRequest(BaseModel):
     key: Optional[List[StrictStr]] = Field(default=None, description="Columns that uniquely identify a row, enabling the key-based load modes (`delete`, `update`, `upsert`) on this table: those loads match rows by these columns' values. Omit (the default) to declare no key; the table can still be loaded with `replace` and `append`, but key-based modes are then rejected.")
     name: StrictStr
     partition_by: Optional[List[TablePartitionKey]] = Field(default=None, description="Partition keys for this table, applied in order. Omit for no partitioning. Declared when the table is created and fixed thereafter.")
-    sorted_by: Optional[List[TableSortKey]] = Field(default=None, description="Sort keys for this table, applied in order. Omit for no sort order. Declared when the table is created and fixed thereafter.")
+    sorted_by: Optional[List[TableSortKey]] = Field(default=None, description="Sort keys for this table, applied in order. Omit for no sort order. Declared when the table is created and fixed thereafter. A table with a sort order cannot be indexed: creating any index on it is refused, so leave this empty on a table you plan to search.")
     __properties: ClassVar[List[str]] = ["constant_per_key", "key", "name", "partition_by", "sorted_by"]
 
     model_config = ConfigDict(

hotdata/models/create_index_request.py

@@ -26,7 +26,7 @@
 
 class CreateIndexRequest(BaseModel):
     """
-    Request body for POST .../indexes  Two constraints span the whole table rather than this request alone.  A table holds at most one vector index, whichever algorithm it uses. A second vector index is refused until the first is dropped.  And a vector index that generates its own embeddings — that is, one created with `embedding_provider_id` — has to be the only index on its table. So a table that already carries any index (sorted, full-text, or vector) will not accept an embedding-backed vector index, and a table that already carries an embedding-backed vector index will not accept any further index of any type. To move between the two arrangements, drop what is there first. Plan for it when designing a table: combining full-text search with generated embeddings on one table is not possible, so use a separate table for the second index, or supply the embeddings yourself.  A vector index over a column that already holds vectors — no `embedding_provider_id` — is not affected by that second rule and coexists with sorted and full-text indexes normally.  Embedding generation also rewrites the table to add its generated column, and that rewrite cannot preserve a declared partition or sort order. An embedding-backed vector index is therefore refused on a table declaring either.
+    Request body for POST .../indexes  Two constraints span the whole table rather than this request alone.  A table holds at most one vector index, whichever algorithm it uses. A second vector index is refused until the first is dropped.  And a vector index that generates its own embeddings — that is, one created with `embedding_provider_id` — has to be the only index on its table. So a table that already carries any index (sorted, full-text, or vector) will not accept an embedding-backed vector index, and a table that already carries an embedding-backed vector index will not accept any further index of any type. To move between the two arrangements, drop what is there first. Plan for it when designing a table: combining full-text search with generated embeddings on one table is not possible, so use a separate table for the second index, or supply the embeddings yourself.  A vector index over a column that already holds vectors — no `embedding_provider_id` — is not affected by that second rule and coexists with sorted and full-text indexes normally.  Embedding generation also rewrites the table to add its generated column, and that rewrite cannot preserve a declared partition. An embedding-backed vector index is therefore refused on a partitioned table.  No index of any type can be created on a table that declares a sort order (`sorted_by`). Such a request is refused with 400.
     """ # noqa: E501
     algorithm: Optional[StrictStr] = Field(default=None, description="How a vector index organises the vectors it searches. Omit this field for `hnsw`, which is the default.  `hnsw` — builds a graph of the vectors and keeps it in memory. Searches are very fast, and the memory a search needs grows with the whole table, so a large enough table cannot be served at all.  `ivf` — groups the vectors into clusters and reads only the clusters nearest the search. Searches are considerably slower than `hnsw`, and the memory a search needs follows how much of the index it reads rather than the size of the table, so a table far too large for `hnsw` can still be searched. It keeps a copy of the table's rows beside the vectors so a search is answered without reading the table; that copy is extra storage, and how much depends on `vector_precision`, which decides how compactly the copied vectors are held. Available for columns that already hold vectors, with the `l2` and `cosine` metrics.")
     var_async: Optional[StrictBool] = Field(default=False, description="When true, create the index as a background job and return a job ID for polling.", alias="async")