diff --git a/content/commands/ft.search.md b/content/commands/ft.search.md index bf7b653cc9..10bf62428b 100644 --- a/content/commands/ft.search.md +++ b/content/commands/ft.search.md @@ -399,13 +399,13 @@ limits the attributes returned from the document. `num` is the number of attribu
SUMMARIZE ... -returns only the sections of the attribute that contain the matched text. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}}) for more information. +returns only the sections of the attribute that contain the matched text. On a JSON index, `SUMMARIZE` requires `RETURN` with explicit attribute names and does not support multi-value JSONPaths; JSON support was added in the 8.4 maintenance line (v8.4.14) and the 8.6, 8.8, and 8.10 lines. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}) for the exact versions and the full rules.
HIGHLIGHT ... -formats occurrences of matched text. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}}) for more information. +formats occurrences of matched text. On a JSON index, `HIGHLIGHT` requires `RETURN` with explicit attribute names and does not support multi-value JSONPaths; JSON support was added in the 8.4 maintenance line (v8.4.14) and the 8.6, 8.8, and 8.10 lines. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}) for the exact versions and the full rules.
diff --git a/content/develop/ai/search-and-query/administration/overview.md b/content/develop/ai/search-and-query/administration/overview.md index 9712a0e920..cda1f5f2e6 100644 --- a/content/develop/ai/search-and-query/administration/overview.md +++ b/content/develop/ai/search-and-query/administration/overview.md @@ -297,6 +297,8 @@ Summarization will fragment the text into smaller sized snippets. Each snippet w Highlighting will highlight the found term and its variants with a user-defined tag. This may be used to display the matched text in a different typeface using a markup language, or to otherwise make the text appear differently. +JSON indexes have additional requirements. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}). + ## Autocomplete Another important feature for Redis Open Source is its autocomplete engine. This allows users to create dictionaries of weighted terms, and then query them for completion suggestions to a given user prefix. Completions can have payloads, which are user-provided pieces of data that can be used for display. For example, completing the names of users, it is possible to add extra metadata about users to be displayed. diff --git a/content/develop/ai/search-and-query/advanced-concepts/highlight.md b/content/develop/ai/search-and-query/advanced-concepts/highlight.md index fb2878cce2..8f0aba2d9c 100644 --- a/content/develop/ai/search-and-query/advanced-concepts/highlight.md +++ b/content/develop/ai/search-and-query/advanced-concepts/highlight.md @@ -90,3 +90,91 @@ The `RETURN` keyword is treated specially, as it overrides any fields specified In the command `RETURN 1 foo SUMMARIZE FIELDS 1 bar HIGHLIGHT FIELDS 1 baz`, the fields `foo` is returned as-is, while `bar` and `baz` are not returned, because `RETURN` was specified, but did not include those fields. In the command `SUMMARIZE FIELDS 1 bar HIGHLIGHT FIELDS 1 baz`, `bar` is returned summarized and `baz` is returned highlighted. + +## JSON indexes + +`HIGHLIGHT` and `SUMMARIZE` work on a JSON index when the field maps to a single-value +[JSONPath]({{< relref "/develop/data-types/json/path" >}}) such as `$.name`. + +Added in the 8.4 maintenance line (v8.4.14), the 8.6 line (v8.6.10), the 8.8 line (v8.8.1), +the 8.10 line (v8.10.1), and Redis Open Source 8.12. Not available in Redis Open Source 8.2 +or earlier, and not present in the initial 8.4.0, 8.6.0, 8.8.0, or 8.10.0 releases, which +reject `HIGHLIGHT` and `SUMMARIZE` on every JSON index. + +Three rules apply to JSON indexes but not to hash indexes: + +* **`RETURN` is required.** Pass `RETURN` with explicit field names. Without it, Redis loads + the document as a single serialized value, so the individual fields are not available to + the highlighter. `HIGHLIGHT` or `SUMMARIZE` with no `RETURN`, or with `RETURN 0`, fails + with `HIGHLIGHT/SUMMARIZE on JSON indexes requires RETURN with explicit field names`. + +* **Multi-value JSONPaths are rejected.** A path such as `$.tags[*]` or `$..name` fails with + `HIGHLIGHT/SUMMARIZE is not supported for JSON fields with multi-value JSONPath`. Each + value in a multi-value path is indexed separately, with its own byte offsets, so there is + no single value to highlight. The error applies to any field in the returned or + highlighted set, whatever its schema type. + +* **Projection aliases created from raw JSONPath cannot be highlighted.** In a `RETURN` clause such as `RETURN 3 $.name AS alias`, the alias + `alias` is not a schema field, so naming it in `HIGHLIGHT FIELDS` or `SUMMARIZE FIELDS` + fails with ``Property `alias` is not in schema``. Name the schema field explicitly instead of using the alias. + +A single-value JSONPath that resolves to a JSON array or object, such as `$.colors` where +`colors` is an array, is accepted but not highlighted. Redis returns the loaded value +unchanged, without an error. + +Hash indexes keep their existing behavior. `RETURN` is optional, and `HIGHLIGHT` without +`RETURN` highlights every returned `TEXT` field. + +### JSON examples + +Index two fields with single-value JSONPaths and one with a multi-value JSONPath: + +```sql +127.0.0.1:6379> JSON.SET item:1 $ '{"name":"Noise-cancelling Bluetooth headphones","description":"Wireless Bluetooth headphones with noise-cancelling technology","tags":["audio","wireless"]}' +OK +127.0.0.1:6379> FT.CREATE itemIdx ON JSON PREFIX 1 item: SCHEMA $.name AS name TEXT $.description AS description TEXT $.tags[*] AS tags TEXT +OK +``` + +`RETURN` names both fields, and `HIGHLIGHT FIELDS` highlights only `name`: + +```sql +127.0.0.1:6379> FT.SEARCH itemIdx '@name:(bluetooth)' RETURN 2 name description HIGHLIGHT FIELDS 1 name TAGS '' '' +1) "1" +2) "item:1" +3) 1) "name" + 2) "Noise-cancelling Bluetooth headphones" + 3) "description" + 4) "Wireless Bluetooth headphones with noise-cancelling technology" +``` + +Without `RETURN`, the same query fails: + +```sql +127.0.0.1:6379> FT.SEARCH itemIdx '@name:(bluetooth)' HIGHLIGHT +(error) HIGHLIGHT/SUMMARIZE on JSON indexes requires RETURN with explicit field names +``` + +Highlighting `tags`, which uses the multi-value JSONPath `$.tags[*]`, also fails: + +```sql +127.0.0.1:6379> FT.SEARCH itemIdx 'bluetooth' RETURN 1 tags HIGHLIGHT FIELDS 1 tags +(error) HIGHLIGHT/SUMMARIZE is not supported for JSON fields with multi-value JSONPath +``` + +The equivalent hash index needs no `RETURN`: + +```sql +127.0.0.1:6379> HSET item:hash name "Noise-cancelling Bluetooth headphones" +(integer) 1 +127.0.0.1:6379> FT.CREATE hashIdx ON HASH PREFIX 1 item:hash SCHEMA name TEXT +OK +127.0.0.1:6379> FT.SEARCH hashIdx '@name:(bluetooth)' HIGHLIGHT FIELDS 1 name TAGS '' '' +1) "1" +2) "item:hash" +3) 1) "name" + 2) "Noise-cancelling Bluetooth headphones" +``` + +For more about indexing and querying JSON documents, see +[Index and query JSON documents]({{< relref "/develop/ai/search-and-query/indexing/" >}}). diff --git a/content/develop/ai/search-and-query/indexing/_index.md b/content/develop/ai/search-and-query/indexing/_index.md index 8a6cca093b..472cce3c2e 100644 --- a/content/develop/ai/search-and-query/indexing/_index.md +++ b/content/develop/ai/search-and-query/indexing/_index.md @@ -14,6 +14,7 @@ categories: - kubernetes - clients description: How to index and search JSON documents +hideListLinks: true linkTitle: Indexing title: Indexing weight: 3 @@ -592,12 +593,14 @@ This query returns the field as the alias `"stock"` instead of the JSONPath expr You can [highlight]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}}) relevant search terms in any indexed `TEXT` attribute. -For JSON documents, you must use the `RETURN` parameter to specify the attributes, followed by `HIGHLIGHT` to indicate which of those attributes to highlight. +For JSON documents, you must use the `RETURN` parameter to specify the attributes, followed by `HIGHLIGHT` to indicate which of those attributes to highlight. A query with no `RETURN`, or with `RETURN 0`, is rejected. Use the optional `TAGS` keyword to specify the strings that will surround (or highlight) the matching search terms. +JSON support for `HIGHLIGHT` and `SUMMARIZE` was added in the 8.4, 8.6, 8.8, and 8.10 maintenance lines. Redis Open Source 8.2 and earlier reject both options on every JSON index. + {{< note >}} -`HIGHLIGHT` and `SUMMARIZE` are not supported when the JSONPath leads to multiple values (such as arrays indexed as `TEXT`). See [Index limitations](#index-limitations) for details. +`HIGHLIGHT` and `SUMMARIZE` are not supported when the JSONPath leads to multiple values (such as arrays indexed as `TEXT`). A single-value JSONPath that resolves to an array or object is accepted, but the value is returned unhighlighted. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}) for the full rules, error messages, and the exact version in each maintenance line, and [Index limitations](#index-limitations) for other multi-value behavior. {{< /note >}} For example, highlight the word "bluetooth" with bold HTML tags in item names and descriptions: @@ -698,7 +701,7 @@ During index creation, you need to map the JSON elements to `SCHEMA` fields as f When a JSONPath leads to an array or to multiple values: -- No `HIGHLIGHT` and `SUMMARIZE` support. +- No `HIGHLIGHT` and `SUMMARIZE` support. Such a query fails with `HIGHLIGHT/SUMMARIZE is not supported for JSON fields with multi-value JSONPath`. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}). - `SORTBY` only sorts by the first value. - `RETURN` of a schema attribute returns the values as a JSON string. - If a JSONPath is specified by the `RETURN`, instead of a schema attribute, all values are returned (as a JSON string).