From 999fc50087225e1ef9e55005828bc27958270802 Mon Sep 17 00:00:00 2001 From: "David W. Dougherty" Date: Tue, 25 Aug 2026 08:23:15 -0700 Subject: [PATCH 1/4] DOC-6994 Document JSON HIGHLIGHT/SUMMARIZE support and limitations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MOD-16530 relaxes the engine's blanket rejection of HIGHLIGHT and SUMMARIZE on JSON indexes. Four surfaces touched the topic and all three of the ones with prose disagreed with each other and with the engine, so the rules now live in one place on the Highlighting page and indexing, ft.search and the administration overview point at it instead of restating. The surprise was which direction the docs were wrong in. indexing/_index.md already described roughly the post-fix behaviour and shipped a worked example, while the engine had rejected every JSON highlight since 2024 — so this change mostly makes the docs true rather than new. Two rules were never documented anywhere, and they are the substance of the addition: JSON requires an explicit RETURN, and a single-value path resolving to an array or object is accepted but silently skipped. Nothing here was executed. The fix is unreleased, so every example output is transcribed from the assertions in RediSearch PR 9410's pytest suite. I left out a SUMMARIZE reply body and the array-skip reply on purpose — their output is dialect-dependent and the tests only assert substring containment, so inventing those two blocks was the one way this page could have shipped fabricated output. Learned: docs claimed post-fix behaviour years before the engine had it; the fix made prose true rather than adding to it Constraint: every FT.SEARCH reply on the Highlighting page is transcribed from RediSearch PR 9410 pytest assertions, not from a running engine — re-derive from the tests rather than hand-editing Directive: do not add SUMMARIZE or array-skip reply bodies to the JSON examples without running a build that has the fix; their output is dialect-dependent Rejected: hand-editing data/commands_redisearch.json and data/commands_core.json | regenerated at release-docs time, so the edit would be silently superseded Rejected: page-level bannerText as the unreleased-feature guard | it banners the whole page, which would misdescribe 900 lines of GA content on ft.search.md — the version statement is the right section-level guard Recheck: the version lines in both TODO comments, once RediSearch backport PR 11046 merges and a maintenance release actually ships Gaps: no engine verification was possible; the vendored tmp/RediSearch checkout predates the fix and contradicts it Ticket: DOC-6994 Co-Authored-By: Claude Opus 5 (1M context) --- content/commands/ft.search.md | 4 +- .../administration/overview.md | 2 + .../advanced-concepts/highlight.md | 89 +++++++++++++++++++ .../ai/search-and-query/indexing/_index.md | 10 ++- 4 files changed, 100 insertions(+), 5 deletions(-) diff --git a/content/commands/ft.search.md b/content/commands/ft.search.md index bf7b653cc9..3dd57eea3d 100644 --- a/content/commands/ft.search.md +++ b/content/commands/ft.search.md @@ -399,13 +399,13 @@ limits the attributes returned from the document. `num` is the number of attribu
SUMMARIZE ... -returns only the sections of the attribute that contain the matched text. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}}) for more information. +returns only the sections of the attribute that contain the matched text. On a JSON index, `SUMMARIZE` requires `RETURN` with explicit attribute names and does not support multi-value JSONPaths. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}}) for more information.
HIGHLIGHT ... -formats occurrences of matched text. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}}) for more information. +formats occurrences of matched text. On a JSON index, `HIGHLIGHT` requires `RETURN` with explicit attribute names and does not support multi-value JSONPaths. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}}) for more information.
diff --git a/content/develop/ai/search-and-query/administration/overview.md b/content/develop/ai/search-and-query/administration/overview.md index 9712a0e920..cda1f5f2e6 100644 --- a/content/develop/ai/search-and-query/administration/overview.md +++ b/content/develop/ai/search-and-query/administration/overview.md @@ -297,6 +297,8 @@ Summarization will fragment the text into smaller sized snippets. Each snippet w Highlighting will highlight the found term and its variants with a user-defined tag. This may be used to display the matched text in a different typeface using a markup language, or to otherwise make the text appear differently. +JSON indexes have additional requirements. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}). + ## Autocomplete Another important feature for Redis Open Source is its autocomplete engine. This allows users to create dictionaries of weighted terms, and then query them for completion suggestions to a given user prefix. Completions can have payloads, which are user-provided pieces of data that can be used for display. For example, completing the names of users, it is possible to add extra metadata about users to be displayed. diff --git a/content/develop/ai/search-and-query/advanced-concepts/highlight.md b/content/develop/ai/search-and-query/advanced-concepts/highlight.md index fb2878cce2..7cb0875b39 100644 --- a/content/develop/ai/search-and-query/advanced-concepts/highlight.md +++ b/content/develop/ai/search-and-query/advanced-concepts/highlight.md @@ -90,3 +90,92 @@ The `RETURN` keyword is treated specially, as it overrides any fields specified In the command `RETURN 1 foo SUMMARIZE FIELDS 1 bar HIGHLIGHT FIELDS 1 baz`, the fields `foo` is returned as-is, while `bar` and `baz` are not returned, because `RETURN` was specified, but did not include those fields. In the command `SUMMARIZE FIELDS 1 bar HIGHLIGHT FIELDS 1 baz`, `bar` is returned summarized and `baz` is returned highlighted. + +## JSON indexes + + + +`HIGHLIGHT` and `SUMMARIZE` work on a JSON index when the field maps to a single-value +[JSONPath]({{< relref "/develop/data-types/json/path" >}}) such as `$.name`. Earlier releases +reject `HIGHLIGHT` and `SUMMARIZE` on every JSON index. + +Three rules apply to JSON indexes but not to hash indexes: + +* **`RETURN` is required.** Pass `RETURN` with explicit field names. Without it, Redis loads + the document as a single serialized value, so the individual fields are not available to + the highlighter. `HIGHLIGHT` or `SUMMARIZE` with no `RETURN`, or with `RETURN 0`, fails + with `HIGHLIGHT/SUMMARIZE on JSON indexes requires RETURN with explicit field names`. + +* **Multi-value JSONPaths are rejected.** A path such as `$.tags[*]` or `$..name` fails with + `HIGHLIGHT/SUMMARIZE is not supported for JSON fields with multi-value JSONPath`. Each + value in a multi-value path is indexed separately, with its own byte offsets, so there is + no single value to highlight. The error applies to any field in the returned or + highlighted set, whatever its schema type. + +* **Raw JSONPath aliases cannot be highlighted.** In `RETURN 3 $.name AS alias`, the alias + `alias` is not a schema field, so naming it in `HIGHLIGHT FIELDS` or `SUMMARIZE FIELDS` + fails with ``Property `alias` is not in schema``. Name the schema field instead. + +A single-value JSONPath that resolves to a JSON array or object, such as `$.colors` where +`colors` is an array, is accepted but not highlighted. Redis returns the loaded value +unchanged, without an error. + +Hash indexes keep their existing behavior. `RETURN` is optional, and `HIGHLIGHT` without +`RETURN` highlights every returned `TEXT` field. + +### JSON examples + +Index two fields with single-value JSONPaths and one with a multi-value JSONPath: + +```sql +127.0.0.1:6379> JSON.SET item:1 $ '{"name":"Noise-cancelling Bluetooth headphones","description":"Wireless Bluetooth headphones with noise-cancelling technology","tags":["audio","wireless"]}' +OK +127.0.0.1:6379> FT.CREATE itemIdx ON JSON PREFIX 1 item: SCHEMA $.name AS name TEXT $.description AS description TEXT $.tags[*] AS tags TEXT +OK +``` + +`RETURN` names both fields, and `HIGHLIGHT FIELDS` highlights only `name`: + +```sql +127.0.0.1:6379> FT.SEARCH itemIdx '@name:(bluetooth)' RETURN 2 name description HIGHLIGHT FIELDS 1 name TAGS '' '' +1) "1" +2) "item:1" +3) 1) "name" + 2) "Noise-cancelling Bluetooth headphones" + 3) "description" + 4) "Wireless Bluetooth headphones with noise-cancelling technology" +``` + +Without `RETURN`, the same query fails: + +```sql +127.0.0.1:6379> FT.SEARCH itemIdx '@name:(bluetooth)' HIGHLIGHT +(error) HIGHLIGHT/SUMMARIZE on JSON indexes requires RETURN with explicit field names +``` + +Highlighting `tags`, which uses the multi-value JSONPath `$.tags[*]`, also fails: + +```sql +127.0.0.1:6379> FT.SEARCH itemIdx 'bluetooth' RETURN 1 tags HIGHLIGHT FIELDS 1 tags +(error) HIGHLIGHT/SUMMARIZE is not supported for JSON fields with multi-value JSONPath +``` + +The equivalent hash index needs no `RETURN`: + +```sql +127.0.0.1:6379> HSET item:hash name "Noise-cancelling Bluetooth headphones" +(integer) 1 +127.0.0.1:6379> FT.CREATE hashIdx ON HASH PREFIX 1 item:hash SCHEMA name TEXT +OK +127.0.0.1:6379> FT.SEARCH hashIdx '@name:(bluetooth)' HIGHLIGHT FIELDS 1 name TAGS '' '' +1) "1" +2) "item:hash" +3) 1) "name" + 2) "Noise-cancelling Bluetooth headphones" +``` + +For more about indexing and querying JSON documents, see +[Index and query JSON documents]({{< relref "/develop/ai/search-and-query/indexing/" >}}). diff --git a/content/develop/ai/search-and-query/indexing/_index.md b/content/develop/ai/search-and-query/indexing/_index.md index 8a6cca093b..f78ad846d5 100644 --- a/content/develop/ai/search-and-query/indexing/_index.md +++ b/content/develop/ai/search-and-query/indexing/_index.md @@ -592,12 +592,16 @@ This query returns the field as the alias `"stock"` instead of the JSONPath expr You can [highlight]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}}) relevant search terms in any indexed `TEXT` attribute. -For JSON documents, you must use the `RETURN` parameter to specify the attributes, followed by `HIGHLIGHT` to indicate which of those attributes to highlight. +For JSON documents, you must use the `RETURN` parameter to specify the attributes, followed by `HIGHLIGHT` to indicate which of those attributes to highlight. A query with no `RETURN`, or with `RETURN 0`, is rejected. Use the optional `TAGS` keyword to specify the strings that will surround (or highlight) the matching search terms. + + {{< note >}} -`HIGHLIGHT` and `SUMMARIZE` are not supported when the JSONPath leads to multiple values (such as arrays indexed as `TEXT`). See [Index limitations](#index-limitations) for details. +`HIGHLIGHT` and `SUMMARIZE` are not supported when the JSONPath leads to multiple values (such as arrays indexed as `TEXT`). A single-value JSONPath that resolves to an array or object is accepted, but the value is returned unhighlighted. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}) for the full rules and error messages, and [Index limitations](#index-limitations) for other multi-value behavior. {{< /note >}} For example, highlight the word "bluetooth" with bold HTML tags in item names and descriptions: @@ -698,7 +702,7 @@ During index creation, you need to map the JSON elements to `SCHEMA` fields as f When a JSONPath leads to an array or to multiple values: -- No `HIGHLIGHT` and `SUMMARIZE` support. +- No `HIGHLIGHT` and `SUMMARIZE` support. Such a query fails with `HIGHLIGHT/SUMMARIZE is not supported for JSON fields with multi-value JSONPath`. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}). - `SORTBY` only sorts by the first value. - `RETURN` of a schema attribute returns the values as a JSON string. - If a JSONPath is specified by the `RETURN`, instead of a schema attribute, all values are returned (as a JSON string). From bb82ca3156a615e0798385152d4455032d396a56 Mon Sep 17 00:00:00 2001 From: David Dougherty Date: Wed, 26 Aug 2026 06:44:42 -0700 Subject: [PATCH 2/4] Update content/develop/ai/search-and-query/advanced-concepts/highlight.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../ai/search-and-query/advanced-concepts/highlight.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/develop/ai/search-and-query/advanced-concepts/highlight.md b/content/develop/ai/search-and-query/advanced-concepts/highlight.md index 7cb0875b39..4eed38f558 100644 --- a/content/develop/ai/search-and-query/advanced-concepts/highlight.md +++ b/content/develop/ai/search-and-query/advanced-concepts/highlight.md @@ -115,9 +115,9 @@ Three rules apply to JSON indexes but not to hash indexes: no single value to highlight. The error applies to any field in the returned or highlighted set, whatever its schema type. -* **Raw JSONPath aliases cannot be highlighted.** In `RETURN 3 $.name AS alias`, the alias +* **Raw JSONPath aliases cannot be highlighted.** In a `RETURN` clause such as`RETURN 3 $.name AS alias`, the alias `alias` is not a schema field, so naming it in `HIGHLIGHT FIELDS` or `SUMMARIZE FIELDS` - fails with ``Property `alias` is not in schema``. Name the schema field instead. + fails with ``Property `alias` is not in schema``. Name the schema field explicitly instead of using the alias. A single-value JSONPath that resolves to a JSON array or object, such as `$.colors` where `colors` is an array, is accepted but not highlighted. Redis returns the loaded value From 47ac77e4fce9048f97102550523764232f7e7002 Mon Sep 17 00:00:00 2001 From: "David W. Dougherty" Date: Wed, 26 Aug 2026 06:49:12 -0700 Subject: [PATCH 3/4] Apply requested changes --- content/develop/ai/search-and-query/indexing/_index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/develop/ai/search-and-query/indexing/_index.md b/content/develop/ai/search-and-query/indexing/_index.md index f78ad846d5..22c91fa538 100644 --- a/content/develop/ai/search-and-query/indexing/_index.md +++ b/content/develop/ai/search-and-query/indexing/_index.md @@ -14,6 +14,7 @@ categories: - kubernetes - clients description: How to index and search JSON documents +hideListLinks: true linkTitle: Indexing title: Indexing weight: 3 From 6e508459bec9bc8e724c2f999b31a7435cb38ca8 Mon Sep 17 00:00:00 2001 From: "David W. Dougherty" Date: Tue, 8 Sep 2026 12:47:02 -0700 Subject: [PATCH 4/4] Apply review comments --- content/commands/ft.search.md | 4 ++-- .../search-and-query/advanced-concepts/highlight.md | 13 ++++++------- .../develop/ai/search-and-query/indexing/_index.md | 6 ++---- 3 files changed, 10 insertions(+), 13 deletions(-) diff --git a/content/commands/ft.search.md b/content/commands/ft.search.md index 3dd57eea3d..10bf62428b 100644 --- a/content/commands/ft.search.md +++ b/content/commands/ft.search.md @@ -399,13 +399,13 @@ limits the attributes returned from the document. `num` is the number of attribu
SUMMARIZE ... -returns only the sections of the attribute that contain the matched text. On a JSON index, `SUMMARIZE` requires `RETURN` with explicit attribute names and does not support multi-value JSONPaths. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}}) for more information. +returns only the sections of the attribute that contain the matched text. On a JSON index, `SUMMARIZE` requires `RETURN` with explicit attribute names and does not support multi-value JSONPaths; JSON support was added in the 8.4 maintenance line (v8.4.14) and the 8.6, 8.8, and 8.10 lines. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}) for the exact versions and the full rules.
HIGHLIGHT ... -formats occurrences of matched text. On a JSON index, `HIGHLIGHT` requires `RETURN` with explicit attribute names and does not support multi-value JSONPaths. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}}) for more information. +formats occurrences of matched text. On a JSON index, `HIGHLIGHT` requires `RETURN` with explicit attribute names and does not support multi-value JSONPaths; JSON support was added in the 8.4 maintenance line (v8.4.14) and the 8.6, 8.8, and 8.10 lines. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}) for the exact versions and the full rules.
diff --git a/content/develop/ai/search-and-query/advanced-concepts/highlight.md b/content/develop/ai/search-and-query/advanced-concepts/highlight.md index 4eed38f558..8f0aba2d9c 100644 --- a/content/develop/ai/search-and-query/advanced-concepts/highlight.md +++ b/content/develop/ai/search-and-query/advanced-concepts/highlight.md @@ -93,13 +93,12 @@ In the command `SUMMARIZE FIELDS 1 bar HIGHLIGHT FIELDS 1 baz`, `bar` is returne ## JSON indexes - - `HIGHLIGHT` and `SUMMARIZE` work on a JSON index when the field maps to a single-value -[JSONPath]({{< relref "/develop/data-types/json/path" >}}) such as `$.name`. Earlier releases +[JSONPath]({{< relref "/develop/data-types/json/path" >}}) such as `$.name`. + +Added in the 8.4 maintenance line (v8.4.14), the 8.6 line (v8.6.10), the 8.8 line (v8.8.1), +the 8.10 line (v8.10.1), and Redis Open Source 8.12. Not available in Redis Open Source 8.2 +or earlier, and not present in the initial 8.4.0, 8.6.0, 8.8.0, or 8.10.0 releases, which reject `HIGHLIGHT` and `SUMMARIZE` on every JSON index. Three rules apply to JSON indexes but not to hash indexes: @@ -115,7 +114,7 @@ Three rules apply to JSON indexes but not to hash indexes: no single value to highlight. The error applies to any field in the returned or highlighted set, whatever its schema type. -* **Raw JSONPath aliases cannot be highlighted.** In a `RETURN` clause such as`RETURN 3 $.name AS alias`, the alias +* **Projection aliases created from raw JSONPath cannot be highlighted.** In a `RETURN` clause such as `RETURN 3 $.name AS alias`, the alias `alias` is not a schema field, so naming it in `HIGHLIGHT FIELDS` or `SUMMARIZE FIELDS` fails with ``Property `alias` is not in schema``. Name the schema field explicitly instead of using the alias. diff --git a/content/develop/ai/search-and-query/indexing/_index.md b/content/develop/ai/search-and-query/indexing/_index.md index 22c91fa538..472cce3c2e 100644 --- a/content/develop/ai/search-and-query/indexing/_index.md +++ b/content/develop/ai/search-and-query/indexing/_index.md @@ -597,12 +597,10 @@ For JSON documents, you must use the `RETURN` parameter to specify the attribute Use the optional `TAGS` keyword to specify the strings that will surround (or highlight) the matching search terms. - +JSON support for `HIGHLIGHT` and `SUMMARIZE` was added in the 8.4, 8.6, 8.8, and 8.10 maintenance lines. Redis Open Source 8.2 and earlier reject both options on every JSON index. {{< note >}} -`HIGHLIGHT` and `SUMMARIZE` are not supported when the JSONPath leads to multiple values (such as arrays indexed as `TEXT`). A single-value JSONPath that resolves to an array or object is accepted, but the value is returned unhighlighted. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}) for the full rules and error messages, and [Index limitations](#index-limitations) for other multi-value behavior. +`HIGHLIGHT` and `SUMMARIZE` are not supported when the JSONPath leads to multiple values (such as arrays indexed as `TEXT`). A single-value JSONPath that resolves to an array or object is accepted, but the value is returned unhighlighted. See [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight#json-indexes" >}}) for the full rules, error messages, and the exact version in each maintenance line, and [Index limitations](#index-limitations) for other multi-value behavior. {{< /note >}} For example, highlight the word "bluetooth" with bold HTML tags in item names and descriptions: