feat(executor): parse RESP3 map replies for FT.SEARCH and FT.AGGREGATE

[vishal-bala]

Motivation

sql-redis raises KeyError: 2 on any FT.SEARCH or FT.AGGREGATE query over a RESP3 connection. The executor sends commands with client.execute_command and reads the raw reply, and redis-py registers no response callback for either command on the base client: Search.search() calls execute_command and then its own _parse_results. So the executor receives the wire shape verbatim, a flat array on RESP2 and a map on RESP3, and its parser assumed the array.

The failure needs no opt-in: redis-py 8 defaults to RESP3, so a caller who upgrades redis-py hits it without ever passing protocol. Measured on Redis 8.4.6 with redis-py 8.1.0, every RESP3 client mode fails: protocol unset, protocol=3 and legacy_responses=False. Only an explicit protocol=2 works, which is why RedisVL pins its factory-created clients to it (redis/redis-vl-python#695). This PR removes the need for that pin.

Changes

Shape dispatch, with a RESP3 map folded into the array shape

_parse_reply dispatches on the shape of the reply rather than on the protocol the connection negotiated. A map goes through _fold_map_to_array, which rewrites it into the flat array RESP2 sends, and both shapes then reach the same row parser, _parse_array_reply.

Folding rather than writing a second parser keeps the four existing branch behaviours in one place: score-alias resolution over the union of every row's field names, the collision loop, nil field-array tolerance, and count extraction. _parse_array_reply is the pre-existing parser moved unchanged apart from one dead assignment, which is what keeps protocol=2 output identical. The approach mirrors RedisVL's own _parse_batch_search_result in redis/redis-vl-python#695.

# RESP3 FT.SEARCH reply, as Redis 8.4.6 sends it
{"total_results": 2, "attributes": [], "format": "STRING", "warning": [],
 "results": [{"id": "vp:5", "extra_attributes": {"title": "redis in action"}, "values": []}, ...]}

# folded into the RESP2 layout the parser already understands
[2, "vp:5", ["title", "redis in action"], ...]

Shape dispatch was chosen over two alternatives. Routing through client.ft(index) fixes nothing across the declared redis>=5.0.0 range: on redis-py 7.4.0 and earlier _parse_results returns the raw map under RESP3, and redis-py 8.1.0's protocol=3 callback table has no FT.AGGREGATE entry. Reading connection_kwargs["protocol"] is wrong for cluster and sentinel clients and cannot tell redis-py 8's four callback modes apart.

Layout derived once, unrecognised replies rejected

The parser strides over the array, so the fold and the parser must agree on which slots each document occupies. A frozen _ReplyLayout is derived once per query and passed to both, so they cannot drift apart.

A reply that is neither shape now raises a ValueError naming what arrived, instead of parsing into empty rows. A map without a results key (a cluster node-keyed reply, an FT.PROFILE reply) and a non-array non-map would otherwise look like an empty result set.

Bytes structural keys

_map_get looks up structural keys (total_results, results, id, score, extra_attributes) as str and then as bytes, because they arrive as bytes when decode_responses=False. Reading only str keys returns zero rows without raising, which is the bug redis-py shipped in 8.0.0 (redis/redis-py#4107). Document field keys are passed through untouched, as before.

Executor structure

_ScoreParseMixin is removed. Every method was a @staticmethod called through the explicit class name, and only two of eight concerned scores, so the reply parsers are now module-level functions shared by Executor and AsyncExecutor. Both execute() methods lose their duplicated 65-line parse block.

Tests and CI

• tests/test_reply_shapes.py: canned replies through both executors, every literal captured from Redis 8.4.6 rather than written by hand.
• tests/test_protocol_matrix.py: the same behaviour against a real server across all four client modes, plus a canary asserting each protocol really sends the shape assumed.
• A redis-py-compat CI job runs the tests marked protocol against redis-py 6.3.0, 7.4.0 and 8.1.0, after checking the overlay installed the requested version.
• The stub executor factories move into tests/conftest.py, and tests/test_nil_field_array.py uses them.
• Docs: docs/concepts/result-shape.md covers the protocol-dependent values, and docs/for-ais-only/FAILURE_MODES.md records why dispatch is by shape.

Notes

The score's type differs by protocol and is deliberately not normalised. RESP3 sends a double, so callers get a float; RESP2 sends text, so they get a str or bytes. Stringifying the double cannot reproduce the text Redis would have sent on RESP2, so normalising would produce a third form rather than parity. float() is the protocol-safe conversion and works on both.

count for an FT.AGGREGATE without a GROUP BY also differs by protocol. Measured on Redis 8.4.6 over 5 documents, SELECT price * 2 AS dbl reports count=1 on RESP2, a placeholder the Redis docs call "not a valid value", and count=5 on RESP3. With a GROUP BY the two agree. Neither value can be derived from the other without changing protocol=2 output, so both are reported as Redis sent them, and tests pin the difference so it is not later "fixed" by deriving count from the row count.

The dependency range stays at redis>=5.0.0. RESP3 has been reachable through protocol=3 since redis-py 5.0, so the fix is needed at the floor too. RedisVL excludes redis-py 8.0.0 because of redis/redis-py#4107, but that defect lives in Result.from_resp3, which sql-redis never calls.

The evidence for the RESP3 reply layout comes from one server version. Redis 8.4.6 standalone was measured across redis-py 6.3.0, 7.4.0 and 8.1.0 and five client modes, sync and async, and rows matched the protocol=2 baseline in all 30 cells. No cluster deployment was exercised, and redis.io documents only the top-level keys of the RESP3 map, not the per-document layout the fold relies on.

Next Steps

• Surface the RESP3 warning key. On a query timeout RediSearch returns partial results with a warning, and the fold discards it, so a truncated result set looks complete. RESP2 has no equivalent channel, which is why it was left out of this change.
• Deduplicate the two execute() bodies, which are still near-identical above the _parse_reply call. Both error re-wrapping paths are now tested, so the refactor has a safety net.

Release Notes

sql-redis now works over RESP3 connections, including redis-py 8's default. On RESP3, score() columns are returned as float rather than text, and count for an aggregation without GROUP BY reports the real total where RESP2 reports 1.