FT.SEARCH reply parser crashes on a nil field-array (TypeError: 'NoneType' object is not subscriptable)

[vishal-bala]

Summary

Executor's FT.SEARCH reply parsing assumes every per-document field-array in the reply is a list and slices it directly. When one document's field-array comes back as None, dict(zip(row_data[::2], row_data[1::2])) raises TypeError: 'NoneType' object is not subscriptable and the entire query fails.

Where

sql_redis/executor.py, the standard (non-WITHSCORES) FT.SEARCH branch:

# Standard format: [count, key1, [fields1], key2, [fields2], ...]
for i in range(2, len(raw_result), 2):
    row_data = raw_result[i]
    row = dict(zip(row_data[::2], row_data[1::2]))  # row_data is None -> TypeError

The same unguarded pattern is in every reply branch — sync and async, across WITHSCORES / standard / FT.AGGREGATE (≈ lines 256, 271, 276 and the async mirror ≈ 382, 397, 402 on main, v0.6.2). Any of them crashes if its row_data is nil.

Reproduction

The crash is deterministic given the reply shape — a FT.SEARCH reply where one document's field-array is None instead of a [field, value, …] list:

# Mirrors sql_redis/executor.py, standard FT.SEARCH branch.
#   reply layout: [count, key1, <fields1>, key2, <fields2>, ...]
#   here <fields1> came back nil:
raw_result = [1, "doc:1", None]

for i in range(2, len(raw_result), 2):
    row_data = raw_result[i]                        # None
    row = dict(zip(row_data[::2], row_data[1::2]))  # TypeError: 'NoneType' object is not subscriptable

How we hit it

We use sql-redis in redis-vl-python (a Redis vector/search library), whose integration tests run SQL SELECTs through Executor.execute against several Redis server versions. The crash appears:

• Only against the redis:latest Docker image (currently Redis 8.8.0); never on pinned redis:8.2 / 8.4.
• Across redis-py 5.x / 6.x / 7.x.
• On different SELECT queries from run to run — i.e. data-independent, which points at the reply shape rather than any specific query.

Confirmed vs. unconfirmed

• Confirmed: the parser isn't defensive against a nil field-array — the proximate cause of the exception.
• Unconfirmed: why a slot comes back nil. We could not reproduce it directly, even with matched versions (Redis 8.8.0, redis-py 7.4.0, sql-redis 0.6.x) and high concurrency. Leading guesses: a RESP2/RESP3 reply-shape difference negotiated with the newer server/client, or a document expiring between FT.SEARCH returning ids and field materialization. We are not claiming a Redis 8.8.0 regression — it doesn't reproduce on 8.8.0.

Suggested fix

Tolerate a nil/empty field-array at each parse site, turning an intermittent hard crash into a well-defined result regardless of the underlying trigger:

row_data = raw_result[i]
if not row_data:
    continue  # or rows.append({})
row = dict(zip(row_data[::2], row_data[1::2]))

If the nil turns out to originate from a RESP3 reply layout, that path may warrant dedicated handling — happy to help probe HELLO 3 against 8.8.0.

Environment

• sql-redis 0.6.0–0.6.2 (pattern present on main)
• redis-py 5.x / 6.x / 7.x (observed on 7.4.0)
• Redis server: 8.8.0 (redis:latest); not seen on 8.2 / 8.4
• Python 3.11–3.14

[vishal-bala]

Root cause identified (and confirmed with the Redis search team)

Following up with what we found while investigating this.

The nil is emitted by the server, on the wire, under RESP2. Reading the FT.SEARCH reply bytes directly (bypassing redis-py so it cannot be a client-side decoding artifact), a matched document comes back as its key bulk string immediately followed by $-1, a nil bulk string, in the slot where a field-array (*N) is required:

...$19\r\nrepro_nil_idx:doc:0\r\n$-1\r\n...

redis-py decodes that $-1 to None, which is the NoneType the parser then sliced.

The trigger is multithreaded background query execution, not the Redis version per se. In 8.8 the default number of background search worker threads was raised above zero (aligning OSS Redis with similar projects, which runs searches on background executors by default); in 8.6 and earlier it defaulted to zero. When a query runs on a background worker and a matching document's key or indexed field expires (TTL) while the query is in flight, the document id can still surface in the results but with its fields slot set to nil.

This explains the version pattern in the original report. An identical probe reproduced readily on 8.8.0 and never on 8.2 / 8.4 / 8.6, but that is because of the worker-thread default, not the version: the search team reproduced the same nil on 8.6 once background workers were enabled. 8.8 simply turns them on out of the box, which is why it appears with the stock redis:latest image and no special configuration. Hash-field expiry (HPEXPIRE) is the most reliable trigger, especially on SORTABLE fields, though whole-key TTL hits it too.

Two parts, judged differently upstream. That a document expiring mid-query can still surface in results is, to an extent, expected and documented behavior (see the FT.SEARCH RETURN docs). Returning a nil in the field-array slot specifically is treated as a bug on the Redis side, and a server-side fix is in progress.

Why we still fix it here. Regardless of the upstream outcome, the client should not turn this reply shape into a hard crash that aborts the whole query. Tolerating a nil/empty field-array is correct both as a guard against the current bug and as graceful handling of the legitimately expected "the document just expired" case. That is what #39 does, normalizing the field-array to an empty field set at every parse site (sync and async, across WITHSCORES / standard / FT.AGGREGATE).

[github-actions]

🚀 Issue was released in v0.7.1 🚀

[github-actions]

🚀 Issue was released in v0.7.1 🚀