A welded co-element filter hangs Typesense when the nested fields hold arrays

[ddeboer]

The co-element filter #771 was built for – this agent in this role – hangs Typesense 30.2 indefinitely whenever the nested fields hold arrays, which is every document a qualified relation produces in practice.

Reproducer

One document, typesense/typesense:30.2, no LDE code in the path:

curl -XPOST 'http://localhost:8108/collections' -H 'X-TYPESENSE-API-KEY: dev' -d '{
  "name": "weld", "enable_nested_fields": true,
  "fields": [ {"name": "credits", "type": "object[]"},
              {"name": "credits.role", "type": "string[]"},
              {"name": "credits.agent_id", "type": "string[]"} ] }'

# array-valued nested fields
curl -XPOST 'http://localhost:8108/collections/weld/documents' -H 'X-TYPESENSE-API-KEY: dev' \
  -d '{"id": "w1", "credits": [{"role": ["etser"], "agent_id": ["p1"]}]}'

curl -G 'http://localhost:8108/collections/weld/documents/search' -H 'X-TYPESENSE-API-KEY: dev' \
  --data-urlencode 'q=*' --data-urlencode 'query_by=' \
  --data-urlencode 'filter_by=credits.{role:=etser && agent_id:=p1}'

The search never returns. The identical document with scalar nested values – {"role": "etser", "agent_id": "p1"} – answers 1 hit instantly, same schema and same filter.

What we narrowed it to

• It is a hang, not a slow scan: 0.4 % CPU throughout, inflight_searches: 0 in the log, no response in 300 s. Other queries keep being served normally.
• The trigger is the array inside each object, not the syntax, the field types or the values. These all answer in milliseconds: one condition inside the braces (either field); the two conditions joined outside the braces (credits.role:=etser && credits.agent_id:=p1); two string[] conditions welded over scalar-valued objects; and Typesense’s own documented example, ingredients.{name:=cheese && concentration:<50}, whose objects also hold scalars.
• IRIs, backquoting and the partial-match operator make no difference.

The docs’ example holding scalars is presumably why the design work behind #771 cleared the query side against a live 30.2 container.

Why it reaches every real document

A qualified relation declares its edge fields array: true for good reasons – a role may be stated once per language, and the endpoint is multi-valued – so the projected entries hold arrays and the welded filter is unusable across the whole collection. Measured on a real index: creator.role:=fotograaf answers 46 works in 0.0 s, creator.creator_id:=<iri> answers 22 works in 0.0 s, and creator.{role:=fotograaf && creator_id:=<iri>} never returns. Through the GraphQL API that surfaces as Unexpected error, the 5 s Typesense timeout retried once.

So the data is fine – roles, agents and the resolved lookup are all indexed and readable, and either condition alone filters – but the weld itself, the one thing a nested edge buys over two flat fields, cannot be asked. Unwelded, role: fotograaf && creator: X still matches a work where X is the publisher and somebody else is the photographer.

Suggestion

The fix looks like Typesense’s: the syntax is documented and the input valid. What seems worth doing here is a note in #771’s documentation on which value shapes the capability can serve today, and – if it is worth dodging until Typesense fixes it – deciding whether a single-valued edge field should project a scalar rather than a one-element array.

Downstream context: https://codeberg.org/limburg/lol/issues/162.

[ddeboer]

Two follow-ups from tracing which queries can actually reach this, both measured on a fresh typesense/typesense:30.2.

The dotted-path explanation in compileWelded does not hold

query-compiler.js states it as a correctness requirement:

Typesense 30.2 hangs on a dotted path inside a group (creator.{creator.name:=…} never returns – no error, no result), so the compiler must be structurally incapable of emitting one.

The hang is real, but the dotted path is not what causes it. On a document shaped like a projected edge, with creator.creator a nested object[]:

filter_by	result
creator.{creator.name_und:=[Leufkens]} – dotted path inside the group	1 hit, 0.0 s
creator.{role:=[fotograaf] && creator_id:=[p1]} – two leaf conditions	hangs

So a dotted path inside a group answers fine, and the leaf-only form the compiler guarantees is the one that hangs. The trigger is what the first comment describes: two &&-joined conditions inside one group where the nested fields hold arrays. Whatever the dotted-path run showed at the time, keeping conditions leaf-only does not avoid this, and the stated guarantee is load-bearing in a way it cannot deliver.

Only a multi-criterion where reaches it

Worth recording, because it bounds the blast radius. compileMembership renders a multi-value in as one condition (role:=[a,b]), so the braces hold two conditions only when a caller puts two criteria inside one edge’s where. Everything else a consumer can send today is fine:

what the caller writes	emitted	result
creator: { in: [iri] }	creator_id:=[p1]	1 hit, 0.0 s
creator: { where: { creator: { in: [iri] } } }	creator.{creator_id:=[p1]}	1 hit, 0.0 s
creator: { where: { role: { in: ["fotograaf"] } } }	creator.{role:=[fotograaf]}	1 hit, 0.0 s
creator: { where: { role: { in: ["fotograaf", "uitgever"] } } }	creator.{role:=[fotograaf,uitgever]}	1 hit, 0.0 s
creator: { where: { creator: {…}, role: {…} } }	creator.{role:=… && creator_id:=…}	hangs

So the exposed surface is safe apart from the one query the feature exists for – which is also the example in its own documentation.