Every place that lines a language tag up against a declared locale compares the
two strings with ===. A tag that names a region – nl-NL, en-GB, pt-BR –
therefore matches no declared locale at all, even when the declaration names its
primary subtag. Browsers send exactly such tags, so this is the common case, not
the edge one.
Three instances, all the same root:
compileSort (packages/search-typesense/src/query-compiler.ts) resolves the
request locale with field.locales.indexOf(locale) and, on -1, silently
sorts on locales[0]. A request carrying Accept-Language: nl-NL against a
field declaring ['en', 'nl'] sorts Dutch readers by the English title.
The fallback is quiet, so the result looks ordered and is ordered by the wrong
key.
defaultLanguageOrder (packages/search-api-graphql/src/language.ts) filters
the accepted languages by available.includes(language). A nl-NL request
finds no Dutch label available, so the Dutch value drops out of the requested
block into “the rest”, and [0] – the value a surface displays, and the one
the per-field Content-Language reports – can come back in another language
the document happens to carry. Negotiator does emit the bare nl after
nl-NL for a typical browser header, which softens this one to a wrong
ranking rather than a wrong language, but the ordering it produces is still
not the one the header asked for.
- The projection (
applyText in packages/search/src/project.ts) buckets a
literal into a locale with value.lang === locale. A publisher tagging its
titles nl-NL gets no search or sort key under a declared nl: the
values are displayed and then never matched or sorted on. The existing
normalisation only repairs the separator (pt_BR → pt-BR); it does not
bring a subtagged literal to its declared locale.
Language tags should be matched as BCP-47 language ranges – a declared nl
matches nl, nl-NL and nl-BE, most-specific first – in all three places,
with the exact match still winning where both are present (en and en-GB
declared side by side). und stays the untagged bucket and matches nothing but
untagged. One shared helper is the natural home, since the three today disagree
only by accident.
Found while fixing #792, which addresses a different sort-key defect on the same
path (a document titled in one language had no sort key for the others). That
fix leaves this one intact: it makes every declared locale's key present, but
the request locale is still resolved by exact string.
Reported downstream at https://codeberg.org/limburg/lol/issues/158 – LOL is not
bitten by the first instance only because its locales happen to start with
nl.
Every place that lines a language tag up against a declared locale compares the
two strings with
===. A tag that names a region –nl-NL,en-GB,pt-BR–therefore matches no declared locale at all, even when the declaration names its
primary subtag. Browsers send exactly such tags, so this is the common case, not
the edge one.
Three instances, all the same root:
compileSort(packages/search-typesense/src/query-compiler.ts) resolves therequest locale with
field.locales.indexOf(locale)and, on-1, silentlysorts on
locales[0]. A request carryingAccept-Language: nl-NLagainst afield declaring
['en', 'nl']sorts Dutch readers by the English title.The fallback is quiet, so the result looks ordered and is ordered by the wrong
key.
defaultLanguageOrder(packages/search-api-graphql/src/language.ts) filtersthe accepted languages by
available.includes(language). Anl-NLrequestfinds no Dutch label available, so the Dutch value drops out of the requested
block into “the rest”, and
[0]– the value a surface displays, and the onethe per-field
Content-Languagereports – can come back in another languagethe document happens to carry. Negotiator does emit the bare
nlafternl-NLfor a typical browser header, which softens this one to a wrongranking rather than a wrong language, but the ordering it produces is still
not the one the header asked for.
applyTextinpackages/search/src/project.ts) buckets aliteral into a locale with
value.lang === locale. A publisher tagging itstitles
nl-NLgets no search or sort key under a declarednl: thevalues are displayed and then never matched or sorted on. The existing
normalisation only repairs the separator (
pt_BR→pt-BR); it does notbring a subtagged literal to its declared locale.
Language tags should be matched as BCP-47 language ranges – a declared
nlmatches
nl,nl-NLandnl-BE, most-specific first – in all three places,with the exact match still winning where both are present (
enanden-GBdeclared side by side).
undstays the untagged bucket and matches nothing butuntagged. One shared helper is the natural home, since the three today disagree
only by accident.
Found while fixing #792, which addresses a different sort-key defect on the same
path (a document titled in one language had no sort key for the others). That
fix leaves this one intact: it makes every declared locale's key present, but
the request locale is still resolved by exact string.
Reported downstream at https://codeberg.org/limburg/lol/issues/158 – LOL is not
bitten by the first instance only because its
localeshappen to start withnl.