Fix non-Latin record titles collapsing to identical untitled slug - #235
Fix non-Latin record titles collapsing to identical untitled slug#235grimicorn-agent wants to merge 2 commits into
Conversation
Normalize titles with NFKD and drop combining marks before slugifying so accented Latin folds to ASCII (café -> cafe) and non-Latin scripts strip cleanly to the FALLBACK_SLUG. Operation order fixed so compatibility decompositions that emit ASCII (№ -> No) survive. Closes #229
Independent code review trailReviewer: fresh Claude (Opus), did not write the code. 3 rounds. Round 1 — flagged:
Round 2 — flagged:
Round 3 — flagged, all declined with reasons:
Unresolved after 3 rounds: none blocking. Two declined items captured as follow-up suggestions in the PR body. |
Merge maintenance + review trailMerged Independent review (Opus)Round 1 flagged code-standards clean (small function, full names, no nesting). Behavioral findings, all skipped with reasons:
No correctness/security/standards issues to fix in the diff, so no code changes were made and no further rounds were needed. |
What & why
titleToSluginserver/utils/markdown.tsran.toLowerCase().replace(/[^a-z0-9\s-]/g, ""), which deleted every non-ASCII letter. Any title written in a non-Latin script (Cyrillic, CJK, Arabic) therefore produced an identical empty base and collapsed ontountitled, forcinguntitled-1/untitled-2disambiguation, and accented Latin titles were mangled (Café→caf).This normalizes the title with Unicode NFKD and drops combining marks before slugifying:
Café Meeting→cafe-meeting,Zürich Naïve→zurich-naive.FALLBACK_SLUGuntitled— the fallback is applied deliberately and downstreamresolveUniqueFilePathhandles uniqueness (per the issue's acceptance criteria).U+00A0, ideographic spaceU+3000) NFKD-normalizes to a plain space, so it still acts as a word separator instead of fusing words.Decisions
FALLBACK_SLUG, not for transliterated slugs. NFKD achieves that while also improving the accented-Latin case, with no new dependency.№→No→o). Normalizing first preserves it (№5 Meeting→no5-meeting).COMBINING_MARKS_PATTERN,NON_SLUG_CHAR_PATTERN) replace the inline literal for clarity.Tests
Added
titleToSlugcases intests/server/utils/markdown.test.ts: non-Latin-script titles →untitled, accented-Latin folding, compatibility-decomposition ASCII survival, and non-ASCII whitespace as a separator. Full suite: 1753 passing.lint:fix,lint:ciclean.Viewable
Server-side slug utility; exercised on ingest via
parseWebhookPayload/parseEmailPayload(webhook + email routes). Behavior is covered bytests/server/utils/markdown.test.ts.Closes #229
Follow-up suggestions
Fold stroke/ligature Latin letters— letters with no NFKD decomposition (ß,ø,ł,æ,þ,đ) are stripped rather than folded, soStraße→strae; an explicit fold map would produce readable slugs (suggested: P3, effort: S, evidence: server/utils/markdown.ts titleToSlug)Bound title length before NFKD normalization— NFKD can expand a string up to ~18x (e.g.U+FDFA), andtitleToSlugnormalizes the full untruncated title; pre-slicing would cap the transient allocation (suggested: P4, effort: S, evidence: server/utils/markdown.ts titleToSlug)