Localization QA that produces evidence, not alarms.
It finds text overflow, clipped buttons, silently truncated copy, RTL breakage and untranslated strings in multilingual web UIs — and then proves in the browser that the translation caused them.
The premise: an AI wandering around a page looking for bugs should not be the main engine. The browser should produce measurable evidence. The agent should handle state-space discovery and the genuinely ambiguous cases. And the model's important calls should be re-tested experimentally in the browser.
So a finding is not "AI looked and thinks something might be wrong here." It is:
in which UI state, because of which locale change, which layout invariant broke — and that the problem disappears when only that variable is reverted.
Chrome / CDP
→ UI state discovery (build the state graph, not a blind crawl)
→ cross-locale state replay (same screen in every language, or fail loudly)
→ semantic component matching (EN "Continue" ≡ DE "Weiter")
→ deterministic invariants (measured, no model involved)
→ locale-delta attribution (is this new, or was it already broken?)
→ counterfactual verification (revert only the string, re-measure)
→ AI adjudication (only what measurement cannot settle)
→ actionable finding
npm install
npm run buildA demo app with five deliberately seeded localization bugs ships with the repo:
node fixtures/app.js & # http://127.0.0.1:4173/billing (en / de / tr / ar)
node dist/cli.js scan # → .lqa/report.htmlExpected: 10 findings, 8 of them counterfactually proven — including the two
a naive checker cannot see (a line-clamp truncation that reports no overflow,
and a button clipped by a sibling's growth rather than its own). The remaining
two are the Arabic RTL and untranslated-string findings, which are not
string-length dependent and so are attributed without a counterfactual.
Point it at your own app:
node dist/cli.js scan --no-config \
--entry https://your-app.example.com/en/ --locale path \
--source en --locales de,tr,ar \
--depth 0 --out /tmp/scan--depth 0 navigates and measures without clicking anything — the right
starting point against a site you do not own.
A route is not a test unit. /billing on its own means little. The same
route has default, empty, validation-error, modal-open, dropdown-open, loading,
toast and paginated variants, and a localization bug may exist in exactly one of
them. So the first pass does not look for bugs at all — it builds the
application's testable state graph.
Screenshots of two different states are not a comparison. If the German run
took a different path than the English run, any visual diff between them is
noise. States are addressed by a text-free fingerprint and replayed by
semantic component identity, so en / de / ar land on provably the same
screen — or the comparison is skipped and reported, never faked.
scrollWidth > clientWidth misses the most common real bug. See below.
A component with line-clamp: 2; overflow: hidden is perfectly valid CSS. It
reports no overflow — scrollHeight === clientHeight. If English needs one
line and German naturally needs three, the third line silently disappears and
every naive check passes.
So alongside current geometry, each text element is measured unconstrained:
cloned into an offscreen sandbox with its computed style copied and its
constraints removed (max-*, min-*, overflow, line-clamp,
text-overflow), then measured twice — once at max-content for intrinsic
width, once at the element's real content width for unclamped height and line
count.
Real output from the bundled fixture:
card-desc scroll=300x42 == client=300x42 ← no overflow reported anywhere
clamp=2 natural={naturalLines:4} ← two lines are being eaten
The absolute value of a measurement is weak evidence — plenty of UIs ship with a permanently tight button. The change between source and target locale, under identical DOM and CSS, is strong:
EN: button width 120px, overlap 0
DE: button width 194px, overlap 28px
Violations that already exist in the source locale are subtracted; they are
pre-existing layout bugs, not localization regressions. Invariants that cannot
exist in the source locale (RTL correctness, untranslated strings) skip the
growth heuristic entirely — "the Arabic string is narrower than English" says
nothing about whether text-align: left is wrong.
This is what makes a finding defensible:
- locate the component by semantic key
- record its geometry with the target-locale string in place
- swap only that string back to the source value — identical DOM, identical CSS, identical state
- force layout, re-measure
- restore
If the violation disappears, this is no longer correlation: under the same DOM and CSS, the bug appears only when the localized string is present.
Group scope. A single-string revert is not always enough, and treating it as decisive produces false "pre-existing" verdicts. When a wide German primary CTA starves the secondary action, reverting the secondary string changes nothing — its sibling is the cause. A failed single revert escalates to a group counterfactual over every localized leaf string in the responsible container:
text-clipped-by-ancestor cta-secondary en→de
single revert → violation persists
group revert (cta-primary, cta-secondary) → clears
de: clipEscape 40.14px en: clipEscape 0
→ the cause is collective; cta-primary is the dominant contributor
Three genuinely different verdicts come out of this: this translation broke it, these translations collectively broke it, and the layout was already broken.
Each invariant is re-tested on its own terms — a page-scroll finding re-measures the document, an overlap finding recomputes the collision. Using "the element got narrower" as a universal proxy would let a page that still scrolls horizontally be reported as proven.
The model is the smallest and last stage. It only sees findings the deterministic engine could not settle (a configurable confidence band). Linguistic questions always reach it, since geometry can never decide them.
It never receives just "source string / translated string". It receives the component's role, its UI state, nearby text, both screenshots, full geometry, the cross-locale delta, your glossary and style guide, and the counterfactual result — the same evidence a human reviewer would need.
It answers what measurement cannot:
- the geometry is fine but the phrasing is wrong for this context or register
- nothing is clipped, but a CTA wrapping to three lines wrecks visual hierarchy
- the measurement is technically true and has no user impact
A real verdict from a live run, where the engine flagged a nav link on geometry:
"Geometry is a false positive (80.3×21px matches the source string exactly in the counterfactual, so the 21px height is baseline nav styling), but the string itself is wrong: 'Solutions' is rendered 'Solucion' — missing the required accent and in the singular."
It rejected the measurement and found a spelling error instead. Neither the invariants nor the counterfactual could have done that.
The verdict adjusts confidence and severity; it can never invent a finding. If no model is available the run degrades to deterministic + counterfactual with nothing silently dropped.
By default the judge shells out to a locally installed CLI you are already logged into, so your existing subscription is reused:
node dist/cli.js doctor # which backends are installed
node dist/cli.js doctor --probe # which ones actually answer right now| Preset | Binary | Images |
|---|---|---|
claude-code |
claude |
via read-only file tool |
codex |
codex |
native -i, plus --output-schema |
grok |
grok |
off by default |
agy |
agy |
off by default |
opencode |
opencode |
native -f |
custom |
anything | {{PROMPT}} {{SYSTEM}} {{MODEL}} {{SCHEMA_FILE}} {{IMAGE_ARGS}} |
api |
Anthropic API | base64 inline |
Resolution is auto → local CLI if present → API key if set → run without a
judge. Force it with --judge cli|api and --judge-cli <preset>.
Because CLI backends give no structured-output guarantee and each wraps its
answer differently, responses are located rather than assumed: the parser
searches for the object carrying the requested fields, descending into transport
envelopes, and out-of-vocabulary severities (high, critical) are normalized
before they can reach the report.
Real translations only tell you about the strings you already have. Stress profiles tell you which components will break the next time a locale is added:
node dist/cli.js profiles
node dist/cli.js scan --pseudo| Profile | Models |
|---|---|
expand-1.3x / expand-1.6x |
typical and worst-case expansion (de/fr/fi/hu) |
accents |
diacritics and non-ASCII glyph metrics |
longtoken |
unbreakable compound words |
rtl |
right-to-left rendering and mirroring |
mixed-bidi |
mixed RTL/LTR runs without isolation |
Every stage's input and output can be written to disk:
node dist/cli.js scan --trace --out /tmp/scan
ls /tmp/scan/trace/00-config.json what you asked for
10-…en-snapshot.json everything the browser measured in the source locale
11-…en-violations.json source-locale violations
20-…de-snapshot.json target-locale measurement
21-…de-violations-raw.json raw invariant hits
22-…carried-from-source.json dropped: already broken in the source
23-…locale-attributed.json survivors, with attribution reasons
24-…component-matches.json cross-locale component pairs
30-…counterfactual-*.json the browser experiment, before/after numbers
40-…verdict-*.json how the three signals combined
judge-NNN-*.txt the exact prompt sent and the raw reply received
The judge trace contains the full system prompt, the full evidence block, the image paths, and the backend's unmodified response — including token counts and cost. A finding can be re-derived from these files without re-running anything.
Running every state × every locale on every pull request is waste. A full scan
writes a string index; on a PR, lqa impact selects only what a change can
affect, expanding downstream through the state graph.
# main branch, nightly / pre-release
node dist/cli.js scan --pseudo
# pull request
node dist/cli.js impact --catalog-before base/en.json --catalog-after head/en.json \
--changed-routes /billing,/checkout
node dist/cli.js scan --states S-a1b2,S-c3d4 --locales de,ar --fail-on majorCatalog diffs handle flat or nested JSON, so Crowdin / Phrase / Lokalise exports drop straight in. When impact analysis cannot narrow anything it says so and falls back to a full scan rather than silently skipping coverage.
A ready-to-adapt workflow is in .github/workflows/localization-qa.yml.
Reuse your actual session. The reason for talking to Chrome directly rather than through a driver abstraction is that cookies, localStorage, feature flags and authenticated state are exactly what you see:
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
--remote-debugging-port=9222 --user-data-dir=/tmp/lqa-chrome
# log in normally, then:
node dist/cli.js scan --attach 127.0.0.1:9222 --entry https://app.example.com/dashboardOr launch a dedicated Chrome against a persistent profile so one login is reused
by every later run, or run a loginScript once per run. Concurrent runs against
the same profile are safe — the profile is claimed with an atomic lock and a
losing run falls back to a temporary profile with a warning.
Tell it how locale is selected:
--locale path # /de/billing
--locale query:lang # ?lang=de
--locale cookie:NEXT_LOCALE
--locale localStorage:i18nextLng
--locale header:X-LocaleAccept-Language and Emulation.setLocaleOverride are always applied too. For
anything stranger, locale: { kind: 'custom', script: '…' } runs your own init
script with LOCALE in scope.
Give it stable identity where it matters. Semantic keys are derived without
help, but a data-testid on components you care about makes cross-locale
matching exact and keeps finding IDs stable across refactors.
One pattern to avoid: putting the locale inside element IDs (
id="menu-header-english"/id="menu-header-turkish"). Those components can never be matched across locales — not by this tool, and not by any visual-diff or snapshot tool either.
| Invariant | Catches |
|---|---|
text-clipped-by-ancestor |
painted text escaping its clipping ancestor |
content-overflows-container |
content larger than its box, with overflow hidden |
line-clamp-truncation |
content lost to line-clamp or a fixed height |
ellipsis-truncation |
single-line text cut by text-overflow: ellipsis |
element-outside-viewport |
element pushed past the viewport edge |
page-horizontal-scroll |
a horizontal scrollbar appearing |
sibling-overlap |
colliding siblings, after filtering intentional overlays |
hit-target-too-small |
controls below the minimum touch target |
rtl-direction-mismatch |
RTL text rendered in an LTR context |
rtl-physical-text-align |
left/right where start/end belongs |
rtl-bidi-isolation-missing |
numbers beside Latin tokens inside RTL text |
unlocalized-string |
byte-identical to source with no target-script characters |
Several classes of false positive are filtered before they reach you: carousel
slides parked outside their track, overlays covering the page behind them
(while still comparing elements inside the same modal), inline boxes whose
clientWidth is zero by spec, and mixed-script runs the Unicode Bidi Algorithm
resolves correctly on its own.
| Command | Purpose |
|---|---|
scan |
full run |
discover |
build and save the state graph only |
impact |
CI: select affected states/locales |
doctor [--probe] |
list judge backends; probe them for real |
profiles |
list pseudo-locale stress profiles |
init |
write a starter config |
Common flags:
--no-config ignore config files; run purely from flags
--entry <url> --locale <kind[:key]> --source / --locales
--depth <n> 0 = navigate only, never click
--actions <n> actionables expanded per state
--max-states <n> --max-elements <n>
--attach <host:port> --headed
--judge auto|cli|api --judge-cli <preset> --judge-max-calls <n>
--no-judge --no-counterfactual --pseudo
--trace --fail-on <severity> --log debug
report.html— findings as cause / mechanism / effect / evidence / fix, with before-and-after screenshots, measurements and the reproduction pathreport.json— the full structured run, including the state graphreport.sarif— GitHub code scanningstate-graph.json,string-index.json— reusable across runstrace/— with--trace, every stage's I/O
Console output for one finding:
MAJOR line-clamp-truncation · /billing [default] en→de
COMPONENT data-testid=card-desc/ (generic)
CAUSE de string "Ihre hinterlegte Kreditkarte…" requires 940px intrinsic
width vs 470px for en "Your card will be charged…" (+99.8%) and
needs 2 more line(s) at the same container width
MECHANISM content wants 940px but the box provides 300px
MECHANISM -webkit-line-clamp:2 caps visible lines and hides the remainder
EFFECT the string needs 4 lines at the current width — 2 line(s) hidden
EVIDENCE counterfactual PROVEN (single) — reverting only this string, same
DOM and CSS, removes the bug
FIX raise or drop `-webkit-line-clamp` for locales that need more lines
FIX shorten the source string so translations have expansion headroom
Coverage is never silently truncated. Every cap the run hits — state limits,
per-state action budgets, unreachable replays, collapsed duplicates, states that
failed to reproduce — is reported in stats.truncations and printed as a
coverage note. A report that looks clean because it stopped early is worse than
no report.
"0 findings" means "nothing found in what was compared." The report tells you what it could not compare, and you should read both numbers together.
Failure modes are explicit. No model available → deterministic + counterfactual, nothing dropped. Replay lands elsewhere → skipped and reported, not compared. Counterfactual cannot run → said plainly, and the finding keeps its deterministic confidence.
A judge verdict is about a component, not a state. A rejection applies everywhere that component appears, so the same question is never paid for twice.
- Cross-locale matching leans on structural paths when an app has no stable test attributes. A DOM that differs between locales weakens it, and findings in unmatched regions cannot be counterfactually verified.
- Each (state, locale) is measured with a fresh navigation, serially. This is the dominant cost on large sites and is parallelizable, but is not parallel today.
- Heavily animated pages can fail to reproduce a discovered state; those are reported as coverage gaps rather than measured.
- The element measurement cap (
--max-elements, default 2500) can truncate very large pages; the run warns when it is hit.
Node ≥ 20 and Chrome or Chromium. No API key required — the judge uses a local CLI you are already signed into, and everything else runs without a model at all.
MIT.