feat(i18n): automated translation pipeline with machine review gate

[Timur Tukaev (tym83)]

Summary

Builds the automated translation pipeline on top of the i18n proof-of-concept in #593. Where #593 established the multi-language site structure and a few hand-checked pages, this turns translation into a repeatable, reviewable process: it discovers what changed in the English source, translates it through a machine review gate, and opens a weekly PR for a maintainer to merge. Nothing reaches production without that merge.

Stacked on #593 — please review and merge that first; this PR targets poc/i18n-multilang and will retarget to main once #593 lands.

What

• Change detection. source_digest (sha256 of the English source, the same convention hack/check-i18n.sh already enforces) drives a worklist of missing/stale pages per language. Scope is the latest docs version only, plus recent blog posts — older docs versions are noindex, so translating them would spend budget on pages search engines ignore.
• Per-page review gate. Each page runs translate → back-translate (meaning-drift check) → two virtual native reviewers (a technical editor for fluency, a Cozystack maintainer for technical correctness) → a bounded revise loop. Code, shortcodes, and inline code are masked during translation and restored after, so they stay byte-for-byte. caption/alt text inside shortcodes is translated (it renders to readers); src/width are not.
• Honest stamping. A page that clears the gate is stamped translation_review: auto-reviewed; one that runs out of revise rounds with findings still open is stamped auto-reviewed-with-findings and its findings are posted to the weekly PR so a maintainer can triage them. Only a human sets ratified.
• Reader-facing disclosure. Documentation pages carry a machine-translation banner linking to the English original until a native speaker marks them ratified. The banner is docs-only by design — the blog, marketing pages, and homepage hero do not carry it.
• Cadence. The runner translates daily (to use each day's model budget) but accumulates into one i18n/week-<ISO week> branch, so maintainers review and merge a single translation PR per week. publish_mode: pr_only — CODEOWNERS and branch protection are untouched.
• Sample output. Two real pipeline translations of the same docs page (ru + de, docs/v1.5/getting-started/install-kubernetes.md) are included so quality can be judged directly.

Why

The English docs are the source of truth and change constantly; native localization takes months. This lets translated docs ship and be indexed immediately (publish-then-ratify), with the banner keeping that honest, while native review happens asynchronously and is tracked per page. The review gate is not a substitute for native ratification — both virtual reviewers are the same model as the translator, so the gate measures self-consistency and catches the obvious failures, no more. It is deliberately conservative: fail-closed on unparseable reviewer verdicts, refuses to write a page with a dropped code block, and never publishes without a maintainer merge.

Notes for reviewers

• Auth. Bootstrapped on a maintainer's Claude subscription via the Claude Agent SDK (no metered billing). The intent is to move the backlog burst to an organization-owned API key; that is a one-line config switch (auth: api-key), no code change. See README "Ownership and continuity".
• Backlog size. 183 pages × 6 languages ≈ 1084 jobs. Measured on the pilot, a page is ~3 min when it clears the gate and ~11 min when it runs the full revise loop, so the first pass is best done on an API key rather than a personal subscription.
• es/pt-br are translated by the pipeline but not yet declared in hugo.yaml (no content shipped). Declaring a language before its content exists publishes empty indexable pages; a test enforces the ordering.
• Tests for the pure functions: python3 hack/i18n/test_i18n.py.

Preview

The included sample pages render under /ru/docs/v1.5/getting-started/install-kubernetes/ and /de/… in the deploy preview, each with the machine-translation banner.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 80a59cdb-e006-4e69-b345-52c94210d6b8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/i18n-pipeline

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[gemini-code-assist]

Code Review

This pull request introduces an automated, machine-reviewed translation pipeline for the Cozystack website, adding localization scripts, prompts, style guides, and tests under hack/i18n/, alongside initial support for Spanish and Brazilian Portuguese. It also integrates a machine-translation disclaimer banner into the documentation layout and adds hreflang alternates for SEO. Feedback on these changes highlights a non-standard Hugo API usage (.Sites.Default) in the translation banner, a duplicate entry in .gitignore, and formatting conflicts in the translation and revision prompts where the instructions ask for key: value lines instead of the JSON format expected by the parsing script.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

.Sites.Default is not a standard Hugo API and may cause template evaluation errors depending on the Hugo version. The standard and safe way to retrieve the default content language is .Site.DefaultContentLanguage.

Suggested change

	{{- $src := .Sites.Default.Language.Lang -}}
	{{- if and (ne .Lang $src) (ne .Params.translation_review "ratified") -}}
	{{- $src := .Site.DefaultContentLanguage -}}
	{{- if and (ne .Lang $src) (ne .Params.translation_review "ratified") -}}

[gemini-code-assist]

The directory .venv-i18n/ is duplicated in the .gitignore file (on lines 32 and 36). Removing the duplicate entry keeps the file clean.

# i18n pipeline venv (bootstrapped by hack/i18n/run-daily.sh)
.venv-i18n/

# i18n pipeline: per-run artifacts, not content
hack/i18n/last-run-findings.md

[gemini-code-assist]

The system prompt instructs the model to output the front matter as key: value lines, but the translation pipeline (translate.py via _FM_PROTOCOL) expects a JSON object. This conflict can confuse the model and lead to protocol parsing errors. Updating the prompt to match the JSON protocol will improve reliability.

Suggested change

	===FRONTMATTER===
	<the translated values, one `key: value` per line, same keys, same order>
	===BODY===
	<the translated body>
	===FRONTMATTER===
	<the translated values as a JSON object, same keys, same order>
	===BODY===
	<the translated body>

[gemini-code-assist]

Similar to the translation prompt, the revision system prompt instructs the model to output key: value lines for the front matter, which conflicts with the JSON object format expected by _FM_PROTOCOL in translate.py. Updating this to instruct the model to return a JSON object will prevent protocol errors during the revision loop.

Suggested change

	===FRONTMATTER===
	<corrected key: value lines, same keys, same order>
	===BODY===
	<corrected body>
	===FRONTMATTER===
	<corrected JSON object, same keys, same order>
	===BODY===
	<corrected body>

[Andrei Kvapil (kvaps)]

Reviewed this PR's own diff (36 files) against its base, with #593 now merged. Ran hack/i18n/test_i18n.py (49 pass) and hack/check-i18n.sh (clean). The design is careful and the README is unusually honest about what the gate is and isn't — nearly all of my comments are about the runner, not the translation logic.

Blocking: the daily runner wedges on its second run

run-daily.sh translates on whatever branch the clone happens to be on, publishes on a different one, and the EXIT trap restores the original branch (hack/i18n/run-daily.sh:79-81, 115-126). In order:

1. Day 1 commits translations onto i18n/week-<N> and pushes. The trap then checks out START_REF, which deletes those files from the working tree — they now only exist on the week branch.
2. Day 2 therefore sees them as missing again (build_worklist checks the filesystem, lib.py:197-208) and re-translates the exact same pages, spending a day of quota redoing day 1.
3. Publishing then fails hard: the re-translated files are untracked, origin/i18n/week-<N> already contains them, and git checkout -B "$BRANCH" origin/$BRANCH (run-daily.sh:121) aborts with "The following untracked working tree files would be overwritten by checkout". Under set -e the run dies leaving the tree dirty, so every later run stops at the clean-tree preflight (run-daily.sh:74-78).

I reproduced this in a scratch repo following the exact command sequence: day 2 exits 1 at the checkout and leaves ?? content/ru/ behind. The first run of a new week hits the same wall through the origin/main path once the previous week's PR is merged.

Root cause is that the working tree is never synced to the remote — git fetch (line 115) updates refs only, so both the English sources and the already-published translations stay frozen at clone time, and worklist correctness depends on that tree.

Suggestion: check out the week branch before translating — fetch, git checkout -B "$BRANCH" origin/$BRANCH (or origin/main), then run translate.py, then commit in place. That makes the tree the branch the work belongs to, keeps day 2's worklist accurate, removes the untracked-file collision, and refreshes content/en in the same move.

Worth fixing

The revise loop's last round is never re-reviewed. translate.py:271-319 — on the final iteration a page with findings is revised and the loop then exits. The text written is post-revision, but findings (stamped, returned, and posted to the weekly PR) describe the pre-revision text. So the finding list in the PR comment doesn't correspond to the pages actually shipped, and the last Opus call for each failing page produces output nothing ever checks. Either re-check after the last revise, or skip revising on it.

"A page is only written once it passes the gate" (README.md:35, config.yaml:115) contradicts the code and the rest of the README: pages that exhaust the revise loop are written, stamped -with-findings (translate.py:342-346, README.md:249-252). The diagram at README.md:13-33 has no edge for that path either. This is the sentence a reviewer leans on, so it's worth making it say what happens.

auto_merge exists. The default is pr_only and the claim in the description holds today, but run-daily.sh:170-173 will call gh pr merge --auto --squash after a one-word config change, with no second signal required. If "nothing reaches production without a maintainer merge" is meant as a property rather than a default, consider dropping that branch — nothing in this PR uses it.

Failures in the publish stage are swallowed. gh pr create ... || true and gh pr comment ... || true (lines 152-154, 167). A failed gh call leaves commits pushed, no PR opened, and exit 0 — indistinguishable from a good run in cron output.

Unpinned dependencies. pip install --quiet claude-agent-sdk pyyaml runs on every invocation (line 34), upgrading in place on a machine that holds a maintainer's CLAUDE_CODE_OAUTH_TOKEN and gh credentials. A pinned requirements.txt costs nothing here.

An i18n problem already visible in the shipped strings

The banner builds one sentence out of four keys plus link text (layouts/partials/translation-banner.html:39-42). That hardcodes English word order and punctuation, and two of the six languages already read wrong:

• hi: "देखें अंग्रेज़ी मूल" — Hindi is verb-final; the natural order is "अंग्रेज़ी मूल देखें".
• de / ru: "Einen Fehler entdeckt? eröffnen Sie ein Issue." and "Нашли ошибку? создайте issue." — lowercase after a question mark.

One key per sentence, with the link as a placeholder, lets a translator move it. Worth noting these UI strings never pass through the pipeline's review gate — only content/ does.

Smaller things

• .gitignore:32 and :36 — .venv-i18n/ listed twice.
• README.md:93 says 183 × 4 = 720 jobs; config.yaml lists six languages and the description says ~1084.
• Nothing removes a translation whose English source was deleted, and check-i18n.sh only checks digests of files that exist, so orphans accumulate silently.
• blog_since: "2026-05-17" is a fixed date that will quietly rot into "nothing new is in scope" — the config comment already suggests computing now-60d.

On the security posture and the gate

No workflow is added, so the usual automation risks don't apply: no pull_request_target, no repo-held secret, no untrusted checkout. Model calls use allowed_tools=[] and max_turns=1 (translate.py:117-122), so the translator has no filesystem or tool access, and git add -- content/<lang> genuinely bounds what a run can commit. What's left is that everything runs from one person's machine on their personal subscription, with their gh token and their DCO sign-off on machine-generated commits — and that the human merge gate is procedural, since no maintainer can meaningfully review Hindi or Chinese docs in a bulk weekly PR. The README says as much and the banner is the right mitigation; I'm noting it, not objecting.

The gate is real machinery rather than a stamp: verdict parsing fails closed (translate.py:149-172), an explicit "revise with no findings" verdict still blocks (lines 288-290, 300-302), and a lost or duplicated placeholder is a hard refusal to write (lines 213-219). Its ceiling is the one the description already admits — same model, so it measures self-consistency. Worth being precise that it never blocks publication, only stamps: it's a triage classifier, not a gate.

Requesting changes for the runner bug alone; the rest is small. Noting for the record that this is a draft and was stacked on #593, which has since merged.

[netlify]

❌ Deploy Preview for cozystack failed.

Name	Link
🔨 Latest commit	ae79b45
🔍 Latest deploy log	https://app.netlify.com/projects/cozystack/deploys/6a5e40370735590008a9cf0e

[Timur Tukaev (tym83)]

All findings addressed in 4653ba7..ae79b45 (8 commits). Point by point:

Runner wedge (blocking). Fixed the way you suggested: the week branch is now checked out before translating (fetch + checkout -B from origin/i18n/week-N or fallback base), and the run commits in place — 1f7f304. Reproduced your two-day scenario in a scratch repo before and after: day 2 now sees day 1's pages as done and the untracked-file collision cannot occur. Going further down that path surfaced three more holes in the reordering, fixed in 025d2a1 and ae79b45:

• a new week always based on origin/main would re-translate an unmerged previous week and open a conflicting PR — a new week now bases on the newest surviving i18n/week-* branch;
• checkout -B origin/... silently discarded commits stranded by a failed push — a local branch ahead of (or unknown to) origin is kept, and stranded branches are re-pushed at the start of every run;
• the re-push logic in turn had to discriminate zombies: a merged-and-deleted branch whose local copy survived is deleted (by PR state via gh pr view, since ancestor checks break under squash merges), not resurrected. One residual edge remains: a morning where gh is broken but git push works could re-plant a zombie once; cheap hardening later is to skip that leg when gh auth status fails.

Revise loop. The final round no longer revises — 5c1269d. The text written is always the text the reviewers last saw, and the findings posted to the weekly PR describe the pages actually shipped.

README vs code. Reworded to say the gate triages rather than blocks, added the missing -with-findings edge to the diagram, fixed the job math (183 × 6 ≈ 1100), and noted in Governance that it's a triage classifier — 62395fc.

auto_merge is gone as a code path, not a default — b522046. || true on gh pr create/comment dropped in the same commit: commits are pushed by then, so a failed publish now fails the run. Deps pinned in hack/i18n/requirements.txt.

Banner. One i18n key per sentence with the link as a template placeholder, all seven languages — b90a72f. Hindi is verb-final now («अंग्रेज़ी मूल देखें»), de/ru capitalize after the question mark, zh-cn uses full-width punctuation. Verified with a full Hugo build rendering the banner in ru/de/hi/zh-cn. Agreed these strings bypass the content gate — they're hand-reviewed.

Smaller things — 45326b6, 62395fc: .gitignore dup removed; orphaned translations are now removed by the pipeline (only source_digest-stamped files, with a mass-deletion floor of 5 English pages and a hard refusal to run against a missing content/en); blog_since accepts a rolling "60d" window and the config uses it. One note: check-i18n.sh did already report orphans (references English source that does not exist) — what was missing was removal.

Tests: 54 pass (5 new), check-i18n.sh clean. Unrelated but spotted along the way: head-end.html drops the latest_version_id fallback — if that param ever disappears, all docs go noindex silently; happy to fix here or separately.