Skip to content

DOC-7104: Add a check for links left in bare, uncanonicalized form - #4102

Merged
andy-stark-redis merged 2 commits into
mainfrom
DOC-7104-uncanonicalized-link-check
Sep 24, 2026
Merged

andy-stark-redis merged 2 commits into
mainfrom
DOC-7104-uncanonicalized-link-check

Conversation

@andy-stark-redis

@andy-stark-redis andy-stark-redis commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • migrate_shortcode_links.py's linkify stage correctly canonicalizes almost every converted link to /content/<path>.md[#anchor], but a link that gets a manual post-hoc text fix after the pipeline already ran (e.g. inserting a separator slash a relref-plus-literal-suffix concatenation was missing) never gets a second pass through linkify. The result is a bare /operate/... path that resolves to the exact same rendered href as the canonical form, so build/diff_rendered_hrefs.py — this migration's usual verification — is blind to it by construction.
  • Found on PR DOC-7104: Migrate content/operate/rs/release-notes/ to render hooks #4086 (release-notes/ unit): human review manually flagged 8 malformed links; a corpus-wide grep for the same shape found 21 across 9 files (13 more than manual review caught), plus 8 more that turned out to be genuinely pre-existing dead links in the identical shape, invisible to any prior check since Hugo's relref shortcode only ever validated its own target, never text concatenated onto it afterward.
  • build/check_uncanonicalized_links.py reuses migrate_shortcode_links.py's own resolver (_find_content_file) so a --fix run applies the exact same rewrite the pipeline would have. Three outcomes: FIXABLE (resolves, --fix rewrites it), MOUNT_ONLY (resolves only through a Hugo module mount — never auto-fixed, same rationale migrate_shortcode_links.py already documents for not following mounts on a rewrite), DEAD (doesn't resolve at all — reported only, never guessed).
  • Verified against the pre-fix state of PR DOC-7104: Migrate content/operate/rs/release-notes/ to render hooks #4086: reproduces the exact same 21 FIXABLE / 8 DEAD split. A full-corpus scan of content/ elsewhere comes back to 3 unrelated hits, confirming it isn't noisy.

Test plan

  • build/test_check_uncanonicalized_links.py — 7 cases (fixable, fixable-with-anchor, dead, already-canonical, external/anchor-only/mailto skipped, /commands/?group= exception, the exact PR DOC-7104: Migrate content/operate/rs/release-notes/ to render hooks #4086 shape), all passing (no pytest in this repo's venv, ran the test functions directly — matches this migration's other build/test_*.py scripts, none of which are wired into CI either)
  • Reproduced PR DOC-7104: Migrate content/operate/rs/release-notes/ to render hooks #4086's exact pre-fix finding set (21 FIXABLE / 8 DEAD) by checking out that commit's release-notes/ content into the full corpus and scanning
  • Full content/ scan: 3 unrelated hits elsewhere, all genuine (2 bare /commands/<cmd> links in already-converted oss_and_stack release notes, 1 in operate/rs/7.4/_index.md)

🤖 Generated with Claude Code


Note

Low Risk
Build-only tooling and tests; no runtime or published docs behavior unless someone runs --fix on markdown files.

Overview
Adds build/check_uncanonicalized_links.py, a corpus scanner that catches Markdown links still pointing at bare content mounts (/operate/, /develop/, /integrate/, /commands) instead of the migration’s canonical /content/<path>.md form—cases diff_rendered_hrefs.py misses because they render the same href.

Each hit is classified as FIXABLE (same resolver as migrate_shortcode_links; optional --fix rewrites in place), MOUNT_ONLY (report only), or DEAD (broken link, report only). /commands links with query/fragment (no slash before ?/#) are treated as fixable to /content/commands even without an on-disk _index.md.

build/test_check_uncanonicalized_links.py adds filesystem-backed tests for fixable/dead/skipped cases, anchors, the /commands?group= shape, and the PR #4086 relref-plus-suffix defect.

Reviewed by Cursor Bugbot for commit d2f6723. Bugbot is set up for automated code reviews on this repo. Configure here.

migrate_shortcode_links.py's linkify stage correctly canonicalizes almost
every converted link to /content/<path>.md[#anchor], but a link that gets
a manual post-hoc text fix after the pipeline already ran (e.g. inserting
a separator slash a relref-plus-literal-suffix concatenation was missing)
never gets a second pass through linkify. The result is a bare
/operate/... path that resolves to the exact same rendered href as the
canonical form, so build/diff_rendered_hrefs.py -- this migration's usual
verification -- is blind to it by construction.

Found on PR #4086 (DOC-7104 release-notes/ unit): human review manually
flagged 8 malformed links; a corpus-wide grep for the same shape found 21
across 9 files (13 more than manual review caught), plus 8 more that
turned out to be genuinely pre-existing dead links in the identical
shape, invisible to any prior check since Hugo's relref shortcode only
ever validated its own target, never text concatenated onto it
afterward.

check_uncanonicalized_links.py reuses migrate_shortcode_links.py's own
resolver (_find_content_file) so a --fix run applies the exact same
rewrite the pipeline would have. Verified against the pre-fix state of
PR #4086: reproduces the same 21 FIXABLE / 8 DEAD split exactly, and a
full-corpus scan of content/ elsewhere comes back to 3 unrelated hits,
confirming it isn't noisy.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

DOC-7104

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

…o slash

The original regex required `/` or end-of-string right after the mount
name, so a link like `/commands?group=cluster` (no trailing slash before
the query) silently passed through unchecked -- this tool's own blind
spot, found the hard way: human review caught it by hand on DOC-7104
PR #4093, and the identical instances recurred in #4094/#4096/#4098
before this fix existed to catch them.

Also reconsiders the `/commands` special case: it has no backing
_index.md on disk, so _find_content_file always reports it unresolvable,
but Hugo auto-generates a section page for the directory and GetPage
finds it anyway (confirmed by building both /commands?group=x and
/content/commands?group=x and comparing rendered hrefs -- identical).
Review wanted the canonical form applied there too, so it's now
hardcoded as FIXABLE instead of silently skipped.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@dwdougherty dwdougherty left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code skimmed. Approved.

@andy-stark-redis
andy-stark-redis merged commit 3b26543 into main Sep 24, 2026
96 of 98 checks passed
@andy-stark-redis
andy-stark-redis deleted the DOC-7104-uncanonicalized-link-check branch September 24, 2026 13:32
@andy-stark-redis

Copy link
Copy Markdown
Contributor Author

Thanks for your heroic review of all these PRs @dwdougherty ! Hopefully this check will prevent any more issues like the ones you've found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants