From 2f669fdab58d10f9cf03bce5ea79996bad2ff1e0 Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Wed, 26 Aug 2026 16:42:05 +0100 Subject: [PATCH 1/2] DOC-7003 add report-only fragment checking and link house rules MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a second, non-gating lychee pass over fragment-bearing URLs, a documented fragment-only exclusion file, and three external-link rules in AGENTS.md. The ticket proposed enabling fragment checking with "only a config key". Testing against the pinned lychee 0.24.2 showed that cannot work. A missing fragment exits 2, exactly as a dead page does, so one run cannot report anchors without also gating on them. And there is no fragment-specific exclude — passing a host to --exclude drops the URL from checking altogether, so suppressing anchor noise on a host would also stop us noticing when its page dies. Hence a separate invocation over just the fragment-bearing URLs, layering CLI flags on the existing config so throttling and the user agent cannot drift between the two runs, with its exit code deliberately never consulted and its findings going to their own tracking issue. Two smaller findings shaped the implementation. A misspelled config key exits 3 with the list of valid keys, so a typo here fails loudly rather than silently disabling the check. And an exclude file does honour "#" comments, which is what lets the exclusions carry their justifications — established with a comment line that would have matched as a regex if it were being applied as one, since the obvious test cannot tell a working comment from a regex that simply misses. The substantive correction is to the exclusion list. The ticket named eight hosts to exclude as JS-rendered. Fetching one failing URL per host and looking for the anchor in the served HTML supports two of them, and the stated reason is wrong even there: on both AWS and Google the anchor is plainly present in the HTML that curl receives, alongside other server-rendered content ids, so these are checker false positives by some mechanism I could not establish rather than JS rendering. For the other six an independent fetch could not find the anchor either, which makes those failures possibly real, so excluding them would have hidden genuine rot. They stay in the report. An id count is not the test to use here, incidentally, because navigation and page chrome inflate it on any real docs page. The first run also earned its keep immediately: it independently rediscovered all four Part A2 anchors, confirmed the railway anchor that shipped as an unverified gap in the previous PR is genuinely dead, found two dead Wikipedia anchors nobody had reported, and showed that cloud.google.com now redirects to docs.cloud.google.com, which is 16 links wanting a new host. Learned: verify a candidate exclusion by finding the anchor in the served HTML — six of the eight hosts this ticket wanted excluded turned out to be plausibly real failures, and the JS-rendered rationale was wrong even where the exclusion held. Constraint: the fragment pass must never feed the gating flag, because lychee returns exit 2 for a dead anchor and a dead page alike. Rejected: enabling fragments on the existing gating run via a config key | no fragment-specific exclude exists and exit 2 is ambiguous, so it would gate the build on anchor noise Directive: don't add a host to .lychee-fragment-excludes.txt without fetching a failing URL and confirming the anchor really is in the served HTML. Gaps: the 34 surviving failures are classified by host, not individually triaged; the railway anchor, two Wikipedia anchors and the cloud.google.com host move are real findings this PR does not fix. Ticket: DOC-7003 Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/link_check.yaml | 121 +++++++++++++++++++++++++++++- .lychee-fragment-excludes.txt | 28 +++++++ AGENTS.md | 10 +++ 3 files changed, 156 insertions(+), 3 deletions(-) create mode 100644 .lychee-fragment-excludes.txt diff --git a/.github/workflows/link_check.yaml b/.github/workflows/link_check.yaml index a7166e055e..8ffcf4ad8a 100644 --- a/.github/workflows/link_check.yaml +++ b/.github/workflows/link_check.yaml @@ -5,9 +5,16 @@ name: link_check # checks them with lychee (.lychee.toml), and opens/updates a tracking issue # when any are broken. Run it on demand from the Actions tab via "Run workflow". # -# The work is split across two jobs so that the third-party lychee action only -# ever receives a read-only token: link_check runs with `contents: read`, and -# only the separate report_broken_links job is granted `issues: write`. +# It also runs a second, report-only lychee pass over just the fragment-bearing +# URLs (`--include-fragments`), reporting candidate dead anchors to a separate +# tracking issue. That pass never gates: lychee cannot distinguish a dead anchor +# from a dead page by exit code, and anchor output needs too much triage to block +# a build on. +# +# The work is split across three jobs so that the third-party lychee action only +# ever receives a read-only token: link_check runs with `contents: read`, and only +# the separate report_broken_links and report_dead_fragments jobs are granted +# `issues: write`. on: schedule: @@ -29,6 +36,7 @@ jobs: contents: read outputs: links_broken: ${{ steps.flag.outputs.broken }} + fragments_flagged: ${{ steps.fragflag.outputs.flagged }} steps: - name: Free up disk space run: | @@ -88,6 +96,64 @@ jobs: echo "broken=false" >> "$GITHUB_OUTPUT" fi + # --- Fragment check: report-only, deliberately not part of the gate above --- + # + # lychee returns exit 2 for a missing fragment, exactly as it does for a dead + # page, so a fragment failure is indistinguishable from a broken link in one + # run. Measured on this corpus, roughly a third of first-run fragment output + # needs triage, which is too noisy to gate a build. Hence a separate run whose + # exit code is never consulted, reporting to its own tracking issue. + # + # Only fragment-bearing URLs are checked (287 of 14,607 at time of writing), so + # this costs a fraction of the gating run rather than doubling it. `#/...` is + # dropped because those are single-page-app hash routes, not anchors, and + # anchor-only mode ignores `#:~:text=` scroll-to-text fragments by design. + - name: Extract fragment-bearing URLs + id: fragurls + run: | + grep '#' external-urls.txt | grep -v '#/' > fragment-urls.txt || true + count=$(wc -l < fragment-urls.txt) + echo "count=$count" >> "$GITHUB_OUTPUT" + echo "Checking fragments on $count of $(wc -l < external-urls.txt) URLs" >&2 + + - name: Check fragments (report-only) + id: fragments + if: steps.fragurls.outputs.count != '0' + uses: lycheeverse/lychee-action@8646ba30535128ac92d33dfc9133794bfdd9b411 + with: + lycheeVersion: v0.24.2 + # Layered on .lychee.toml rather than duplicating it, so throttling, the + # user agent and the accepted status codes cannot drift between the two runs. + args: >- + --config .lychee.toml + --include-fragments=anchor-only + --exclude-file .lychee-fragment-excludes.txt + fragment-urls.txt + output: lychee-fragment-report.md + fail: false + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + - name: Flag whether fragments were found + id: fragflag + run: | + # Note this reads the *fragments* step, never the gating one. A dead anchor + # must not be able to fail this workflow. + if [ "${{ steps.fragments.outputs.exit_code }}" != "0" ] \ + && [ -s lychee-fragment-report.md ]; then + echo "flagged=true" >> "$GITHUB_OUTPUT" + else + echo "flagged=false" >> "$GITHUB_OUTPUT" + fi + + - name: Upload fragment report + if: steps.fragflag.outputs.flagged == 'true' + uses: actions/upload-artifact@v4 + with: + name: lychee-fragment-report + path: lychee-fragment-report.md + if-no-files-found: error + - name: Upload report if: steps.flag.outputs.broken == 'true' uses: actions/upload-artifact@v4 @@ -135,3 +201,52 @@ jobs: else gh issue create --title "$title" --label link-check --body-file lychee-report.md fi + + report_dead_fragments: + name: Open or update fragment tracking issue + needs: link_check + # Independent of links_broken on purpose. A dead anchor is reported whether or not + # the gating check also failed, and it never blocks anything -- this job's own + # success is not a gate either. Kept as a separate issue from the broken-link one + # so a triage backlog of anchors can't bury an actually-dead page. + if: ${{ always() && needs.link_check.outputs.fragments_flagged == 'true' }} + runs-on: ubuntu-latest + permissions: + contents: read + issues: write + steps: + - name: Download fragment report + uses: actions/download-artifact@v4 + with: + name: lychee-fragment-report + path: . + + - name: Open or update fragment tracking issue + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GH_REPO: ${{ github.repository }} + run: | + gh label create link-check-fragments --color FEF2C0 \ + --description "Automated anchor/fragment check (report-only)" 2>/dev/null || true + + # Expect to triage these rather than treat them as a pass/fail list: a + # missing anchor can mean the anchor moved, the page was restructured, or + # lychee simply cannot see it. Confirm against the served HTML before editing + # a link, and see .lychee-fragment-excludes.txt for the hosts already ruled out. + { + echo "Report-only. These are **candidate** dead anchors, not confirmed defects —" + echo "roughly a third of a first run needs triage. Verify one against the served" + echo "HTML before editing any link." + echo + cat lychee-fragment-report.md + } > fragment-issue-body.md + + title="Dead anchors in docs links (report-only)" + existing=$(gh issue list --label link-check-fragments --state open --json number --jq '.[0].number // empty') + + if [ -n "$existing" ]; then + gh issue comment "$existing" --body-file fragment-issue-body.md + echo "Updated existing issue #$existing" + else + gh issue create --title "$title" --label link-check-fragments --body-file fragment-issue-body.md + fi diff --git a/.lychee-fragment-excludes.txt b/.lychee-fragment-excludes.txt new file mode 100644 index 0000000000..edb9e869b1 --- /dev/null +++ b/.lychee-fragment-excludes.txt @@ -0,0 +1,28 @@ +# Hosts excluded from the *fragment* check only (see .github/workflows/link_check.yaml). +# +# These URLs are still checked normally by the gating link check via .lychee.toml — +# this file only suppresses "Cannot find fragment" errors. lychee has no +# fragment-specific exclude (verified, lychee 0.24.2: --exclude drops the URL +# entirely), so the fragment run is a separate invocation that layers this file on +# top of .lychee.toml. +# +# Entries are regexes, one per line. Lines starting with # are comments (verified: +# a "# ..." line is not applied as a regex). +# +# Bar for adding a host: fetch one failing URL and confirm the anchor IS in the +# served HTML. If an independent fetch *also* can't find the anchor, the failure may +# be real -- leave it in the report rather than hiding it. + +# Anchor verified present as id="concepts-availability-zones" in the served HTML, +# alongside other server-rendered content ids, yet lychee reports it missing. +# So these are false positives -- but NOT because the docs are JS-rendered, which +# was the original assumption. The actual mechanism is unestablished; if someone +# works it out, narrow or drop this entry. +^https?://docs\.aws\.amazon\.com/ + +# Same pattern: id="overview-of-cloud-billing-roles-in-cloud-iam" is present in the +# served HTML and lychee still reports it missing. Mechanism unestablished. +# Separately, this host silently moved -- cloud.google.com/... 301s to +# docs.cloud.google.com/... -- so our 16 anchored links here should be repointed at +# the new host, after which this entry may become unnecessary. +^https?://(www\.)?cloud\.google\.com/ diff --git a/AGENTS.md b/AGENTS.md index bbed14d07a..7cfa4dc388 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,6 +57,16 @@ around a literal, rewrite the sentence. - **Cross-references use the relref shortcode**, not markdown paths: `{{< relref "/operate/rs/clusters/new-cluster-setup" >}}`. A broken relref fails the build. Link text is descriptive — never "click here" or a bare URL. + **relref validates the page, never the heading** — a relref with a dead `#anchor` + builds clean, so check the anchor against the built page yourself. +- **Prefer an internal relref to an external anchor** where we document the same thing. + The relref is build-checked; an external anchor is not, and upstream restructures + without telling us. +- **Pin GitHub deep links to a commit SHA**, never a moving branch. Line anchors such as + `#L99-L130` drift silently as the file changes, so verify the lines at the SHA you + pin. A link to a repository or file as a whole may stay on the default branch. +- **Never link a scroll-to-text fragment** (`#:~:text=`). It is not a real anchor, it + breaks on any upstream rewording, and no checker can validate it. - **Preserve shortcodes, frontmatter, and code fences verbatim.** Do not reformat them. - **Frontmatter**: copy the shape from a sibling page in the same directory rather than composing one. `title` and `linkTitle` are effectively universal; `description`, From 805675998be3246567570094147c650ef3d0583c Mon Sep 17 00:00:00 2001 From: Andy Stark Date: Thu, 27 Aug 2026 13:04:17 +0100 Subject: [PATCH 2/2] DOC-7003 stop the fragment steps from suppressing the link report MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bugbot found a real interference path, and it defeats the one property this change was supposed to guarantee. The report-only fragment steps sat between flagging broken links and uploading the gating report. `fail: false` covers a non-zero lychee exit, but not a step that dies outright -- an action setup or artifact failure. Because a step whose `if` omits a status function carries an implicit success(), such a death skipped the gating upload. report_broken_links runs on always(), so it would then have gone looking for an artifact that was never produced, failed, and silently suppressed the broken-link tracking issue. A dead anchor could have hidden a dead page, which is exactly backwards. Fixed by ordering rather than by more conditionals: the gating upload and the failing step now come first, and the fragment block runs last with always() on every step so it still survives the exit 1 without being able to interfere. The extract step also now reports zero when external-urls.txt is absent, so a failed build cannot present itself as a fragment problem. Learned: a report-only step is only non-gating if it cannot fail *before* the gating steps — implicit success() makes step order part of the contract, not a cosmetic choice. Constraint: the fragment steps must stay after the gating upload and the failing step, and must keep always(), or a fragment failure suppresses the broken-link issue. Ticket: DOC-7003 Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/link_check.yaml | 50 ++++++++++++++++++++----------- 1 file changed, 33 insertions(+), 17 deletions(-) diff --git a/.github/workflows/link_check.yaml b/.github/workflows/link_check.yaml index 8ffcf4ad8a..e2b770773a 100644 --- a/.github/workflows/link_check.yaml +++ b/.github/workflows/link_check.yaml @@ -96,7 +96,30 @@ jobs: echo "broken=false" >> "$GITHUB_OUTPUT" fi - # --- Fragment check: report-only, deliberately not part of the gate above --- + - name: Upload report + if: steps.flag.outputs.broken == 'true' + uses: actions/upload-artifact@v4 + with: + name: lychee-report + path: lychee-report.md + if-no-files-found: error + + - name: Fail if links are broken + if: steps.flag.outputs.broken == 'true' + run: | + echo "::error::Broken external links found; see the link-check tracking issue." + exit 1 + + # --- Fragment check: report-only, and deliberately last --- + # + # Ordering is load-bearing. These steps run *after* the gating upload and the + # failing step, because a step that dies here (an action setup error, which + # `fail: false` does not cover) would otherwise skip the gating upload via the + # implicit success() on later steps -- and report_broken_links, which runs on + # always(), would then fail downloading an artifact that was never produced, + # silently suppressing the broken-link tracking issue. Hence every step below + # carries always(): they must survive the exit 1 above without being able to + # interfere with it. # # lychee returns exit 2 for a missing fragment, exactly as it does for a dead # page, so a fragment failure is indistinguishable from a broken link in one @@ -110,7 +133,13 @@ jobs: # anchor-only mode ignores `#:~:text=` scroll-to-text fragments by design. - name: Extract fragment-bearing URLs id: fragurls + if: always() run: | + # The build may have failed before producing this; report nothing rather + # than erroring, so a failed build can't masquerade as a fragment problem. + if [ ! -f external-urls.txt ]; then + echo "count=0" >> "$GITHUB_OUTPUT"; exit 0 + fi grep '#' external-urls.txt | grep -v '#/' > fragment-urls.txt || true count=$(wc -l < fragment-urls.txt) echo "count=$count" >> "$GITHUB_OUTPUT" @@ -118,7 +147,7 @@ jobs: - name: Check fragments (report-only) id: fragments - if: steps.fragurls.outputs.count != '0' + if: always() && steps.fragurls.outputs.count != '0' uses: lycheeverse/lychee-action@8646ba30535128ac92d33dfc9133794bfdd9b411 with: lycheeVersion: v0.24.2 @@ -136,6 +165,7 @@ jobs: - name: Flag whether fragments were found id: fragflag + if: always() run: | # Note this reads the *fragments* step, never the gating one. A dead anchor # must not be able to fail this workflow. @@ -147,27 +177,13 @@ jobs: fi - name: Upload fragment report - if: steps.fragflag.outputs.flagged == 'true' + if: always() && steps.fragflag.outputs.flagged == 'true' uses: actions/upload-artifact@v4 with: name: lychee-fragment-report path: lychee-fragment-report.md if-no-files-found: error - - name: Upload report - if: steps.flag.outputs.broken == 'true' - uses: actions/upload-artifact@v4 - with: - name: lychee-report - path: lychee-report.md - if-no-files-found: error - - - name: Fail if links are broken - if: steps.flag.outputs.broken == 'true' - run: | - echo "::error::Broken external links found; see the link-check tracking issue." - exit 1 - report_broken_links: name: Open or update tracking issue needs: link_check