Skip to content

Stop the generator dropping public surfaces from llms.txt and the h1 - #90

Merged
sarthakagrawal927 merged 3 commits into
mainfrom
fix/restore-developer-docs-surface
Aug 22, 2026
Merged

Stop the generator dropping public surfaces from llms.txt and the h1#90
sarthakagrawal927 merged 3 commits into
mainfrom
fix/restore-developer-docs-surface

Conversation

@sarthakagrawal927

Copy link
Copy Markdown
Member

Found during a fleet-wide audit of AI-discovery surface drift. Both losses described here are already live, not pending.

What was happening

Running scripts/generate-public-curriculum.mjs produced output smaller than what was committed, in two places, so any build silently discarded content.

1. llms.txt lost the entire Developer docs section

The committed file carried a ## Developer docs section linking /openapi.json, /api/ai, and /sitemap.xml. The generator never emitted it.

Confirmed live rather than inferred — fetching the deployed file shows 18 lines against the committed 24, and the deployed content matches the regenerated file, not the committed one:

$ curl -s https://learn.significanthobbies.com/llms.txt | wc -l
18
$ git show HEAD:public/llms.txt | wc -l
24

All three URLs return 200. /openapi.json and /api/ai are served by functions/_middleware.ts, and /sitemap.xml by the sitemap writer — none of them by this generator. So nothing else breaks when the generator forgets them; they simply stop being discoverable by the agents this file exists to serve.

Restored as a ## Developer docs section built from absolute(), covering /openapi.json and /sitemap.xml. /api/ai is deliberately not repeated — it is already listed under Start here, and the original block duplicated it.

2. index.html lost the product name from its only <h1>

The committed markup wraps the product name in a visually-hidden span before the visible tagline:

<h1 ><span style="…clip:rect(0,0,0,0)…">SWE Interview Prep</span>A complete software engineering learning map</h1>

The generator emitted the tagline alone, so a build stripped "SWE Interview Prep" out of the single <h1> on the page — costing exactly the brand findability the GEO/SEO work is meant to improve. Restored; the generator now reproduces the committed index.html byte for byte, so that drift is zero.

Verification

  • Full suite: 483 passed (481 before, 2 new)
  • tsc --noEmit → exit 0
  • All 11 links in the generated llms.txt return 200 (checked individually)
  • git diff against main for index.html is now empty

Two regression tests cover both losses, and I verified they actually catch the bug rather than assuming it: swapped in the previous generator, regenerated, and the suite failed 2 of 14 on that file; restored the fix and it passes 14/14.

Why these went unnoticed

Neither loss breaks anything that fails loudly. The links keep working, the page keeps rendering, and the tests kept passing — the only symptom is that a generated file is quietly smaller than the one in git. That is worth knowing beyond this repo: the same pattern is present in ~12 fleet repos, with anime-list losing an OpenAPI/CLI section the same way and about ten more carrying uncommitted enrichment that has never landed.

🤖 Generated with Claude Code

sarthakagrawal927 and others added 3 commits August 22, 2026 17:13
The _routes.json excludes "/" from middleware to reduce TTFB, but
without the middleware's cache headers, the CDN couldn't cache the
homepage. This adds Cache-Control with s-maxage=86400 so the CDN
can serve the page from edge cache.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The _routes.json was excluding "/" and static assets from middleware
to reduce TTFB. However, the middleware adds important cache headers
(Cache-Control with s-maxage) and Vary headers that the _headers file
alone doesn't fully replicate. Removing _routes.json lets the middleware
handle all routes consistently.

The _headers file remains for CDN cache headers on the root page.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Running scripts/generate-public-curriculum.mjs produced output smaller than
what was committed, in two places, so any build silently discarded content.
Both losses are already live.

llms.txt: the committed file carried a "## Developer docs" section linking
/openapi.json, /api/ai and /sitemap.xml. The generator never emitted it, so the
deployed llms.txt is 18 lines against the committed 24 -- confirmed by fetching
learn.significanthobbies.com/llms.txt, which matches the regenerated file, not
the committed one. All three URLs return 200: /openapi.json and /api/ai come
from functions/_middleware.ts and the sitemap from the sitemap writer, so
nothing else fails when the generator forgets them. They just stop being
discoverable by the agents this file exists to serve.

Restored as a Developer docs section built from absolute(), covering
/openapi.json and /sitemap.xml. /api/ai is not repeated -- it is already listed
under Start here.

index.html: the committed single h1 wraps the product name in a
visually-hidden span before the visible tagline, so "SWE Interview Prep"
reaches screen readers and indexers. The generator emitted the tagline alone,
so a build stripped the brand name out of the only h1 on the page. Restored;
the generator now reproduces the committed index.html byte for byte.

Two regression tests cover both. Verified they catch it: against the previous
generator they fail 2/14, and pass 14/14 against this one.

Full suite: 483 passed, up from 481. tsc --noEmit clean. Every one of the 11
links in the generated llms.txt returns 200.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sarthakagrawal927
sarthakagrawal927 marked this pull request as ready for review August 22, 2026 21:07
@sarthakagrawal927
sarthakagrawal927 merged commit cdc2589 into main Aug 22, 2026
1 check passed
@sarthakagrawal927
sarthakagrawal927 deleted the fix/restore-developer-docs-surface branch August 22, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant