Stop the generator dropping public surfaces from llms.txt and the h1 - #90
Merged
Merged
Conversation
The _routes.json excludes "/" from middleware to reduce TTFB, but without the middleware's cache headers, the CDN couldn't cache the homepage. This adds Cache-Control with s-maxage=86400 so the CDN can serve the page from edge cache. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The _routes.json was excluding "/" and static assets from middleware to reduce TTFB. However, the middleware adds important cache headers (Cache-Control with s-maxage) and Vary headers that the _headers file alone doesn't fully replicate. Removing _routes.json lets the middleware handle all routes consistently. The _headers file remains for CDN cache headers on the root page. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Running scripts/generate-public-curriculum.mjs produced output smaller than what was committed, in two places, so any build silently discarded content. Both losses are already live. llms.txt: the committed file carried a "## Developer docs" section linking /openapi.json, /api/ai and /sitemap.xml. The generator never emitted it, so the deployed llms.txt is 18 lines against the committed 24 -- confirmed by fetching learn.significanthobbies.com/llms.txt, which matches the regenerated file, not the committed one. All three URLs return 200: /openapi.json and /api/ai come from functions/_middleware.ts and the sitemap from the sitemap writer, so nothing else fails when the generator forgets them. They just stop being discoverable by the agents this file exists to serve. Restored as a Developer docs section built from absolute(), covering /openapi.json and /sitemap.xml. /api/ai is not repeated -- it is already listed under Start here. index.html: the committed single h1 wraps the product name in a visually-hidden span before the visible tagline, so "SWE Interview Prep" reaches screen readers and indexers. The generator emitted the tagline alone, so a build stripped the brand name out of the only h1 on the page. Restored; the generator now reproduces the committed index.html byte for byte. Two regression tests cover both. Verified they catch it: against the previous generator they fail 2/14, and pass 14/14 against this one. Full suite: 483 passed, up from 481. tsc --noEmit clean. Every one of the 11 links in the generated llms.txt returns 200. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found during a fleet-wide audit of AI-discovery surface drift. Both losses described here are already live, not pending.
What was happening
Running
scripts/generate-public-curriculum.mjsproduced output smaller than what was committed, in two places, so any build silently discarded content.1.
llms.txtlost the entire Developer docs sectionThe committed file carried a
## Developer docssection linking/openapi.json,/api/ai, and/sitemap.xml. The generator never emitted it.Confirmed live rather than inferred — fetching the deployed file shows 18 lines against the committed 24, and the deployed content matches the regenerated file, not the committed one:
All three URLs return 200.
/openapi.jsonand/api/aiare served byfunctions/_middleware.ts, and/sitemap.xmlby the sitemap writer — none of them by this generator. So nothing else breaks when the generator forgets them; they simply stop being discoverable by the agents this file exists to serve.Restored as a
## Developer docssection built fromabsolute(), covering/openapi.jsonand/sitemap.xml./api/aiis deliberately not repeated — it is already listed under Start here, and the original block duplicated it.2.
index.htmllost the product name from its only<h1>The committed markup wraps the product name in a visually-hidden span before the visible tagline:
The generator emitted the tagline alone, so a build stripped "SWE Interview Prep" out of the single
<h1>on the page — costing exactly the brand findability the GEO/SEO work is meant to improve. Restored; the generator now reproduces the committedindex.htmlbyte for byte, so that drift is zero.Verification
tsc --noEmit→ exit 0llms.txtreturn 200 (checked individually)git diffagainstmainforindex.htmlis now emptyTwo regression tests cover both losses, and I verified they actually catch the bug rather than assuming it: swapped in the previous generator, regenerated, and the suite failed 2 of 14 on that file; restored the fix and it passes 14/14.
Why these went unnoticed
Neither loss breaks anything that fails loudly. The links keep working, the page keeps rendering, and the tests kept passing — the only symptom is that a generated file is quietly smaller than the one in git. That is worth knowing beyond this repo: the same pattern is present in ~12 fleet repos, with
anime-listlosing an OpenAPI/CLI section the same way and about ten more carrying uncommitted enrichment that has never landed.🤖 Generated with Claude Code