feat(HNT-2900): commit the publisher list and its generator - #38
Draft
mmiermans wants to merge 1 commit into
Draft
feat(HNT-2900): commit the publisher list and its generator#38mmiermans wants to merge 1 commit into
mmiermans wants to merge 1 commit into
Conversation
mmiermans
force-pushed
the
claude/hnt-2086-agent-tick
branch
from
July 6, 2026 17:09
422b88b to
b81ce44
Compare
mmiermans
force-pushed
the
claude/hnt-2086-publisher-data
branch
from
July 6, 2026 17:09
6e5b486 to
972641e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
HNT-2900
Commit the crawl-agent's generated
publishers.json, a smallpublishers.example.json, and the Python generator that builds it from the legacy crawl PAGES export, plus a guard test that the committed data validates through the real loader. This gives the agent a self-contained startup config that a reviewer can read and CI can verify.This builds on the message validation slice by using its
validatePublisherListand theloadPublisherListloader from the earlier slice, both exercised by the guard test. See docs/ARCHITECTURE.md in #14 for how the agent loads this list at startup.Implementation decisions
publishers.jsonis committed as generated data and excluded from the reviewable diff, and the smallpublishers.example.jsonand the generator stand in for review.publishers.jsonvialoadPublisherListat its module-relative path and asserts it has pages and an emptylive_articles.scripts/generate-publishers.pyimports the legacypages.pymodule and emits thePublisherListJSON.NEW_TAB_<LOCALE>surface id and each of its topics becomes a lowercased discovery context,interval_minutesdefaults to 20 per the tech spec's example cadence, and duplicate URLs are merged into one entry with de-duplicated contexts.NEW_TAB_EN_USand the like are the Corpus APIScheduledSurfaceGUIDvalues, matching content-monorepo'sScheduledSurfacesEnuminpackages/content-common/src/types.ts. The agent's validator requires unique page URLs, so merging keeps one entry per page carrying all its contexts.live_articlesarraylive_articles: [].