feat(HNT-2487): validate inbound crawl messages and the publisher list - #20
Draft
mmiermans wants to merge 1 commit into
Draft
feat(HNT-2487): validate inbound crawl messages and the publisher list#20mmiermans wants to merge 1 commit into
mmiermans wants to merge 1 commit into
Conversation
mmiermans
force-pushed
the
claude/hnt-2086-domain-types
branch
from
July 6, 2026 17:08
c389458 to
5da2693
Compare
mmiermans
force-pushed
the
claude/hnt-2086-message-validation
branch
from
July 6, 2026 17:08
0b762e8 to
b59806c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
HNT-2487
Validate the two inbound Pub/Sub job messages and the agent's publisher list at the boundary so a malformed payload fails fast rather than reaching a handler that trusts its static type. This slice adds
validateCrawlArticleMessage,validateCrawlArticleDiscoveryMessage,validateLiveArticle,validatePublisherList, and theMessageValidationErrorthey throw, each narrowing an untyped payload to its typed domain shape or throwing with the offending field named.This builds on the domain-types slice by importing the message, corpus item, and publisher list types that these validators narrow untyped payloads to. See docs/ARCHITECTURE.md in #14 for the message contracts these validators enforce.
Implementation decisions
MessageValidationError, which the Pub/Subvalidatehook reports under thevalidation-errorkind and nacks, while at agent startup the same error aborts the processvalidatehook contract in the Pub/Sub package. At agent startup a bad publisher list should fail fast so a config error aborts rather than the agent quietly crawling nothing.publisherandexcerpton a corpus itemexcerptis a real production state for a few genuine corpus items, so rejecting a blank excerpt would drop valid live articles. A blankpublisheris different: production publishers are non-empty because the Corpus API back-fills an empty publisher with a hostname fallback on create, so allowing a blank here is defensive robustness against malformed or dev-seeded input rather than a legitimate production value. It stays harmless becausepublisheris an opaque pass-through the crawler echoes back toupdateApprovedCorpusItemunchanged, so a blank never overwrites a real stored value.statusandlanguagefor string shape only, not the exact union valueinterval_minutesorrefresh_interval_minutes