Skip to content

feat(HNT-2887): add crawl domain message and event types - #19

Draft
mmiermans wants to merge 1 commit into
claude/hnt-2086-redis-state-packagefrom
claude/hnt-2086-domain-types
Draft

feat(HNT-2887): add crawl domain message and event types#19
mmiermans wants to merge 1 commit into
claude/hnt-2086-redis-state-packagefrom
claude/hnt-2086-domain-types

Conversation

@mmiermans

@mmiermans mmiermans commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Goal

HNT-2887

Add the discovery-side and agent-side crawl domain types so the validators, handlers, and consumers that follow reason against one shared set of interfaces. This slice introduces DiscoveryContext, CrawlArticleDiscoveryMessage, LiveArticle, and PublisherList in the messages module, ArticleDiscoveryEvent in the events module, and an optional refresh_interval_minutes on CrawlArticleMessage, all re-exported from the crawl-common barrel. The shapes to reason about are the discovery job, the discovery event that becomes a crawl.article_discoveries row, and the publisher list the agent loads from JSON.

See docs/ARCHITECTURE.md in #14 for the message and event contracts these types encode.

Implementation decisions

Decision Approach Why
Make refresh_interval_minutes optional on CrawlArticleMessage Declare it as an optional field, and the worker falls back to its configured fetch TTL when it is absent A rolling deploy must not reject messages that were enqueued before the field existed, which is the additive-only schema evolution that Google Cloud Pub/Sub schema evolution is designed to support: add the field, keep it optional, then depend on it.

@mmiermans
mmiermans force-pushed the claude/hnt-2086-domain-types branch from c389458 to 5da2693 Compare July 6, 2026 17:08
@mmiermans mmiermans changed the title feat(HNT-2086): add crawl domain message and event types feat(HNT-2887): add crawl domain message and event types Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant