Repository navigation
Document that primary instance scale-out is limited to error ingestion - #5970
Open
johnsimons wants to merge 1 commit into
Open
johnsimons wants to merge 1 commit into
johnsimons wants to merge 1 commit into
Conversation
An audit of every hosted service, handler, controller and persister path in the primary found five places in the retry pipeline, heartbeat monitoring, integration event dispatch and throughput collection that assume a single process. Add a decision record concluding that exactly one full primary instance may run against a database, with `--error-ingestion-only` workers providing horizontal scale and active-passive failover providing HA, plus a topic page listing what each component would need to become deployment-global.
warwickschroeder
approved these changes
Oct 13, 2026
rbev
requested changes
Oct 13, 2026
rbev
left a comment
rbev
left a comment
Contributor
There was a problem hiding this comment.
I think it's worth having some more discussion about this PR before we merge it
| - Subscription rows on SQL Server and PostgreSQL, settings rows, custom check rows and known endpoint rows are written through upserts that handle the insert race. | ||
| - Azure Blob and S3 body storage, and file system body storage on a shared mount. Bodies are immutable and keyed by message id. | ||
|
|
||
| ## What assumes one process |
Contributor
There was a problem hiding this comment.
This section seems to go into a lot of detail about a potential future solution, given there are no plans at this stage to implement that I'm not sure that it is right to document that here
Contributor
There was a problem hiding this comment.
The document as a whole could be consolidated a bit as well. Its a bit too verbose.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An audit of every hosted service, handler, controller and persister path in the primary found five places in the retry pipeline, heartbeat monitoring, integration event dispatch and throughput collection that assume a single process. Add a decision record concluding that exactly one full primary instance may run against a database, with
--error-ingestion-onlyworkers providing horizontal scale and active-passive failover providing HA, plus a topic page listing what each component would need to become deployment-global.