Repository navigation
feat: send dipper's alerts to Slack - #735
Merged
Merged
Conversation
Dipper wrote terminal colour codes into its logs even when they went to a log store, which split fields like the event tag so a search for them found nothing. Logs are now plain text.
Dipper's tagged warnings and errors, such as a cancel that keeps failing, were only in its logs. With a Slack webhook in the config, each listed tag is now posted, at most once per tag every 15 minutes with a count of the rest, without ever holding up the code that logged it.
Slack treats &, < and > in a message as links, mentions or entities, so an error carrying an HTML page, such as a proxy's 502, came through mangled. They are now escaped.
The count of held-back alerts named the window in whole minutes rounded down, so 30 seconds read as "the last 0 minutes", and a window under a minute was only checked once a minute. Both now follow the window as configured.
Alerts dropped because the queue was full only showed up as a warning in dipper's own logs, so the "N more" figure in Slack was too low during a burst. They are now counted by event into that figure.
MoonBoi9001
marked this pull request as ready for review
October 8, 2026 19:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With a Slack webhook set in the new
alertsconfig section, dipper posts itsagreement_cancel_stuck,rpc_blocks_refusedandnonce_gap_fill_failedlog lines to Slack, at most once per tag every 15 minutes with a count of the rest; without one, nothing changes. Logs are also written without colour codes, which had split tagged fields so log stores couldn't find them.