Skip to content

feat(webhooks): add BetterStack status-page webhook route#14

Merged
Bre77 merged 1 commit into
mainfrom
fm/betterstack-receiver-b6
Jul 11, 2026
Merged

feat(webhooks): add BetterStack status-page webhook route#14
Bre77 merged 1 commit into
mainfrom
fm/betterstack-receiver-b6

Conversation

@Bre77

@Bre77 Bre77 commented Jul 11, 2026

Copy link
Copy Markdown
Owner

Intent

  • Subscribe firstmate to status.teslemetry.com status-page updates via a webhook, mirroring the existing ClickStack alert-webhook receiver.
    • BetterStack status-page webhook subscriptions cannot send a custom auth header (confirmed against BetterStack's own docs) - authentication is a single unguessable token carried in the URL query string, validated server-side, generated at arm time and never committed.
    • Events surface as betterstack-alert check: wakes through the existing durable wake queue, exactly like clickstack-alert, so normal intake routes them to whichever secondmate scope covers uptime/observability.
  • Architecture choice: extend the existing ClickStack receiver's listener process with a /betterstack path instead of standing up a second daemon.
    • The reverse proxy (fm.ba.id.au) forwards the whole host to one port (8111 in production). A second listener on a different port would be unreachable from the public internet without an out-of-repo proxy change this task cannot make or verify, so a literal parallel daemon was rejected as unshippable.
    • bin/fm-clickstack-recv.sh / bin/fm-clickstack-arm.sh now start the shared daemon when either gate file is present (config/clickstack-webhook.env or the new config/betterstack-webhook.env), and the listener dispatches by path, 404-ing each route independently unless its own gate is present.
    • ClickStack's own behavior is unchanged when only its gate is present: the full pre-existing tests/fm-clickstack.test.sh suite still passes unmodified.
  • New pieces, mirroring the ClickStack file set: bin/fm-betterstack-lib.sh (config + token generation), bin/fm-betterstack-poll.sh (watcher check shim), bin/fm-betterstack-arm.sh (token generation + delegates daemon lifecycle to fm-clickstack-arm.sh), docs/betterstack-webhook.md, .agents/skills/betterstack-alert-response/SKILL.md, tests/fm-betterstack.test.sh.
  • Scope boundaries: no change to ClickStack's own secret/port/bind config surface; no way to stop only one route without stopping the shared daemon (documented); no attempt to configure a distinct port for BetterStack (it always rides the ClickStack-owned shared port/bind, or the receiver's built-in defaults if ClickStack is never configured).

What Changed

  • bin/fm-clickstack-listener.py: added CSHOOK_ENABLED/BSHOOK_ENABLED/BSHOOK_TOKEN/BSHOOK_INBOX env, path-based dispatch (/betterstack vs everything else), BetterStack event_type-aware id extraction/dedup, a shared _read_body helper. ClickStack's own request path is behaviorally unchanged.
  • bin/fm-clickstack-recv.sh / bin/fm-clickstack-arm.sh: gate check now cshook_enabled || bshook_enabled; recv passes both integrations' config through to the listener.
  • bin/fm-bootstrap.sh: new betterstack_webhook_setup() (mirrors clickstack_webhook_setup()), wires state/betterstack-watch.check.sh, independent BETTERSTACK: status lines.
  • New: bin/fm-betterstack-lib.sh, bin/fm-betterstack-poll.sh, bin/fm-betterstack-arm.sh, docs/betterstack-webhook.md, docs/examples/betterstack-webhook.env, .agents/skills/betterstack-alert-response/SKILL.md, tests/fm-betterstack.test.sh.
  • AGENTS.md section 13: one-line trigger for the new skill. docs/configuration.md: one env-var reference line. .gitignore: config/betterstack-webhook.env.

Risk Assessment

Low-medium. The riskiest change is the shared daemon's enablement gate and env passthrough in fm-clickstack-recv.sh/fm-clickstack-arm.sh, since that file is already live in production serving real ClickStack traffic on port 8111. Mitigated by: the full pre-existing ClickStack test suite passing unmodified, an explicit CSHOOK_ENABLED flag added to the listener so a BetterStack-only daemon cannot accidentally open the ClickStack ingestion path with no secret, and the change being purely additive (no removed code paths).

Testing

  • tests/fm-betterstack.test.sh (new, 14 cases): presence-gate inertness, poll surfacing, dual-gate independence in both directions, both-gates-together, token rejection (missing/wrong/unconfigured), idempotent redelivery including the "in-progress incident update overwrites" semantics, token generation + idempotence, arm's no-restart-when-already-healthy path, the real watcher e2e wake path, bootstrap activate/opt-out. All pass.
  • tests/fm-clickstack.test.sh (pre-existing, unmodified): still fully green, confirming no ClickStack regression.
  • shellcheck bin/*.sh bin/backends/*.sh tests/*.sh (the CI command): clean, exit 0.
  • Live loopback verification with a real BetterStack-shaped incident payload, a rejected no-token request, cross-route 404 isolation, and /healthz - recorded with real output in docs/betterstack-webhook.md "Verification".

Ready-to-paste URL form for the captain (the real token lives only in config/betterstack-webhook.env on the live instance, generated at arm time, never committed):

https://fm.ba.id.au/betterstack?token=<BETTERSTACK_WEBHOOK_TOKEN>

Get the real value with bin/fm-betterstack-arm.sh (first run, generates the token) then bin/fm-betterstack-arm.sh --show-url.

Full narrative / original brief

Epic E5.1: build the BetterStack status-page webhook receiver so the captain can subscribe firstmate to status.teslemetry.com updates, modeled on the existing ClickStack webhook receiver (bin/fm-clickstack-recv.sh + fm-clickstack-arm.sh + fm-clickstack-poll.sh + config/clickstack-webhook.env + docs/clickstack-webhook.md + state/clickstack-inbox/): same architecture - opt-in via a local gitignored config file, listener stashes payloads to a state inbox, a generated check shim surfaces them as check: wakes.

Preference was to add a /betterstack route to the existing receiver process/port (8111; public path via the fm.ba.id.au reverse proxy which forwards the whole host, so a new path needs no proxy change) if that's clean, or a parallel receiver if not, justified in the PR - see "Intent" above for why the shared-process route was the only viable option given the proxy constraint.

Auth: BetterStack status-page webhook subscriptions do not support custom headers, per their own docs, so an unguessable token in the URL query is used and validated server-side.

Wake contract: stashed payloads surface as a check: wake naming betterstack-alert (mirroring clickstack-alert) so firstmate routes them to the oncall secondmate scope. docs/betterstack-webhook.md covers setup, payload notes, and a verification record; a colocated test follows the clickstack test pattern.

Delivered fork-only, never upstreamed.

Extends the existing ClickStack webhook receiver with a /betterstack
path instead of standing up a second daemon: the reverse proxy forwards
the whole host to one port, so a second listener would be unreachable
without an out-of-repo proxy change. Independently gated by its own
config/betterstack-webhook.env, with an unguessable URL-query token
(BetterStack subscriptions cannot send custom headers) generated and
persisted at arm time. Events surface as betterstack-alert check: wakes
through the existing durable wake queue, handled by a new
betterstack-alert-response skill mirroring the ClickStack one.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

if [ "$mode" = arm ] && healthy_receiver; then
echo "clickstack receiver: healthy pid=$HEALTHY_PID (already listening)"
exit 0

P2 Badge Restart when adding ClickStack to a BetterStack-only daemon

Captain, after this change the arm command considers a receiver healthy when either integration has started it, so enabling config/clickstack-webhook.env while a BetterStack-only daemon is already running makes fm-clickstack-arm.sh print healthy and exit without restarting. That existing process was launched with CSHOOK_ENABLED=0, so ClickStack POSTs continue to 404 until someone manually uses --restart; the normal arm path should restart or otherwise verify that the live daemon loaded the ClickStack route.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/fm-betterstack-arm.sh
Comment on lines +90 to +93
if [ "$had_token" = 1 ] && shared_daemon_healthy; then
"$ARM"
else
"$ARM" --restart

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restart when enabling BetterStack on an existing receiver

Captain, if a healthy ClickStack-only receiver is already running and the BetterStack gate is added with a pre-filled BETTERSTACK_WEBHOOK_TOKEN (or the token is supplied via env), this branch treats the receiver as healthy and delegates a plain arm, which exits without restarting the process. Because fm-clickstack-recv.sh only passes BSHOOK_ENABLED/BSHOOK_TOKEN at process start, /betterstack keeps returning 404 even though fm-betterstack-arm.sh reports ready; force a restart when the running shared daemon may not have loaded the BetterStack route.

Useful? React with 👍 / 👎.

@Bre77 Bre77 merged commit 7b5f33e into main Jul 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant