From 7b7869acd475b640870c6a4470f3590bc1546227 Mon Sep 17 00:00:00 2001 From: Fluory Date: Sat, 10 Oct 2026 01:38:20 +0200 Subject: [PATCH] ops: scheduled uptime monitor for /api/health of the showcase (#110) Why: nothing watched the health endpoint; a down showcase went unnoticed. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/uptime.yml | 64 ++++++++++++++++++++++++++++++++++++ CHANGELOG.md | 3 ++ docs/technical/operations.md | 20 +++++++++++ 3 files changed, 87 insertions(+) create mode 100644 .github/workflows/uptime.yml diff --git a/.github/workflows/uptime.yml b/.github/workflows/uptime.yml new file mode 100644 index 0000000..ea03170 --- /dev/null +++ b/.github/workflows/uptime.yml @@ -0,0 +1,64 @@ +# External uptime monitor for the showcase (#110). Calls GET /api/health every 15 minutes from outside +# the deployment. A red run is the alert: GitHub notifies the repository owner about failed workflow runs. +# Fails on: timeout, non-200, or status != "ok". `dependencies.aiService` "starting" (cold start of the +# scaled-to-zero AI container) is only reported, never a failure. See docs/technical/operations.md. + +name: Uptime + +on: + schedule: + - cron: "*/15 * * * *" + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: uptime + cancel-in-progress: false + +jobs: + health: + runs-on: ubuntu-latest + timeout-minutes: 3 + env: + SHOWCASE_URL: ${{ vars.SHOWCASE_URL || 'https://requestflow-doc-edit.vercel.app' }} + steps: + - name: Check /api/health + run: | + set -uo pipefail + out="$(mktemp)" + code="$(curl -sS --max-time 20 --retry 2 --retry-delay 5 --retry-all-errors \ + -o "$out" -w '%{http_code}' "${SHOWCASE_URL%/}/api/health")" || code="000" + { + echo "### Showcase health" + echo + echo "HTTP status: \`$code\`" + } >> "$GITHUB_STEP_SUMMARY" + if [ "$code" != "200" ]; then + echo "::error::/api/health answered HTTP $code (000 = timeout or no connection)" + echo "Result: **DOWN**" >> "$GITHUB_STEP_SUMMARY" + exit 1 + fi + if ! jq -e . "$out" > /dev/null 2>&1; then + echo "::error::/api/health did not return JSON" + echo "Result: **DOWN** (no JSON)" >> "$GITHUB_STEP_SUMMARY" + exit 1 + fi + { + echo + echo "| Figure | Value |" + echo "|---|---|" + jq -r '"| status | \(.status) |", + "| database | \(.checks.database // "n/a") |", + "| storage | \(.checks.storage // "n/a") |", + "| aiService | \(.dependencies.aiService // "n/a") |", + "| waiting: request-process | \(.backlog["request-process"] // "n/a") |", + "| waiting: request-export | \(.backlog["request-export"] // "n/a") |"' "$out" + } | tee -a "$GITHUB_STEP_SUMMARY" + if [ "$(jq -r '.status' "$out")" != "ok" ]; then + echo "::error::/api/health status is not ok" + echo "Result: **DEGRADED**" >> "$GITHUB_STEP_SUMMARY" + exit 1 + fi + echo "Result: **OK**" >> "$GITHUB_STEP_SUMMARY" diff --git a/CHANGELOG.md b/CHANGELOG.md index 524bf74..b19ef7f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,9 @@ This file records what changes **in the product** – process and session state architecture map (#115). ### Added +- Uptime monitor for the showcase: a scheduled GitHub Actions workflow calls `/api/health` every 15 minutes and turns red + on timeout, non-200 or `status != "ok"` (the owner gets GitHub's failed-run notification); database, storage, + AI service and waiting jobs appear in the job summary; runbook in `docs/technical/operations.md` (#110). - Public case study page `/case-study` (no login, static content, no data access): problem and target group, what the AI does and what the person decides, the four steps up to the exactly-once ERP export, screenshots of the prepared samples and the limits; linked from the start page for visitors without an account (#75). diff --git a/docs/technical/operations.md b/docs/technical/operations.md index 1ca88a2..8b03204 100644 --- a/docs/technical/operations.md +++ b/docs/technical/operations.md @@ -140,6 +140,26 @@ company still ends up without one, an operator re-invites an admin with `bootstr endpoints are disabled except `set-active` (404) – every change goes through the audited module. The `user.invited` audit event stores the role only; the invitation id points to the e-mail. +## Uptime monitor (showcase) + +| Item | Value | +|---|---| +| Monitor | GitHub Actions workflow `.github/workflows/uptime.yml` – calls `GET /api/health` from outside the deployment | +| Target | repository variable `SHOWCASE_URL`, default `https://requestflow-doc-edit.vercel.app` | +| Interval | every 15 minutes (cron; GitHub may delay scheduled runs by a few minutes), plus manual start (`workflow_dispatch`) | +| Check | 20 s timeout, 2 retries; red run on timeout, non-200, non-JSON or `status != "ok"` | +| Alert recipient | role Orchestrator – GitHub's failed-workflow notification to the repository owner (check the notification settings: Actions → "Send notifications for failed workflows only"); no address is stored in the repo | +| Key figures | job summary of every run: `database`, `storage`, `aiService`, waiting jobs `request-process` / `request-export` | + +- `aiService: starting` is the cold start of the scaled-to-zero AI container: shown, never an alert. + `failed` is shown too but does not turn the run red (informational, see "Logs and correlation"). +- On an alert: open the failed run, read the HTTP status and the summary table, then follow + [Frequent failures](#frequent-failures) (`storage`, `config`); HTTP `000`/5xx without a body means the + Vercel deployment itself is down – check the latest deployment, then [Rollback](#rollback). +- Test of the alert: start the workflow with a wrong `SHOWCASE_URL` variable (or pause the target) once and + confirm the notification; reset the variable afterwards. +- Out of scope: error-rate alerts from log aggregation, monitoring of a customer environment. + ## Frequent failures | Symptom | Cause | Action |