v3.0.0: Outpost support, Gateway MCP write mode, and the gateway_ tool rename

.agents/skills/hookdeck-cli-manual-qa/SKILL.md

@@ -0,0 +1,129 @@
+---
+name: hookdeck-cli-manual-qa
+description: >-
+  Runs exploratory manual QA of the Hookdeck CLI and its MCP servers against
+  real acceptance-test projects, safely. Provides a credential guard that keeps
+  destructive commands off any project not named up front and never touches the
+  operator's own login, an MCP stdio driver, and per-surface checklists. Use
+  when manually testing CLI commands or MCP tools, exercising write paths,
+  validating a release candidate, or reproducing a defect against the live API.
+---
+
+# Hookdeck CLI — manual QA
+
+Exploratory testing against the live API, in acceptance-test projects, without
+putting anything else at risk.
+
+This exists because the automated suites cannot cover everything. They run in
+fixed shapes; a QA pass goes looking for the things nobody thought to assert —
+wrong success messages, defaults that never get sent, filters that silently do
+nothing. Every defect this process has found so far was of one kind: **a wrong
+answer that reads like a right one.** Look for those specifically.
+
+## Safety model — read before running anything
+
+Manual QA runs destructive commands against real projects. Two rules make that
+survivable, and both are enforced by [scripts/hd-testenv.sh](scripts/hd-testenv.sh):
+
+1. **Never use the default config.** Every command takes
+   `--hookdeck-config "$HD_CONFIG"`. The operator's own `hookdeck login` is
+   never read or overwritten. This is not hypothetical — a suite run without an
+   isolated config has previously written CI credentials over a working login.
+2. **The project is confirmed from the credential, not from the argument.** You
+   name the project you expect up front; the guard authenticates, resolves the
+   project, and refuses to hand back a config if it is anything else or is
+   outside the acceptance-test organisation.
+
+Credentials come only from `test/acceptance/.env` (gitignored, CI test keys), so
+the reachable blast radius is the test projects those keys can see.
+
+**Before any delete, call `hd_assert "$HD_PROJECT"` and abort on failure.** It is
+one cheap line and it catches a config swapped or a project switched mid-run.
+
+If the guard refuses, **stop and read the message**. It fails closed on purpose;
+working around it defeats the point.
+
+### Setup
+
+```bash
+source .agents/skills/hookdeck-cli-manual-qa/scripts/hd-testenv.sh
+hd_testenv HOOKDECK_CLI_TESTING_API_KEY tm_xxxxxxxxxxxx          # gateway project
+hd_testenv HOOKDECK_CLI_OUTPOST_TESTING_API_KEY tm_yyyyyyyyyyyy  # outpost project
+
+go build -o hookdeck .            # test the build, not `go run` each time
+./hookdeck --hookdeck-config "$HD_CONFIG" whoami
+```
+
+**Rebuild before every pass, and after every fix.** A stale binary reports the
+old behaviour convincingly: missing tools, missing flags, old names. If
+something looks broken in a way that seems too fundamental to be true, check
+that first — it has already produced one false finding.
+
+Ask the maintainer which project IDs to use if you do not already know them;
+they are deliberately not hardcoded here.
+
+## Running a pass
+
+Two independent surfaces. Run them as separate agents when doing a full sweep —
+they do not share state beyond the projects, and one must not wait on the other.
+
+- **CLI surface** — [references/cli-checklist.md](references/cli-checklist.md)
+- **MCP surface** — [references/mcp-checklist.md](references/mcp-checklist.md)
+
+For MCP, drive the server with [scripts/mcp-call.py](scripts/mcp-call.py) rather
+than wiring up an editor:
+
+```bash
+python3 .agents/skills/hookdeck-cli-manual-qa/scripts/mcp-call.py \
+  --config "$HD_CONFIG" --server gateway --list
+```
+
+## What to look for
+
+Ordinary "does it work" testing finds little at this point. These are the
+patterns that have actually yielded defects:
+
+- **Does the message match the state?** Run the command, then read the resource
+  back independently. `enable` printing "enabled" proves nothing; a subsequent
+  `get` showing it enabled does. Several commands announced the verb they were
+  asked to perform rather than the state the API returned.
+- **Does an omitted flag mean what the help says?** Documented defaults are not
+  necessarily sent. A destination type whose schema says `tls` defaults to on
+  was being created with `tls` unset.
+- **Can you undo it?** Setting a value is usually tested; clearing it is not.
+  `--filter '{}'` reported success and changed nothing because `omitempty`
+  dropped it before it reached the wire.
+- **Does an error exit non-zero?** An endpoint that answers `200` with
+  `{"success": false}`, or a body carrying its own status field, will be
+  reported as success by anything that only checks the HTTP status.
+- **Does a filter actually filter?** Pass one and confirm the result set
+  narrows. A silently ignored filter returns plausible data that is wrong.
+- **Do the read and write paths agree?** A fix applied to a CLI command and not
+  its MCP sibling leaves the defect open on the surface with no human watching.
+  Check both.
+
+When something looks wrong, **verify it against the live API before reporting
+it** — construct the minimal call that distinguishes the two explanations. A
+report that says "verified: log_level is severity, not a completion flag, here
+are the five cases" is actionable; "this looks suspicious" costs someone else
+the same investigation.
+
+## Clean up
+
+Acceptance-test projects accumulate resources fast, and a QA pass adds to it.
+
+- Name everything you create with a run-specific prefix (`qa-<date>-<n>`) so it
+  can be found and removed later.
+- Delete what you created before finishing, in reverse order of creation.
+  Deleting a tenant cascades to its destinations.
+- If you leave anything behind, say so explicitly in the report with the names
+  and the reason — silent residue is indistinguishable from a leak.
+
+## Reporting
+
+Return findings ranked by severity, each with: what you ran, what you expected,
+what happened, and the evidence. Separate **confirmed defects** from **things
+worth a look** — mixing them makes the confirmed ones cheaper to ignore.
+
+State plainly what you did **not** cover. A pass that reports only successes is
+usually a pass that did not go looking.

.agents/skills/hookdeck-cli-manual-qa/references/cli-checklist.md

@@ -0,0 +1,80 @@
+# CLI surface — manual QA checklist
+
+Read [../SKILL.md](../SKILL.md) first, including the safety model. Every command
+below takes `--hookdeck-config "$HD_CONFIG"`.
+
+Two projects, two credentials:
+
+| Surface | Key in `test/acceptance/.env` |
+|---|---|
+| Gateway (`hookdeck gateway …`) | `HOOKDECK_CLI_TESTING_API_KEY` |
+| Outpost (`hookdeck outpost …`) | `HOOKDECK_CLI_OUTPOST_TESTING_API_KEY` |
+
+This is a checklist of **surfaces**, not a script. The value is in what you try
+against each one, so vary it between runs rather than replaying the same calls.
+
+## Gateway resources
+
+`source` · `destination` · `connection` · `event` · `request` · `attempt` ·
+`transformation` · `issue` · `metrics`
+
+For each resource that supports them, exercise `list`, `get`, `create`,
+`update`, `upsert`, `delete`, and any state verbs (`enable`/`disable`,
+`pause`/`unpause`, `retry`/`cancel`/`mute`, `dismiss`).
+
+Beyond the happy path:
+
+- [ ] **State verbs report the resulting state.** After every `enable`,
+      `disable`, `pause`, `cancel`, `mute`, run a separate `get` and confirm the
+      resource actually changed. Repeat the verb on a resource already in that
+      state and check the second call does not claim to have changed anything.
+- [ ] **`retry` on a body carrying its own status.** Confirm a declined retry
+      exits non-zero rather than printing a tick.
+- [ ] **Filters narrow the result set.** For every list filter (`--status`,
+      `--source-id`, `--search-term`, date ranges), compare filtered and
+      unfiltered counts. A filter the API ignores returns plausible wrong data.
+      `--search-term` matches a whole field value, not a substring — confirm
+      that is still true and that the help says so.
+- [ ] **Pagination.** `--next` / `--prev` across a boundary; confirm no repeats
+      and no skips.
+- [ ] **`transformation run`.** A handler that returns a request; one that calls
+      `console.error` and still returns (must succeed, exit 0, and show the
+      console output); one that throws; one that returns nothing (must fail).
+- [ ] **Output modes.** `--output json` parses as JSON on both success and
+      failure paths, and the exit code matches the outcome in both.
+
+## Outpost resources
+
+`tenant` · `destination` · `destination-type` · `event` · `attempt` · `topic` ·
+`config` · `metrics` · `publish` · `status`
+
+- [ ] **Destination types beyond `webhook`.** Create one of each supported type
+      (AWS, GCP, Azure, RabbitMQ, Kafka) and confirm the stored config matches
+      what the schema declares — particularly any field with a default, which
+      must be sent rather than left unset.
+- [ ] **Credentials are masked on read, except where the platform generates
+      them.** Webhook secrets are produced server-side and must come back;
+      caller-supplied credentials must not.
+- [ ] **Clearing works, not just setting.** `--filter '{}'`, `--unset` on
+      config, removing a topic. Read the resource back to confirm.
+- [ ] **`config set`.** Round-trip a value and restore it; confirm `--dry-run`
+      changes nothing and that a real set is visible in `config get`.
+- [ ] **`tenant portal`** returns a URL, and `--theme light|dark` is accepted.
+      Do not test `--open`; it launches a browser.
+- [ ] **`publish`** an event and follow it through `event list` → `attempt list`.
+
+## Cross-cutting
+
+- [ ] **Unknown flags and unknown enum values** are rejected with a message that
+      names the offending value.
+- [ ] **Missing required arguments** fail before any network call.
+- [ ] **A resource id containing `/` or `..`** is rejected as an invalid
+      identifier rather than addressing a different resource.
+- [ ] **`--help` matches behaviour** — documented defaults are actually applied,
+      documented flags exist, and `REFERENCE.md` is current
+      (`go run ./tools/generate-reference --check`).
+
+## Cleanup
+
+Delete every resource created, in reverse order. Deleting an Outpost tenant
+cascades to its destinations. Report anything left behind by name.

.agents/skills/hookdeck-cli-manual-qa/references/mcp-checklist.md

@@ -0,0 +1,64 @@
+# MCP surface — manual QA checklist
+
+Read [../SKILL.md](../SKILL.md) first, including the safety model.
+
+Two servers: `hookdeck gateway mcp` and `hookdeck outpost mcp`. Drive them with
+[../scripts/mcp-call.py](../scripts/mcp-call.py).
+
+```bash
+QA=.agents/skills/hookdeck-cli-manual-qa/scripts/mcp-call.py
+python3 $QA --config "$HD_CONFIG" --server gateway --list
+python3 $QA --config "$HD_CONFIG" --server gateway --allow-write --list
+```
+
+The MCP surface is where defects hide, because no human reads its output. A
+wrong answer here is consumed directly by an agent that will act on it.
+
+## Mode gating
+
+- [ ] **Read-only is the default.** With no flag, `tools/list` advertises no
+      write actions and the descriptions do not mention them.
+- [ ] **`--allow-write` adds them**, and `HOOKDECK_MCP_ALLOW_WRITE` does the
+      same. `--read-only` wins when both are given.
+- [ ] **The guard is real, not cosmetic.** Call a write action without
+      `--allow-write` and confirm it is refused — a tool hidden from the listing
+      but still dispatchable is not gated.
+- [ ] **The refusal explains how to enable it**, naming the help tool.
+- [ ] **`*_help` reports the current mode** and matches the actual listing.
+- [ ] **Actions that change state but stay available read-only** (connection
+      `pause`/`unpause`) still work without the flag. That is deliberate —
+      incident response — and its tests are the proof it was not an oversight.
+- [ ] **Annotations are honest.** `readOnlyHint` false for anything that
+      changes state; `destructiveHint` true for deletes, cancel, mute, dismiss.
+
+## Per tool
+
+Call every action of every tool, in both modes. For each:
+
+- [ ] **Unknown arguments are rejected**, and the message distinguishes "no such
+      argument" from "that argument needs write mode".
+- [ ] **A failure is reported as one.** Check `isError`, not just that a
+      response came back. An envelope whose body says the operation was declined
+      while the result reads as success is the defect to hunt.
+- [ ] **No hardcoded status fields.** Anything reporting `"status": "queued"`,
+      `"deleted"`, or similar must reflect what the API returned, not what the
+      handler intended.
+- [ ] **Write actions are equivalent to their CLI siblings.** Create a resource
+      both ways with the same inputs and diff the stored result. Schema defaults,
+      required-field fallbacks and filter handling have diverged between the two
+      before — a fix applied to one surface and not the other.
+- [ ] **Credential masking** matches the CLI: platform-generated secrets
+      returned, caller-supplied credentials masked.
+
+## Payloads
+
+- [ ] **Raw body tools** (`*_events raw_body`, `*_requests raw_body`) return the
+      body unmodified, including non-JSON and large payloads.
+- [ ] **List filters narrow the result.** Same test as the CLI: compare against
+      unfiltered. An ignored filter is worse here — the agent cannot tell.
+- [ ] **Pagination** is expressible and terminates.
+
+## Cleanup
+
+Anything created through MCP is as real as anything created through the CLI.
+Delete it, and confirm the delete with an independent `get`.

.agents/skills/hookdeck-cli-manual-qa/scripts/hd-testenv.sh

@@ -0,0 +1,106 @@
+#!/usr/bin/env bash
+# Set up an isolated CLI config for one acceptance-test project, and refuse to
+# hand it back unless the credential really resolves to that project.
+#
+#   source hd-testenv.sh <ENV_VAR_NAME> <expected tm_ id>
+#
+# On success exports HD_CONFIG. Use it on EVERY command:
+#   go run . --hookdeck-config "$HD_CONFIG" gateway source list
+#
+# Two things this guarantees:
+#
+#   1. The default config (~/.config/hookdeck/config.toml) is never read or
+#      written, so a QA run cannot clobber the operator's own login. This has
+#      happened: a suite run without an isolated config wrote CI credentials
+#      over a working one.
+#   2. A destructive command cannot run against a project that was not named up
+#      front. The project is confirmed from the credential itself, not from the
+#      argument, so a mistyped or rotated key fails closed rather than pointing
+#      at something unexpected.
+#
+# Safety here comes from where the credential lives, not from a hardcoded list
+# of project IDs. Keys must come from test/acceptance/.env, which is gitignored
+# and holds CI test keys, so the reachable blast radius is the test projects
+# those keys can see. A list of IDs in a public repo would rot and would leak
+# identifiers for no gain.
+# Deliberately no `set -u` / `set -e` at this level: this file is sourced, so
+# those would apply to the caller's interactive shell and break it. The
+# functions below use explicit defaults instead.
+
+# Organisation every acceptance-test project belongs to. Override if your test
+# projects live elsewhere.
+ALLOWED_ORG="${HOOKDECK_QA_ALLOWED_ORG:-Automated Testing}"
+
+# Resolve the repo from the working directory rather than from the script's own
+# path: BASH_SOURCE does not exist under zsh, which is where this is most often
+# sourced from, and a wrong answer here silently looks for .env in the wrong place.
+_hd_repo() {
+  if [ -n "${HOOKDECK_QA_REPO:-}" ]; then
+    printf '%s\n' "$HOOKDECK_QA_REPO"
+    return 0
+  fi
+  git rev-parse --show-toplevel 2>/dev/null && return 0
+  printf '%s\n' "$PWD"
+}
+
+hd_testenv() {
+  local var="$1" expected="$2"
+  local repo scratch
+  repo="$(_hd_repo)"
+  scratch="${HOOKDECK_QA_SCRATCH:-${TMPDIR:-/tmp}/hookdeck-qa}"
+  mkdir -p "$scratch"
+
+  case "$expected" in
+    tm_*) ;;
+    *) echo "REFUSING: $expected does not look like a project id" >&2; return 1 ;;
+  esac
+
+  local envfile="$repo/test/acceptance/.env"
+  [ -f "$envfile" ] || { echo "REFUSING: $envfile not found — QA keys must come from there" >&2; return 1; }
+  set -a; . "$envfile"; set +a
+
+  local key
+  eval "key=\${$var:-}"
+  [ -n "$key" ] || { echo "REFUSING: $var is not set in test/acceptance/.env" >&2; return 1; }
+
+  # Authenticate into a scratch path and move it into place only on success.
+  # Wiping the target first meant a failed attempt destroyed a config that had
+  # been working, and the next run then failed for an unrelated reason.
+  local cfg="$scratch/hdcfg-$expected.toml"
+  # The extension has to stay .toml: the config format is inferred from it, and
+  # a temp name ending in the PID fails with Unsupported Config Type.
+  local tmp="$scratch/hdcfg-$expected.new-$$.toml"
+  rm -f "$tmp"
+  (cd "$repo" && go run . --hookdeck-config "$tmp" ci --api-key "$key" >/dev/null 2>&1) \
+    || { rm -f "$tmp"; echo "REFUSING: could not authenticate with $var (existing config left alone)" >&2; return 1; }
+  mv -f "$tmp" "$cfg"
+
+  # Verify against the credential, not against what we hoped it was.
+  local who actual
+  who=$(cd "$repo" && go run . --hookdeck-config "$cfg" whoami 2>&1)
+  actual=$(grep -oE 'tm_[A-Za-z0-9]+' "$cfg" | head -1)
+
+  if [ "$actual" != "$expected" ]; then
+    echo "REFUSING: config resolved to $actual, expected $expected" >&2
+    rm -f "$cfg"; return 1
+  fi
+  if ! grep -qF "$ALLOWED_ORG" <<<"$who"; then
+    echo "REFUSING: not the $ALLOWED_ORG organization:" >&2
+    echo "$who" >&2
+    rm -f "$cfg"; return 1
+  fi
+
+  export HD_CONFIG="$cfg"
+  export HD_PROJECT="$expected"
+  echo "OK: $expected — $(grep -oE 'on project .*' <<<"$who")"
+}
+
+# Re-assert before anything destructive. Cheap, and it catches a config that was
+# swapped or a project switched mid-run. Call it immediately before any delete,
+# and let a non-zero return abort the step.
+hd_assert() {
+  local expected="$1" actual
+  [ -n "${HD_CONFIG:-}" ] || { echo "ABORT: HD_CONFIG is not set — run hd_testenv first" >&2; return 1; }
+  actual=$(grep -oE 'tm_[A-Za-z0-9]+' "$HD_CONFIG" | head -1)
+  [ "$actual" = "$expected" ] || { echo "ABORT: $HD_CONFIG points at $actual, not $expected" >&2; return 1; }
+}