Test the nodes against a real Hookdeck project - #8
Merged
Conversation
The unit suite drives the nodes against a stub, so it can prove what a node sends but never what Hookdeck does with it. These four suites close that gap: they provision real resources, put real events through the public ingest URL, and feed the resulting requests into the node's own webhook handler. api every provisioning option and all 29 action operations verification Stripe and GitHub signatures, signed and forged delivery real events over `hookdeck listen`, retries, dedup, pause stripe a genuine Stripe webhook, endpoint created and torn down These sit alongside test/integration.test.mjs rather than replacing it: that suite covers source adoption and the CLI destination, which these do not. Each skips with a stated reason when its prerequisites are absent, so the default test run needs no credentials. They run serially, because count assertions are project-wide and cannot be isolated by naming. Running them corrected several assumptions. A platform source reveals nothing about whether it verifies, while a generic one exposes config.auth_type. A forged payload is answered 200 by Stripe and refused outright by GitHub, so the status code is not the verdict in either direction. An automatic retry reports AUTOMATIC, never RETRY. Real Stripe signs with three schemes, not the documented two. Delivery groups are a plan entitlement, so that case skips rather than passing quietly. The live suites run against a project that also carries production sources, so cleanup deletes only names carrying the run's own id — including the one the node derives from the workflow id — and reports anything it could not remove instead of swallowing the failure. `hookdeck listen` is spawned detached and killed by process group: the `hookdeck` on PATH is an npm wrapper around the real binary, and signalling only the wrapper leaves that binary holding the connection open. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
garethx
force-pushed
the
test/live-suites
branch
from
August 13, 2026 12:12
19c6632 to
daa577a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds live test suites that exercise the built nodes against a real Hookdeck
project, so the README's claims are backed by measurements rather than by the
unit suite's stubs.
The unit suite proves what the node sends. These prove what Hookdeck does
with it — which is where every finding below came from.
test/integration.test.mjsmoves in here too, astest/live/provisioning.test.mjs. The repo had two live-test conventionscovering different ground, each with its own API client, node contexts and
cleanup; that split is now legible rather than accidental. CI runs the whole
test/live/directory in place oftest:integration.Suites
api.test.mjsverification.test.mjsdelivery.test.mjshookdeck listen, output shape, retry metadata, dedup, pause/resumehookdeckCLIprovisioning.test.mjsstripe.test.mjsstripe loginAll skip with a stated reason when prerequisites are missing, so forks and the
default
npm testare unaffected. The delivery suite now checks for the HookdeckCLI on PATH rather than spawning it blind — CI installs neither CLI, so it would
otherwise fail with
ENOENTand read like a broken node.Moving the integration suite also brought it under the delete guard. Its
resources were previously removed by an exact-name match, which would have
deleted a same-named production source; they now carry the run id like
everything else.
What running them turned up
STRIPEsource with a secret and one without return byte-identical config.Generic
WEBHOOKsources do exposeconfig.auth_type. So the README's"looks identical to an unconfigured one" is right for platform sources and
wrong for generic ones.
STRIPEanswers 200 withverified: falseandrejection_cause: VERIFICATION_FAILED;GITHUBrefusesoutright. Never infer verification from the status code, in either direction.
attemptTriggerisINITIAL/AUTOMATIC/MANUAL/BULK_RETRY/UNPAUSE—never
RETRY. Anything branching on'RETRY'never fires.t,v1,v0— three schemes, where the documentedexample shows two.
upsert is rejected. That case skips with the reason rather than passing quietly.
existing source untouched, so applying a secret to one that already exists
needs
updateExistingSource. Without it the secret is accepted, silentlydiscarded, and deliveries arrive unverified — caught by the Stripe suite
against a real endpoint.
destination.config.rate_limit; deleted resourcesanswer
410, not404; there is no/events/count.Safety
These run against a project that also holds production sources, so
destroy()refuses to delete anything not carrying the run's own id, across all three naming
forms — including the one the node derives from the workflow id, which was
missed at first and left resources behind. Cleanup failures are reported loudly
rather than swallowed.
hookdeck listenis spawned detached and killed by process group: thehookdeckon
PATHis an npm wrapper around the real binary, and signalling only thewrapper leaves that binary holding its connection open and the parent process
alive. Fixing that took one suite from a 600s timeout to 64s.
Verification
Rebased on current
main.npm run test:live— 45 pass, 1 skip (deliverygroups, not enabled on the test org), exit 0, no leaked resources, no orphaned
processes.
npm test106 pass. Skip paths checked directly: no API key skipsevery suite, and a missing CLI skips delivery and Stripe.
Unrelated and left alone: both nodes now trip
icon-prefer-themed-variants— the icon is a singlefile:hookdeck.svgwhere itwas previously the
{ light, dark }form, and onlyhookdeck.svgremains ondisk. That is a verification-programme rule, so worth a separate fix.