Skip to content

Charge AI training crawlers a day pass (x402 crawl gateway 0.11) - #11

Merged
ralyodio merged 1 commit into
mainfrom
feat/crawl-gateway
Oct 21, 2026
Merged

ralyodio merged 1 commit into
mainfrom
feat/crawl-gateway

Conversation

@ralyodio

Copy link
Copy Markdown
Contributor

Wires @profullstack/x402-gateway ^0.11.0 (Hono middleware) into buildApp, first after the header middleware.

  • GPTBot and other training crawlers get 402 with PAYMENT-REQUIRED; /crawl sells the $1 day pass (USDC on Base/Polygon/Ethereum, or the CoinPay Buy a day pass checkout). The page CSP is set per page, so the gateway's pages are not under form-action 'self'.
  • Browsers, verified search crawlers, AI chat fetchers and sf_session holders are never charged; /healthz, /readyz, /install.sh, /llms.txt, /api/, /mcp and /oauth/ are never metered.
  • Off without COINPAY_X402_KEY + CRAWL_PAY_TO (now in vault submitfleet--prod). pnpm 11 added a minimumReleaseAgeExclude entry for the new package.
  • test/crawl-gate.test.ts covers GPTBot 402, the /crawl offer, browser/API/health passing.

🤖 Generated with Claude Code

GPTBot and other training crawlers get a 402 offer with PAYMENT-REQUIRED and
/crawl sells the fleet day pass (x402 USDC or CoinPay checkout). People,
verified search crawlers, AI chat fetchers and signed-in users are never
charged; health, the installer, /api/, /mcp and /oauth/ are never metered.
Off without COINPAY_X402_KEY and CRAWL_PAY_TO.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ralyodio
ralyodio merged commit 36580e7 into main Oct 21, 2026
1 check passed
@ralyodio
ralyodio deleted the feat/crawl-gateway branch October 21, 2026 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant