Skip to content

feat(web): x402 crawl gateway for training crawlers - #138

Merged
ralyodio merged 1 commit into
mainfrom
feat/crawl-gateway
Oct 21, 2026
Merged

ralyodio merged 1 commit into
mainfrom
feat/crawl-gateway

Conversation

@ralyodio

Copy link
Copy Markdown
Contributor

Wires @profullstack/x402-gateway 0.11 into a new Next 16 proxy.ts (apps/web).

  • GPTBot and other training crawlers: 402 with PAYMENT-REQUIRED
  • /crawl: $1/day pass offer (USDC on Base/Polygon/Ethereum) plus the CoinPay "Buy a day pass" checkout
  • Browsers, verified Googlebot, AI chat fetchers: unchanged
  • Open: /api/ (health, votes, book checkout + webhook, download), /install.sh, /demo.sh, robots, sitemap
  • CSP form-action adds https://coinpayportal.com so /crawl/checkout can redirect to CoinPay
  • Env read at runtime (COINPAY_X402_KEY, CRAWL_PAY_TO), already in dev2 app.env

Test: apps/web/test/crawl-gateway.test.ts. bun test apps/web/test 31 pass, next build shows the proxy.

🤖 Generated with Claude Code

Training crawlers get a 402 with PAYMENT-REQUIRED and /crawl sells a $1 day
pass (x402 USDC or CoinPay checkout). Browsers, verified search crawlers and
AI chat fetchers pass through; /api/ and the install scripts stay open.
form-action now allows coinpayportal.com so the pass checkout can redirect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatednpm/​@​profullstack/​footer@​0.2.1 ⏵ 0.2.578 +4100100 +194 +5100
Addednpm/​@​profullstack/​x402-gateway@​0.11.08810010096100

View full report

@ralyodio
ralyodio merged commit 2724146 into main Oct 21, 2026
16 checks passed
@ralyodio
ralyodio deleted the feat/crawl-gateway branch October 21, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant