test: add hosted Supabase compatibility checks

[aparajon]

Why

The local Supabase suite cannot establish compatibility with the hosted service. We need repeatable checks against real Auth, Data API, Realtime, and database connections as pg-sprite evolves.

What

Add an opt-in Go test harness for a disposable hosted Supabase project, with setup instructions and an explicit account of what each result proves.

How

• Exercise the built CLI for desired schemas, export/convergence, Auth integration, RLS, and failure/refusal paths
• Keep two real users subscribed through one initialized Realtime fixture, then verify events and tenant isolation after schema changes
• Test TLS verification and certificate rejection; provide an optional session pooler case
• Read credentials from private local files; register table cleanup before creation and remove test tables and Auth users during cleanup

Realtime setup must establish reader readiness, authenticated subscriptions, and actual delivery before any schema change runs. Initialization failures fail the test. No writes are retried or connections restarted.

Risk

No production engine changes. Hosted tests skip unless explicitly enabled and require a disposable project because they create users, tables, and publication entries. Ordinary CI needs no hosted credentials.

Testing

• Review fixes: hosted cleanup, destructive refusal, and RLS create/export/convergence passed with race detection. Without opt-in, the package skips hosted cases without Docker or database requests

• go test -count=1 -timeout=10m -v ./integration/supabase/hosted: all 18 configured compatibility cases passed against hosted Supabase in 149 seconds. Pooler endpoints remain unconfigured

• scripts/test-flaky.sh TestHostedRealtimeContinuity 10 ./integration/supabase/hosted: all ten race-enabled iterations passed, with no retries of failed runs

• After narrowing the scope, go test -race -count=1 -timeout=3m -v ./integration/supabase/hosted -run '^TestHostedRealtimeContinuity$': all six phases passed in 53 seconds

• Read-only cleanup audit: zero remaining test tables or Auth users

Bigger picture

This establishes a hosted regression harness, not blanket Supabase support. Hosted session pooler validation remains a follow-up. Transaction-pooler refusal stays in the controlled local suite because hosted backend reuse makes that assertion nondeterministic. The harness verifies that pg-sprite preserves established Realtime delivery; Supabase startup investigation stays outside this PR.

Generated with Codex (GPT-6)

[Copilot]

Copilot review overview

🟡 Changes recommended

The transaction-pooler assertion is nondeterministic, and CLI-created tables can escape cleanup on post-commit failures.

Review effort: Balanced
Findings: 2

Open (2)

• Register table cleanup before invoking the mutating CLI · New
• Avoid nondeterministic session-affinity assertion on hosted pooling · New

What changed in this PR

Adds an opt-in hosted Supabase regression suite covering CLI schema changes, Auth, RLS, Realtime continuity, TLS, and optional poolers.

Changes:

• Adds reusable hosted-project fixtures and cleanup.
• Exercises successful, refused, failed, and rollback paths.
• Documents setup, guarantees, limitations, and hosted results.

Fixture maintenance note: A separate update PR should evaluate Postgres 17.6.1.136→17.6.1.177, PostgREST 14.17→16.4 (major drift), Auth 2.196.0→2.197.0 (Auth schema additions), Realtime 2.134.10→2.138.3, and Supavisor 2.9.12→2.9.13 (pool throttling changes). Exact Postgres container-tag and digest verification remains incomplete.

File	Description
integration/​supabase/​README.md	Links the hosted suite.
integration/​supabase/​hosted/​README.md	Documents setup, scope, and evidence.
integration/​supabase/​hosted/​fixture_test.go	Provides hosted users, tables, API access, and cleanup.
integration/​supabase/​hosted/​auth_test.go	Tests Auth defaults, foreign keys, and index failures.
integration/​supabase/​hosted/​ddl_test.go	Provides schema-change and refusal helpers.
integration/​supabase/​hosted/​lifecycle_test.go	Tests CLI creation, export, columns, and indexes.
integration/​supabase/​hosted/​lifecycle_failure_test.go	Tests refusal and failure preservation.
integration/​supabase/​hosted/​pooler_test.go	Adds optional pooler checks.
integration/​supabase/​hosted/​realtime_test.go	Implements authenticated Realtime streams.
integration/​supabase/​hosted/​realtime_lifecycle_test.go	Verifies Realtime continuity across changes.
integration/​supabase/​hosted/​rls_test.go	Tests declarative RLS behavior.
integration/​supabase/​hosted/​rls_rollback_test.go	Tests atomic RLS cancellation.
integration/​supabase/​hosted/​tls_test.go	Tests encryption and certificate rejection.
docs/​supabase.md	Updates hosted compatibility guidance.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Kiran01bm]

🤖 Review findings - created by Kiran's code review agent - for pg-sprite/pull/129, e770605.

Verdict: 8 findings — 5 non-blocking (README/doc accuracy, weak refusal assertions), 3 suggestions.

Non-blocking

1. The README says the hosted package needs no Docker, but TestTableCleanup always starts PostgreSQL. README.md:44 says "No Docker services ... are used" and that, with the opt-in unset, the package makes no database requests. cleanup_test.go:34 calls testutil.StartPostgres(t) with no opt-in check, so the documented full-suite command fails on a machine without Docker, or creates and drops public.cleanup_test on whatever PG_DSN points at. Gate the test on the opt-in, move it out of the package, or fix the README.

2. The destructive-refusal test checks only exit code 2, and every refusal exits 2. lifecycle_failure_test.go:43 throws away the CLI result, and main.go sends every verdict.ErrRefused to the same exit code. A refusal for backend-unavailable, privileges or review-required would still pass this test and count as proof for the README row. Pin reason the way the local failure_test.go:64 does; the RLS dry-run at lifecycle_test.go:83 has the same gap.

3. The convergence check matches human-facing detail text. lifecycle_test.go:89 asserts "already converged" in the prose from migrate_row_security.go:44. AGENTS.md:200 forbids asserting on human-facing wording, and outcome is never asserted. Assert outcome == executed with empty executed_sql instead.

4. docs/supabase.md contradicts itself about hosted RLS evidence. supabase.md:197 still says "Hosted validation remains a separate step". Line 270 now says hosted checks passed for atomic RLS rollback and the RLS preview/apply path.

5. The RLS design doc still lists hosted validation as an open follow-up (plausible). declarative-row-security.md:282 was not updated for the hosted RLS cases this PR adds. It is only partly stale: every hosted case runs as the owning postgres role, so a non-owner privilege check is still missing. Record the owner-role evidence and keep the non-owner gap open.

General suggestions

6. The flake-validation procedure passes when every iteration is skipped. README.md:142 treats "PASSED all 10 iterations" as proof, but scripts/test-flaky.sh runs go test without -v and does not check whether the test ran. If SUPABASE_HOSTED_TEST is unset, all 10 runs skip and the script prints the same success line. Say in the README that the opt-in must be exported, or make the script fail on skips.

7. Two comments will go stale. realtime_lifecycle_test.go:20 ("observed at ~60s") and line 13 ("the unchanged 30s delivery deadline") repeat values that belong to constants and refer to an earlier revision. .agents/checks/review.md asks reviewers to flag both.

8. The pooler project check runs only after hosted fixtures exist. pooler_test.go:21 creates Auth users, a table, a policy and grants before checkPoolerProject rejects a session URL for the wrong project. The check reads only SUPABASE_HOSTED_URL, so it can run before newFixture. Cleanup removes the fixtures, so the impact is low.

The one thing that could have broken, verified

The RLS cancellation test depends on its tracer matching the exact live DROP POLICY and on the policy being restored after the cancel. The tracer text is byte-identical to what row_security.go:136 sends, since both sanitize pgx.Identifier{schema, table}. After the cancel, the next tx.Exec fails, and rowSecurityError wraps caller.Err(), so require.ErrorIs(err, context.Canceled) holds. The deferred rollback uses context.WithoutCancel, so the policy is restored.

Verified correct

• Cleanup runs in LIFO order: the table with the FK to auth.users is dropped before those users are deleted.
• The RLS lock holder rolls back before DROP TABLE, so a failed assertion cannot leave the drop blocked.
• The heartbeat goroutine has a stop-and-join path and is the only websocket writer after the join frame.
• Each stream.await has a fixed 30s read deadline, so heartbeat replies that don't match cannot extend the wait.
• dbconn.NewPool pings eagerly, and verify-full uses ServerName=host, so the TLS rejection cases are sound.
• Every CLI JSON field the tests read exists on the wire types, and exit codes 2 and 1 match cmd/pg-sprite/main.go.

This review was generated by Claude Code (claude-opus-5).

[Kiran01bm]

🤖 Approved on Kiran's (@kmuddukrishna) behalf by the scheduled review agent — no blocking findings at e770605. See the review comment above; non-blocking findings and suggestions, if any, are not merge gates.

[aparajon]

🤖 Addressed all eight findings in 1bbae42 and 0076f0f.

• Cleanup coverage now uses the hosted opt-in and project validation, with random fixture names. No Docker or PG_DSN fallback
• Destructive preview/apply assert destructive-change; RLS preview asserts unsupported-statement and includes the structured RLS review
• Convergence asserts the typed verdict.OutcomeExecuted and empty executed SQL. Its wire value is executed-natively, rather than executed
• Both RLS guides record hosted owner-role evidence and leave non-owner privilege validation open
• The flake procedure checks the exported opt-in and explains why skipped iterations are not evidence
• Removed historical timing language from comments
• Pooler project validation runs before creating users or tables

Focused hosted checks passed with race detection: cleanup, destructive refusal, and RLS creation/export/convergence. The no-opt-in package check and required lint/pre-push hooks also passed. The hosted session endpoint remains unconfigured and untested.

— Codex (GPT-6)