diff --git a/docs/v3/02_PRODUCTION_ROLLOUT_2026-08-30.md b/docs/v3/02_PRODUCTION_ROLLOUT_2026-08-30.md new file mode 100644 index 0000000..a9b0319 --- /dev/null +++ b/docs/v3/02_PRODUCTION_ROLLOUT_2026-08-30.md @@ -0,0 +1,104 @@ +# PayGate v3 production rollout — 2026-08-30 + +## Scope + +This record captures the gated production rollout of the PayGate v3 API/operator platform, customer checkout, and Android relay/operator app. + +Production services: +- API: `main-payment-17aqux` +- Customer frontend: `main-payment-frontend-t1n1x8` +- API domain: `https://pay.mulearnscet.in` +- Customer domains: `https://payment.mulearnscet.in` and `https://pay.ieeesahrdaya.com` + +## Released source and images + +- API reviewed source tree: `67bcc8f5e2d178aca251bc0dc26f72b1326909dd` +- API canonical main merge: `9acce4fc93090c9a7fa245f7a20c8d446e36fef5` +- Both commits have identical Git trees: `02f64e29556fa180e40e9744718f3f3ed43ee331` +- API production image: `main-payment-17aqux:v3-67bcc8f5` +- Customer frontend merge: `e08d599a23b901a640d1be228d1d66dc44465cad` +- Customer production image: `main-payment-frontend-t1n1x8:v3-e08d599a` +- Android main merge: `0bcf2156d6ee7bd4afc18b423005f8b921eabb27` +- Android release: `0.4.0-alpha1`, versionCode `8` + +## Backup and rollback gates + +Before the v3 API deployment, PayGate created a fresh production backup through its own backup command: + +- backup: `paygate_manual_20260830_071509.zip` +- size: `3815559` bytes +- exported checksum: verified +- restore drill: passed for both active database files + +Rollback metadata was captured before service changes. API and frontend remained independently rollbackable, and the API deployment retained one replica with stop-first semantics. + +## Pre-deploy v2 SQLite incident + +The mandatory baseline correctly blocked deployment when the still-running v2 process reported `db:error` and checkout readiness returned HTTP 502. + +Logs showed repeated SQLite `disk I/O error (522)` on ordinary reads. The fresh online backup and restore drill still passed, which indicated the stored databases were readable and internally consistent. + +The exact same pinned v2 image was restarted with stop-first semantics. Fresh database handles immediately restored: +- `db=ok` +- checkout readiness HTTP 200 +- connector connected +- no new SQLite 522 errors in the replacement task + +No database restore or direct live-database mutation was required. + +## API and customer acceptance + +The v3 API candidate passed immediate and post-cron production gates: +- database healthy +- connector connected +- relay ready +- checkout origin allowlist preserved +- denied origins still rejected +- trusted write API remained private +- new operator endpoints remained private anonymously +- no new task/database errors + +The customer frontend passed both-domain verification with the same static assets, security headers, SPA routing, and checkout API connectivity. + +Harmless unpaid production smoke sessions were created and immediately cancelled on Kotak, Slice, and Paytm. Public payment-status responses remained redacted. No real payment was made. + +## Android in-place upgrade + +The installed phone was reached over Tailscale ADB at `100.99.14.85:5555` and independently verified before upgrade: +- package: `io.github.phloraxx.paygaterelay` +- previous version: `0.3.1`, versionCode `7` +- previous installed APK SHA-256: `a971b4ffa09e41877bc1b5858b1a427d743f0ad62ba5249dfd640012f4161e72` +- stable signer SHA-256: `412b8f66c06acd93958c4dd11caa3214ba950059c00e57159d5673f94700d44a` + +The verified v0.4 release APK SHA-256 was `b55ef0f50309f3ac370b1ad6d8a32168b372dc03df6dd47ee2d6d0fbd06b687b`. + +The update used only `adb install -r`. It preserved: +- first-install lineage and device identity +- notification-listener access +- battery-optimization exemption +- background execution allowance +- foreground relay service +- pairing and signed heartbeat + +After upgrade the same relay device reported `0.4.0-alpha1`, `power_health_reported=1`, `battery_optimization_exempt=1`, `background_restricted=0`, `foreground_service_active=1`, and no client error. + +## Locked + Battery Saver acceptance + +With the phone locked and dozing, Battery Saver was enabled for a controlled acceptance check. The app remained battery-whitelisted and its foreground relay/listener services stayed active. + +The app launcher path was invoked without bypassing keyguard; `PayGateActivity` called the relay's normal `ensureRunning()`/`kick()` path. The server received a fresh signed heartbeat while keyguard remained active, and persisted telemetry reported `power_save_mode=1` with the foreground service active and no client error. + +The relay also has a 15-minute WorkManager safety pass. Autonomous-heartbeat observation should be checked after that interval when validating long-idle behavior. + +## Historical local queue item + +One historical failed local item remains on the phone: a GPay observation that previously received HTTP 401. It is not a pending Paytm relay item. Pending queue is zero. The record was intentionally retained rather than deleting history to make the counter zero. + +## Remaining acceptance + +- visually exercise authenticated Android Home / Payments / Action / Health after normal user unlock +- verify operator token refresh does not interrupt relay operation +- observe the 15-minute background safety heartbeat while locked/Battery Saver is active +- restore Battery Saver to the user's original off state after the controlled test +- continue ordinary production soak monitoring +- Razorpay Live remains out of scope without explicit approval