Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions deploy/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,11 @@ executes an immutable release selected through `/home/aipg/current`.
reduced-account response checks passed; all six workers returned. Billing,
generation admission, payout controls and worker runtimes were unchanged.
See `docs/MEDIA_ONBOARDING_FOLLOWUP.md`. The pre-existing missing reward-policy
payout failure is not fixed by this release. Image editing/batch canaries
and worker upgrades remain unexecuted; do not infer activation from merging.
payout failure is not fixed by this release. The later September28 22:36 UTC
configuration-only cutover admitted image editing/batches after real staged
billing qualification. Six paid production canaries passed; image-worker
`50476b9` and Gallery PR #39 are live. Other controls are unchanged and
timeline/3D remain closed. See the same follow-up for exact receipts/rollback.
- Core `3384c223` / Alembic `0042` is live as of September17 01:55 UTC.
Exact-main CI, production restore, drained cutover and a real paid Qwen Chat
canary passed. Charging, daily free spending, compensation contracts and
Expand Down
11 changes: 8 additions & 3 deletions deploy/DEMAND_BILLING_LAUNCH_2026_09_08.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,14 @@

GLOBAL DEMAND CHARGING ACTIVE; admitted-path frontend canaries passed and
prospective hourly worker payouts resumed after a supervised send and replay.
The current runtime is `793fe904` / Alembic `0041`. Director timeline rendering
remains disabled; Gallery's rejected-request spinner was repaired and verified
on release `24a0fbf0` at 14:35 UTC. No additional paths were enabled.
Current runtime is `ca1f2a12` / Alembic `0042`. On September28, image editing
and image batches were admitted after real billing qualification; six paid
production API jobs passed, costing USD 0.09 (USD 0.08 purchased plus USD 0.01
daily credit). Gallery `29f472d5` exposes qualified four-image batches.
See `../docs/MEDIA_ONBOARDING_FOLLOWUP.md` for receipts, isolated failure/refund
proof and rollback. Timeline/3D remain disabled. The pre-existing missing
reward-policy payout failure is not repaired by this image rollout; the
September10 payout observations below are historical, not current proof.
The goal covers every public generation path and all first-party frontends.
Unverified paths must be disabled or fail closed before public charging launch.
Historical accrual and disputed payments are outside this rollout.
Expand Down
5 changes: 3 additions & 2 deletions docs/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,9 @@ for humans and agents.
- `MEDIA_ONBOARDING_FOLLOWUP.md` - September28 Core `ca1f2a12` deployment proof,
isolated Klein GPU execution/seed-replay evidence, verified BF16/FP8 file
distinction, image-worker `50476b9` deployment/rollback and locked-runtime
seven-case GPU proof, and remaining billed media-canary acceptance criteria.
Worker upgrades and advanced-path activation remain separate gates.
seven-case GPU proof, followed by nine real staged billing cases, failure/
recovery proofs and six paid production API canaries. Image edits/batches
are admitted; Gallery PR #39 is deployed. Timeline/3D remain closed.
- `architecture/DECENTRALIZATION_ROADMAP.md` - accepted post-preview Base
validator and trusted-partner Core federation phases, event contract, and
go-live gates.
Expand Down
94 changes: 87 additions & 7 deletions docs/MEDIA_ONBOARDING_FOLLOWUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,89 @@

## September 28 Review

Worker PRs #26 and #27 are merged. They do not deploy a worker or enable a
generation path. Core PR #211 was deployed as recorded below; advanced media
canaries and worker upgrades remain separate work.
Worker PRs #26 and #27 are merged. Subsequent image-worker PR #30, real billing
qualification, Core image-path activation and Gallery PR #39 are deployed as
recorded below. Timeline/3D and the separate LTX/audio fleet are not covered.

## Image Billing Qualification and Activation: September 28

Core remains `ca1f2a127a878ed264589adba177181151da3c5c` / Alembic `0042`;
the image worker remains `50476b943dfa1e55ceac5536ccad1e6a4bf17a00`.
At `2026-09-28T22:36:36Z`, the owner-approved configuration-only cutover added
`image-to-image` and `image-batch` to the seven existing generation paths.
Charging stays `on`. No price, schema, recipe, payout, validator policy or
model/backend configuration changed. Timeline and 3D remain disabled.

Qualification before opening the global switches used that exact Core release,
a separate PostgreSQL database/role, separate Redis process, real R2 storage,
and the owned image GPU. Only synthetic scratch balances were used. The bridge
was borrowed after ComfyUI was idle and restored afterward; ComfyUI and the
other production workers were not restarted.

- Nine successful cases: four-image batches for Klein/Krea/Z-image, source
edits for all three, a second Klein edit, and Klein PNG/JPEG encoding.
Each proved a hold before dispatch, exact price, one settled reservation,
one completion ledger row, complete output count, distinct batch hashes,
assigned seeds, decodable stored bytes and matching R2 receipt hashes.
- Read-only result recovery returned every output. Replayed committed
terminals returned `duplicate`; late releases could not refund completion.
- Empty-account 402 and invalid-source 400 created no reservation or payout.
Incomplete batches, missing R2 objects and explicit worker errors returned
502, refunded once and created no completion payout. Repeated release was inert.
- A real client timeout while held recovered through the read-only result API:
one generation, one charge, no replacement POST and no premature refund.
- All 61 exact-release PostgreSQL billing/output regressions passed, including
concurrent debits, settlement/refund races and killed-refund recovery. Queue,
auth and storage mocks in those regression tests are distinct from the real
GPU/Redis/R2 qualification above.
- Test harness corrections were needed: the worker handshake is `ready`,
invalid-source returns 400, and subprocess tests needed an importable source
path. A synthetic test balance exhausted correctly with 402 before a second
idempotent scratch grant. No production credit was granted or test waived.

Activation gated new generation/probes, stopped MCP, observed empty held/queue
state twice and once with Core stopped, edited only `GENERATION_ENABLED_PATHS`,
then restarted the same release. All six workers returned before ingress
reopened. Payout/backup timer states and the payout drop-in were preserved;
the pre-existing reward-policy payout failure remains a separate issue.

After activation, six public production API canaries used the owner's existing
balance through an inference-only, 15-minute key. Each model passed a four-image
batch and a single source edit. Total usage was USD 0.09: USD 0.01 daily credit
plus USD 0.08 purchased credit. Every job has one completion, a settled hold,
matching stored output hashes and successful durable recovery. Global credit
reconciliation passed: zero negative balances, mismatches, stale holds or
invalid splits. The temporary key was revoked and then rejected with 401.
No test output was published in Gallery and no payout transfer was sent.

| Model | Four-image batch receipt | Source-edit receipt |
| --- | --- | --- |
| Klein | `c24bce71-b09b-4772-81fb-621f227a6155` | `3de48482-7642-48f8-9115-8cb443e5feb8` |
| Krea | `fb5b67c5-b08d-40a7-97f6-5ae1c512f031` | `2ff2820e-0a47-43c1-914e-40eff8215e96` |
| Z-image | `34fa8654-65df-4fdb-8f88-58745a334988` | `c827b1f1-3fa2-4dfd-9a52-907ed30cd3ef` |

Gallery PR #39 merged as `823edfdd`; its tree-identical tested head `29f472d5`
was selected at 22:38:10 UTC. PR/main CI, 125 Jest tests, 18 production-build
browser tests, host build/audits and restored-backup race suite passed.
Desktop/mobile tests verify four decoded images, one POST and source-upload
batch reset. The signed-in live browser was locked, so a fresh Gallery UI
submission is not claimed; production API and mocked browser evidence are
explicitly separate. The rollout preserves source-image single-output limits.

Protected Core activation/config backup and paid receipt evidence:
`/var/lib/aipg-release-proof/image-activation-20260928/`.
Gallery proof: `/var/lib/aipg-release-proof/gallery-29f472d5/`.
Rollback removes only the two new generation paths using the same drained
restart procedure and selects Gallery `0f3d3948`; do not disable billing,
discard receipts or rewrite ledger history. Isolated qualification data is
retained privately as evidence; its services and credentials are not public.

These tests establish execution/delivery/accounting, not identical visual
quality across models. Z-image's latent-blend edit retained the blue body on
the red-car prompt; it is not equivalent to instruction-following reference
editing. The Klein BF16/FP8 labeling discrepancy below is still open.

## Original Review Findings

- Fix the omitted-model image default to `FLUX.2 Klein 4B FP8`, matching the
dispatch name, curated recipe, and price. Explicit selections still win;
Expand Down Expand Up @@ -145,8 +225,8 @@ on the owned image host at `2026-09-28T22:06:29Z`.

This deployment did not change Core, recipes, model files, charging mode or
public admission. These GPU tests use synthetic dispatch/local upload slots,
not real credit/reward ledgers or R2. Paid end-to-end checks below are still
required before public image-to-image or batch activation.
not real credit/reward ledgers or R2. The later billed qualification and
activation above supersede this historical GPU-only evidence.

## Klein Recipe Prerequisite

Expand All @@ -173,8 +253,8 @@ version; a local edit must not bypass a governed commitment or revocation.

## Canary Acceptance

These remain paid end-to-end acceptance requirements; the isolated GPU checks
above do not fulfill them. Use an isolated staging
These are the acceptance requirements used for the qualification above. The
earlier synthetic-coordinator GPU checks alone did not fulfill them. Use an isolated staging
coordinator with disposable database/queue state and an explicitly assigned
owned worker, or first implement and review an account-bound, expiring public
path pilot. The existing generation allowlist is global: temporarily adding a
Expand Down
Loading