Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# CI and contributions

- Read affected package guidance before changing checks.
- Preserve PR, main-push, merge-group, manual-dispatch, and reusable-workflow events. `github.base_ref` is not always populated outside PRs.
- Change detection must include workflows and relevant shared actions/configuration; test-only and tooling-only changes must trigger owning checks.
- Preserve tool versions, lockfiles, reproducible installs, hardware/platform matrices, and pinned action revisions.
- Prefer package completion commands over duplicating them in YAML. For webv2's `pnpm check:release`, install Chromium and required Linux dependencies first.
- Preserve failure diagnostics with `always()` where appropriate; keep generated review/performance artifacts out of Git.
- Match hooks to CI's Ruff version/configuration; validation must not rewrite formatting.
- Follow [the PR template](pull_request_template.md); keep summaries short and omit subagent counts, tiers, and process narration.
- Workflow changes do not configure hosted branch protection or install local hooks; verify before claiming either.
- Keep PR, main-push, merge-group, manual-dispatch, reusable-workflow events. `github.base_ref` is not always set outside PRs.
- Change detection must include workflows and relevant shared actions/config. Test-only and tooling-only changes must trigger owning checks.
- Keep tool versions, lockfiles, reproducible installs, hardware/platform matrices, pinned action revisions.
- Prefer package completion commands over duplicating them in YAML. Webv2 `pnpm check:release`: install Chromium and required Linux dependencies first.
- Keep failure diagnostics with `always()` where fitting. Generated review/performance artifacts stay out of Git.
- Hooks must match CI's Ruff version and config. Validation never rewrites formatting.
- Follow [the PR template](pull_request_template.md). Short summaries. Omit subagent counts, tiers, process narration.
- Workflow changes neither configure hosted branch protection nor install local hooks. Verify before claiming either.
107 changes: 60 additions & 47 deletions AGENTS.md

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion docs/src/content/docs/contributing/new-model-integration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -851,7 +851,8 @@ Paths below are relative to `invokeai/frontend/webv2/src/features/generation/cor
9. **Regenerate the graph contract**

```sh
pnpm -C invokeai/frontend/webv2 test src/features/generation/core/graphCoverage.test.ts -u
cd invokeai/frontend/webv2
pnpm test src/features/generation/core/graphCoverage.test.ts -u
```

This rewrites `__snapshots__/generateGraphNodeTypes.json`. Graph coverage and the remix round trip compile each base in the shapes listed in `graphCoverage.testing.ts`; a variant that changes the graph needs its own entry in `SHAPE_OVERRIDES`, and a component variant a slot filters for needs to be in `CANDIDATE_VARIANTS`, or neither test ever builds that graph. Then raise `SUPPORTED_BASE_COUNTS["generate"]` in `tests/app/invocations/test_frontend_graph_node_types.py`, and `LITERAL_FLOORS` if needed. That Python test checks every node type, edge field and literal value the builders emit against the backend's invocation registry.
Expand Down
11 changes: 6 additions & 5 deletions docs/src/content/docs/development/Guides/tests.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,18 @@ The default frontend is `webv2`, with unit and Chromium suites. Run commands fro
</TabItem>
<TabItem label="pnpm">
```bash
pnpm -C invokeai/frontend/webv2 test
pnpm -C invokeai/frontend/webv2 test:browser
cd invokeai/frontend/webv2
pnpm test
pnpm test:browser
# Complete milestone gates, including performance, project files, and accessibility.
pnpm -C invokeai/frontend/webv2 check:release
pnpm check:release
```
</TabItem>
</Tabs>

Install Chromium with `pnpm -C invokeai/frontend/webv2 exec playwright install chromium` if needed. Read `webv2/AGENTS.md` for test ownership and browser verification rules.
Install Chromium with `pnpm exec playwright install chromium` from `invokeai/frontend/webv2` if needed. Read `webv2/AGENTS.md` for test ownership and browser verification rules.

The legacy frontend remains in `invokeai/frontend/webv1`. Run its separate suite with `pnpm -C invokeai/frontend/webv1 test:no-watch` and its checks with `pnpm -C invokeai/frontend/webv1 lint`.
The legacy frontend remains in `invokeai/frontend/webv1`. From that directory, run its separate suite with `pnpm test:no-watch` and its checks with `pnpm lint`.

## Backend Tests

Expand Down
44 changes: 25 additions & 19 deletions invokeai/app/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,34 @@

## Ownership and contracts

- Routers validate requests, enforce authentication/authorization, and map service errors to HTTP. Services own domain policy; storage owns persistence.
- Before changing flows, inspect `api/dependencies.py`, affected routers, service base/default/storage implementations, and tests. Preserve dependency/start/stop lifecycles; avoid parallel global services.
- Server-authorize every account-owned resource, including queries, writes, events, exports, and recovery; UI guards are insufficient.
- Trace frontend/saved-workflow consumers before changing Pydantic models, invocation fields/versions, HTTP DTOs, or socket-event names, defaults, validation, or errors.
- API changes may require regenerating `frontend/api/openapi.json` and `frontend/api/schema.ts` under `invokeai/`, even for webv2. Follow shared API package generation guidance; never hand-edit output.
- Routers validate requests, enforce authentication/authorization, map service errors to HTTP. Services own domain policy. Storage owns persistence.
- Before changing flows, read `api/dependencies.py`, affected routers, service base/default/storage implementations, tests. Keep dependency/start/stop lifecycles. No parallel global services.
- Server-authorize every account-owned resource, including queries, writes, events, exports, recovery. UI guards are not enough.
- Before changing Pydantic models, invocation fields/versions, HTTP DTOs, or socket-event names/defaults/validation/errors: trace frontend and saved-workflow consumers.
- API changes may need regenerated `frontend/api/openapi.json` and `frontend/api/schema.ts` under `invokeai/`, webv2 included. Follow shared API package generation guidance. Never hand-edit output.

## Persistence and lifecycle

- Preserve atomicity, revision checks, idempotency, and explicit conflicts. Inspect migrations/old-record readers for stored-data changes; add needed migration/compatibility tests.
- SQL lives only in `services/shared/database/`, reached through `Database.queries`; `tests/app/test_database_access_guard.py` enforces this. Query modules build their statements once, at module level with `bindparam()` (a statement built per call costs several times its execution), read rows by position, and build DTOs with `@mapped(...)` outside the transaction. A shape-dependent statement comes from a cached builder whose shapes a client cannot multiply (normalize its lists and flags, round its list lengths; see the guide).
- Use one transaction owner for multi-record writes: `with db.queries.transaction() as q:`, passing `q` to helpers. A second transaction on the same thread raises `NestedTransactionError`. See `services/shared/database/database.py`.
- SQLite serialises everything behind one lock; MySQL/MariaDB do not. Put an invariant like "only if still pending" into the statement (a conditional `UPDATE` with a checked row count), not into a read followed by a write. An invariant that spans rows (the last active administrator) gets a `DatabaseLock`, which every transaction changing what it reads acquires as its first call (`q.locks.acquire(...)`); its row is added by a migration. A decision about one row (may this board be claimed?) locks that row before reading what it decides on (a `@locking` query method, `SELECT ... FOR UPDATE`); every transaction changing what it reads writes or locks the same row, and rows are locked in one order (a project's before its board's). Run a unit whose work has no effects outside the database with `db.queries.run(work)`, which retries it after a deadlock.
- Tables are defined in `services/shared/database/schema/`, which must match the migrated SQLite schema exactly (`test_schema_parity.py`). Change the schema with a migration and the same change to the metadata, in one change. No database has triggers: queries set `updated_at` (also in upserts) and the queue's `started_at`, `completed_at`, `session_revision`.
- New migrations are `PortableMigration`s (Alembic operations; tables via `context.create_table()`) and idempotent: MySQL/MariaDB commit DDL as it runs, so a migration that failed halfway runs again. On SQLite they run with foreign keys off and are checked before commit. Migrations up to `PORTABLE_CUTOVER` stay SQLite-only and unchanged. See the Database Migrations guide.
- Preserve project/account isolation, document bounds, queue admission/receipts, and lost-response retry safety. Never silently discard recovery data or resolve conflicts with last-write-wins.
- Bound concurrency; support cancellation, timeouts/retries, and file/process/listener/task cleanup. Keep CPU/blocking I/O off the async server loop.
- Preserve file/path validation, remote-fetch protections, and sanitized errors. Never modify real user models, databases, outputs, or credentials during tests/debugging.
- Preserve atomicity, revision checks, idempotency, explicit conflicts. Stored-data change: read migrations and old-record readers; add needed migration/compatibility tests.
- SQL lives only in `services/shared/database/`, reached through `Database.queries`. `tests/app/test_database_access_guard.py` enforces this.
- Query modules build statements once, at module level, with `bindparam()` (per-call building costs several times execution). Read rows by position; build DTOs with `@mapped(...)` outside the transaction. Shape-dependent statement: cached builder whose shapes a client cannot multiply (normalize lists and flags, round list lengths; see the guide).
- Multi-record writes: one transaction owner, `with db.queries.transaction() as q:`, pass `q` to helpers. Second transaction on the same thread raises `NestedTransactionError`. See `services/shared/database/database.py`.
- Concurrency. SQLite serialises everything behind one lock; MySQL/MariaDB do not.
- Single-statement invariant ("only if still pending"): put it in the statement, a conditional `UPDATE` with checked row count. Never a read then a write.
- Invariant spanning rows (last active administrator): a `DatabaseLock`. Every transaction that changes rows the invariant reads acquires the lock as its first call (`q.locks.acquire(...)`). A migration adds its row.
- Decision about one row (may this board be claimed?): lock that row before reading what the decision uses (`@locking` query method, `SELECT ... FOR UPDATE`). Every transaction changing those inputs writes or locks the same row. Lock rows in one order (e.g. project before its board).
- Unit with no effects outside the database: run with `db.queries.run(work)`; it retries after a deadlock.
- Tables live in `services/shared/database/schema/` and must match the migrated SQLite schema exactly (`test_schema_parity.py`). Change schema with a migration plus the same metadata change, in one change.
- No database has triggers. Queries set `updated_at` (upserts too) and the queue's `started_at`, `completed_at`, `session_revision`.
- New migrations: `PortableMigration`s (Alembic operations; tables via `context.create_table()`), idempotent. MySQL/MariaDB commit DDL as it runs, so a half-failed migration runs again. On SQLite they run with foreign keys off and are checked before commit. Migrations up to `PORTABLE_CUTOVER` stay SQLite-only and unchanged. See the Database Migrations guide.
- Preserve project/account isolation, document bounds, queue admission/receipts, lost-response retry safety. Never silently discard recovery data. Never resolve conflicts with last-write-wins.
- Bound concurrency. Support cancellation, timeouts/retries, file/process/listener/task cleanup. Keep CPU and blocking I/O off the async server loop.
- Preserve file/path validation, remote-fetch protections, sanitized errors. Never modify real user models, databases, outputs, or credentials in tests or debugging.

## Efficiency and verification

- Inspect queries for repeated lookups, full scans/materialization, missing bounds, and serialization. Measure representative data/query counts before adding caches or indexes.
- Use real temporary SQLite for transaction, migration, revision, and ownership tests; cover affected malformed input, cross-user access, cancellation, repeated requests, and partial failures. Tests on the database fixtures also run against MySQL and MariaDB when `INVOKEAI_TEST_DB_URL` is set (see `tests/fixtures/database.py`).
- Read `tests/AGENTS.md`; use `tests/app/` or the existing owning suite. Run focused pytest and root Ruff; broaden to relevant service/router/invocation suites for milestones.
- For graph execution, consult `services/shared/README.md`; test scheduling and invocation lifecycles through owning interfaces.
- Report schema generation, migration validation, and unavailable integration/hardware checks.
- Check queries for repeated lookups, full scans/materialization, missing bounds, serialization. Measure representative data and query counts before adding caches or indexes.
- Transaction, migration, revision, ownership tests: real temporary SQLite. Cover affected malformed input, cross-user access, cancellation, repeated requests, partial failures. Database-fixture tests also run against MySQL and MariaDB when `INVOKEAI_TEST_DB_URL` is set (`tests/fixtures/database.py`).
- Read `tests/AGENTS.md`. Use `tests/app/` or the existing owning suite. Run focused pytest and root Ruff. Milestones: broaden to relevant service/router/invocation suites.
- Graph execution: read `services/shared/README.md`. Test scheduling and invocation lifecycles through owning interfaces.
- Report schema generation, migration validation, unavailable integration/hardware checks.
26 changes: 13 additions & 13 deletions invokeai/backend/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,22 @@

## Design and resources

- Trace callers, loaders/caches, tensor shapes, device/dtype handling, and cleanup before inference changes. Reusable model/inference policy belongs here, not UI/API adapters.
- Use existing device/model-management abstractions, including `util/devices.py`; preserve CPU/MPS/ROCm/XPU/multi-device support without hardcoded CUDA, GPU zero, or dtype assumptions.
- Respect session/thread device ownership. Restore temporary model patches, device state, hooks, and allocations on success/cancellation/exceptions; avoid request-specific mutable globals.
- Define model/cache lifetimes and working-memory estimates. Bound queues, caches, downloads, batches, and buffers; consider peak memory and throughput.
- Preserve model validation/loading safeguards, vendored exclusions, and upstream provenance; avoid unrelated vendored rewrites/formatting.
- Before inference changes, trace callers, loaders/caches, tensor shapes, device/dtype handling, cleanup. Reusable model/inference policy lives here, not in UI/API adapters.
- Use existing device/model-management abstractions, `util/devices.py` included. Keep CPU/MPS/ROCm/XPU/multi-device support. No hardcoded CUDA, GPU zero, or dtype assumptions.
- Respect session/thread device ownership. Restore temporary model patches, device state, hooks, allocations on success, cancellation, and exceptions. No request-specific mutable globals.
- Define model/cache lifetimes and working-memory estimates. Bound queues, caches, downloads, batches, buffers. Consider peak memory and throughput.
- Keep model validation/loading safeguards, vendored exclusions, upstream provenance. No unrelated vendored rewrites or formatting.

## Performance

- Inspect duplicate work, device transfers, dtype conversions, tensor copies, synchronization, repeated loading, and retained tensors.
- Measure material latency/throughput/peak-memory changes with representative shapes, resolutions, batches, and devices; distinguish cold loading/compilation from warm inference.
- Preserve output semantics, cancellation, and correctness at larger inputs, using appropriate numerical tolerances and reproducible seeds where supported.
- Kernels, caches, compilation, or concurrency need demonstrated benefit and clear fallback/lifecycle behavior.
- Check duplicate work, device transfers, dtype conversions, tensor copies, synchronization, repeated loading, retained tensors.
- Measure material latency/throughput/peak-memory changes with representative shapes, resolutions, batches, devices. Separate cold loading/compilation from warm inference.
- Keep output semantics, cancellation, correctness at larger inputs. Use fitting numerical tolerances and reproducible seeds where supported.
- Kernels, caches, compilation, concurrency need demonstrated benefit and clear fallback/lifecycle behavior.

## Verification

- Read `tests/AGENTS.md` and relevant suites in `tests/backend/`, `tests/model_identification/`, or `tests/test_model_manager/`.
- Use small CPU fixtures and targeted device checks; real tensor operations must expose shape/dtype/numerical/memory errors mocks would hide.
- Cover affected failure cleanup, unsupported configurations, boundary shapes, and cancellation. No routine large-model downloads or user model/output directories for unit tests.
- Run focused pytest and root Ruff. For material inference changes, report hardware, dtype, shapes, measurements, and untested platforms/model paths; CPU passes do not prove accelerator behavior.
- Read `tests/AGENTS.md` and relevant suites in `tests/backend/`, `tests/model_identification/`, `tests/test_model_manager/`.
- Small CPU fixtures plus targeted device checks. Use real tensor operations: they expose shape/dtype/numerical/memory errors mocks hide.
- Cover affected failure cleanup, unsupported configurations, boundary shapes, cancellation. Unit tests never use user model/output directories and never routinely download large models.
- Run focused pytest and root Ruff. Material inference changes: report hardware, dtype, shapes, measurements, untested platforms/model paths. CPU passes do not prove accelerator behavior.
12 changes: 6 additions & 6 deletions invokeai/frontend/api/AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Shared API contracts

This package owns `openapi.json`, generated `schema.ts`, and the generator used to check backend/frontend contract drift. It has no UI runtime. Frontend CI runs webv2; contract CI installs this package only.
Owns `openapi.json`, generated `schema.ts`, and the generator that checks backend/frontend contract drift. No UI runtime. Frontend CI runs webv2; contract CI installs only this package.

- Use pnpm 10 and this package's lockfile. Keep the pinned generator, TypeScript 5 peer, and formatter versions intentional; webv2's TypeScript version is independent.
- Never hand-edit generated output. In the repository's activated Python environment, run `make frontend-api-install`, `make frontend-openapi`, and `make frontend-typegen` from the root. Shell pipelines must preserve generator failures.
- Run `pnpm lint` and `pnpm test` here. The generator tests compile independent consumer expectations against real generated types and exercise invalid input.
- Keep the legacy `../webv1/src/services/api/schema.ts` type-only re-export compatible. Validate legacy TypeScript when shared types or that boundary change.
- API shape changes follow `invokeai/app/AGENTS.md` and require owning backend tests. Moving artifact ownership must preserve the generated contracts.
- pnpm 10 and this package's lockfile. Change pinned generator, TypeScript 5 peer, and formatter versions only on purpose. Webv2's TypeScript version is independent.
- Never hand-edit generated output. From the repo root, in the activated Python environment: `make frontend-api-install`, `make frontend-openapi`, `make frontend-typegen`. Shell pipelines must keep generator failures.
- Run `pnpm lint` and `pnpm test` here. Generator tests compile independent consumer expectations against real generated types and exercise invalid input.
- Keep legacy `../webv1/src/services/api/schema.ts` type-only re-export compatible. Shared types or that boundary changed: validate legacy TypeScript.
- API shape changes follow `invokeai/app/AGENTS.md` and need owning backend tests. Moving artifact ownership must keep generated contracts intact.
Loading
Loading