diff --git a/docs/src/content/docs/Users Guide/06.Gallery/gallery-reference.mdx b/docs/src/content/docs/Users Guide/06.Gallery/gallery-reference.mdx index f04e43b0807..5e6667c5e63 100644 --- a/docs/src/content/docs/Users Guide/06.Gallery/gallery-reference.mdx +++ b/docs/src/content/docs/Users Guide/06.Gallery/gallery-reference.mdx @@ -65,7 +65,7 @@ On a multi-user installation, only a board's owner or an administrator can delet Right-click an item, or several selected items, for its menu. -The context menu for an image, with quick-action icons at the top followed by Load Workflow, Recall Metadata, Add to Current Canvas, New Project from Image, Send to Upscale, Send to Video, Use as Reference Image, Use as Prompt Template, Select for Compare, Find in Gallery, Change Board and Delete Image. +The context menu for an image, with quick-action icons at the top followed by Load Workflow, Recall Metadata, Add to Current Canvas, New Project from Image, Send to Upscale, Send to Video, Use as Reference Image, Use as Prompt Template, Select for Compare, Generate PBR Maps, Find in Gallery, Change Board and Delete Image. For an image, the menu offers: @@ -74,6 +74,7 @@ For an image, the menu offers: * **Add to Current Canvas** — add the image to the open project's canvas as a new layer of the kind you choose. * **New Project from Image** — start a new project whose canvas holds the image, sized to fit it. * **Send to Upscale**, **Send to Video**, **Use as Reference Image**, **Use as Prompt Template** and **Select for Compare** — use the image elsewhere in InvokeAI. +* **Generate PBR Maps** — queue normal, roughness and displacement maps of the image. They are saved to the board the Gallery is auto-adding to, or to the project's Inbox if you can't add to that board (Uncategorized when there is neither), and a notice reports when they are ready. The first run downloads the PBR models. * **Change Board** and **Delete Image**. A video's menu has the equivalent actions for video, including **Extend in Video**, **Use as Reference Video** and **Use as Conditioning Clip**. With several items selected, the menu offers the actions that apply to all of them, such as **Star All**, **Download Selection**, **Change Board** and **Delete Selection**; for several images it also leads with **Add to Current Canvas** and **New Project from Images**. diff --git a/docs/src/content/docs/Users Guide/06.Gallery/image-storage-maintenance.mdx b/docs/src/content/docs/Users Guide/06.Gallery/image-storage-maintenance.mdx index 7f5ca0e97cb..ec8d93f46b3 100644 --- a/docs/src/content/docs/Users Guide/06.Gallery/image-storage-maintenance.mdx +++ b/docs/src/content/docs/Users Guide/06.Gallery/image-storage-maintenance.mdx @@ -9,15 +9,21 @@ This operation changes where image files and thumbnails are stored on disk. It d ## Access -Image storage maintenance is available from the in-application Settings panel. +Open **Settings → Server → Image storage**. The **Folder layout for new images** select changes `image_subfolder_strategy` immediately for images saved from then on. Below it, **Move existing images** shows how many images are stored outside the current layout, the state of the last move job, and the **Start Move** and **Recover** buttons; each asks for confirmation before it starts. -Administrators can start image moves, force recovery, and inspect the current move status. If multi-user mode is disabled, the single local user has the same access. Non-admin users in multi-user mode cannot run or inspect image move status. +- **Start Move** is available when images are stored outside the current layout and no interrupted job is waiting for recovery. The server still refuses it while any queue item is pending or in progress. +- **Recover** is available only when a move was interrupted and still needs finishing. +- Both buttons are unavailable while a job runs or a layout change is being saved. The status line or a note below the buttons says why a button is unavailable. + +While a job runs and the Image storage setting is open, the status refreshes every few seconds and the folder layout cannot be changed. Closing Settings stops the refresh; reopen it to see the current state. + +Administrators can start image moves, force recovery, and inspect the current move status. If multi-user mode is disabled, the single local user has the same access. Non-admin users in multi-user mode do not see the Server section and cannot run or inspect image move status. ## Maintenance Mode -Image moves run as a maintenance operation. Before a move starts, InvokeAI checks that no queue work is active. While the move is running, InvokeAI prevents image reads, uploads, deletes, generation jobs, and gallery mutations from racing with the filesystem move. +Image moves run as a maintenance operation. Before a move starts, InvokeAI checks that no queue work is active. While the move is running, or an interrupted move is waiting for recovery, InvokeAI prevents image reads, uploads, deletes, generation jobs, and gallery mutations from racing with the filesystem move. -The UI shows the move or recovery state until the job is complete or requires manual attention. Gallery images and thumbnails may be unavailable while maintenance is active. +The UI shows the move or recovery state until the job is complete or requires manual attention. Gallery images and thumbnails may be unavailable while maintenance is active; an open gallery keeps checking periodically while its tab is visible and reloads them on its own shortly after maintenance ends, so you do not need to refresh the page. ## Crash Recovery @@ -25,7 +31,7 @@ The move process is crash-recoverable. InvokeAI records each move job in its dat If InvokeAI stops during a move, recovery resumes incomplete jobs. If recovery finds an ambiguous filesystem state, such as both the old and new full-size image files existing or neither file existing, it halts that item for manual repair instead of blindly updating the database. Other items in the same job are still finalized, and the maintenance lock is released when the job is recorded as failed. -Recovery finalizes jobs that were already recorded before the stop. Images that had not yet been planned into a move job are not moved by recovery itself. The Settings panel reports how many images still need moving; run **Start Move** again to continue. +Recovery finalizes jobs that were already recorded before the stop. Images that had not yet been planned into a move job are not moved by recovery itself. The Image storage setting reports how many images still need moving; run **Start Move** again to continue. While a job is interrupted, **Start Move** stays unavailable and the status asks you to run **Recover** first. Missing intermediate image files are treated as already cleaned up and do not halt the move. Missing non-intermediate image files, unreadable images, and truncated images are recorded as failed items. When the full-size file is still at its original path, recovery leaves it there so it can be repaired or removed safely. If a previous filesystem move already placed the full-size file at its destination, recovery keeps that destination and reconciles the database before recording the item as failed. After recovery releases the lock, back up the failed file and either replace it with a valid image or delete its image record from the gallery, then run **Start Move** again if images still need moving. Do not delete an active journal row or edit `images.image_subfolder` while maintenance is active. diff --git a/docs/src/content/docs/configuration/01.invokeai-yaml.mdx b/docs/src/content/docs/configuration/01.invokeai-yaml.mdx index 28f3514631a..2c1dfa11865 100644 --- a/docs/src/content/docs/configuration/01.invokeai-yaml.mdx +++ b/docs/src/content/docs/configuration/01.invokeai-yaml.mdx @@ -197,7 +197,7 @@ Background indexing of new images already waits until no generation is running, #### Image Subfolder Strategy -By default, generated images are stored in a single flat directory under `outputs/images/`. The `image_subfolder_strategy` setting lets you organize newly-created images into subfolders automatically. You can edit this setting in `invokeai.yaml` or, as an admin user, in the Settings panel. +By default, generated images are stored in a single flat directory under `outputs/images/`. The `image_subfolder_strategy` setting lets you organize newly-created images into subfolders automatically. You can edit this setting in `invokeai.yaml` or, as an admin user, under **Settings → Server → Image storage**. ```yaml image_subfolder_strategy: flat # default value diff --git a/invokeai/frontend/webv2/performance/architecture-baseline.json b/invokeai/frontend/webv2/performance/architecture-baseline.json index 0126e029233..3b32282024d 100644 --- a/invokeai/frontend/webv2/performance/architecture-baseline.json +++ b/invokeai/frontend/webv2/performance/architecture-baseline.json @@ -135,6 +135,8 @@ "source:src/features/fonts/launchpad.tsx", "source:src/features/fonts/react.tsx", "source:src/features/fonts/runtime.ts", + "source:src/features/gallery/data/imageRecovery.ts", + "source:src/features/gallery/ui/useGalleryImageRecovery.ts", "source:src/features/identity/capabilities.ts", "source:src/features/identity/core/schemas.ts", "source:src/features/identity/core/sessionPolicy.ts", @@ -515,6 +517,7 @@ "source:src/features/gallery/core/settings.ts", "source:src/features/gallery/core/videoPaths.ts", "source:src/features/gallery/data/backend.ts", + "source:src/features/gallery/data/imageRecovery.ts", "source:src/features/gallery/data/imageUrls.ts", "source:src/features/gallery/data/queries.ts", "source:src/features/gallery/data/queryCache.ts", @@ -543,6 +546,7 @@ "source:src/features/gallery/ui/galleryStateView.ts", "source:src/features/gallery/ui/picker/GalleryPickerPopover.tsx", "source:src/features/gallery/ui/useGalleryData.ts", + "source:src/features/gallery/ui/useGalleryImageRecovery.ts", "source:src/features/gallery/ui/useGalleryUploadAction.ts", "source:src/features/gallery/ui/useGalleryUploadInput.ts", "source:src/features/gallery/utility.ts", diff --git a/invokeai/frontend/webv2/performance/browser-baseline.json b/invokeai/frontend/webv2/performance/browser-baseline.json index 0182034957a..12f4863aa06 100644 --- a/invokeai/frontend/webv2/performance/browser-baseline.json +++ b/invokeai/frontend/webv2/performance/browser-baseline.json @@ -521,6 +521,7 @@ "source:src/features/gallery/core/settings.ts", "source:src/features/gallery/core/videoPaths.ts", "source:src/features/gallery/data/backend.ts", + "source:src/features/gallery/data/imageRecovery.ts", "source:src/features/gallery/data/imageUrls.ts", "source:src/features/gallery/data/videoUrls.ts", "source:src/features/gallery/launchpad.ts", @@ -528,6 +529,7 @@ "source:src/features/gallery/ui/GalleryUiContext.tsx", "source:src/features/gallery/ui/galleryStateView.ts", "source:src/features/gallery/ui/launchpad/RecentOutputs.tsx", + "source:src/features/gallery/ui/useGalleryImageRecovery.ts", "source:src/features/identity/capabilities.ts", "source:src/features/identity/core/schemas.ts", "source:src/features/identity/core/sessionPolicy.ts", @@ -912,6 +914,7 @@ "source:src/features/gallery/core/settings.ts", "source:src/features/gallery/core/videoPaths.ts", "source:src/features/gallery/data/backend.ts", + "source:src/features/gallery/data/imageRecovery.ts", "source:src/features/gallery/data/imageUrls.ts", "source:src/features/gallery/data/queries.ts", "source:src/features/gallery/data/queryCache.ts", @@ -968,6 +971,7 @@ "source:src/features/gallery/ui/useGalleryData.ts", "source:src/features/gallery/ui/useGalleryGridHotkeys.ts", "source:src/features/gallery/ui/useGalleryGridSelection.ts", + "source:src/features/gallery/ui/useGalleryImageRecovery.ts", "source:src/features/gallery/ui/useGalleryStarredStrip.ts", "source:src/features/gallery/ui/useGalleryUploadAction.ts", "source:src/features/gallery/ui/useGalleryUploadInput.ts", @@ -1605,6 +1609,7 @@ "source:src/workbench/image-actions/imageRecall.ts", "source:src/workbench/image-actions/index.ts", "source:src/workbench/image-actions/mediaUsage.ts", + "source:src/workbench/image-actions/pbrMapsGraph.ts", "source:src/workbench/image-actions/recallParametersBridge.ts", "source:src/workbench/image-actions/useDeletionConfirmation.tsx", "source:src/workbench/image-actions/useFindGalleryItem.ts", @@ -2022,6 +2027,7 @@ "source:src/features/gallery/core/settings.ts", "source:src/features/gallery/core/videoPaths.ts", "source:src/features/gallery/data/backend.ts", + "source:src/features/gallery/data/imageRecovery.ts", "source:src/features/gallery/data/imageUrls.ts", "source:src/features/gallery/data/queries.ts", "source:src/features/gallery/data/queryCache.ts", @@ -2078,6 +2084,7 @@ "source:src/features/gallery/ui/useGalleryData.ts", "source:src/features/gallery/ui/useGalleryGridHotkeys.ts", "source:src/features/gallery/ui/useGalleryGridSelection.ts", + "source:src/features/gallery/ui/useGalleryImageRecovery.ts", "source:src/features/gallery/ui/useGalleryStarredStrip.ts", "source:src/features/gallery/ui/useGalleryUploadAction.ts", "source:src/features/gallery/ui/useGalleryUploadInput.ts", @@ -2733,6 +2740,7 @@ "source:src/workbench/image-actions/imageRecall.ts", "source:src/workbench/image-actions/index.ts", "source:src/workbench/image-actions/mediaUsage.ts", + "source:src/workbench/image-actions/pbrMapsGraph.ts", "source:src/workbench/image-actions/recallParametersBridge.ts", "source:src/workbench/image-actions/useDeletionConfirmation.tsx", "source:src/workbench/image-actions/useFindGalleryItem.ts", @@ -3316,6 +3324,7 @@ "source:src/features/gallery/core/settings.ts", "source:src/features/gallery/core/videoPaths.ts", "source:src/features/gallery/data/backend.ts", + "source:src/features/gallery/data/imageRecovery.ts", "source:src/features/gallery/data/imageUrls.ts", "source:src/features/gallery/data/queries.ts", "source:src/features/gallery/data/queryCache.ts", @@ -3372,6 +3381,7 @@ "source:src/features/gallery/ui/useGalleryData.ts", "source:src/features/gallery/ui/useGalleryGridHotkeys.ts", "source:src/features/gallery/ui/useGalleryGridSelection.ts", + "source:src/features/gallery/ui/useGalleryImageRecovery.ts", "source:src/features/gallery/ui/useGalleryStarredStrip.ts", "source:src/features/gallery/ui/useGalleryUploadAction.ts", "source:src/features/gallery/ui/useGalleryUploadInput.ts", @@ -4086,6 +4096,7 @@ "source:src/workbench/image-actions/imageRecall.ts", "source:src/workbench/image-actions/index.ts", "source:src/workbench/image-actions/mediaUsage.ts", + "source:src/workbench/image-actions/pbrMapsGraph.ts", "source:src/workbench/image-actions/recallParametersBridge.ts", "source:src/workbench/image-actions/useDeletionConfirmation.tsx", "source:src/workbench/image-actions/useFindGalleryItem.ts", diff --git a/invokeai/frontend/webv2/public/locales/en.json b/invokeai/frontend/webv2/public/locales/en.json index d2869ea7f7b..11d236ee342 100644 --- a/invokeai/frontend/webv2/public/locales/en.json +++ b/invokeai/frontend/webv2/public/locales/en.json @@ -2063,6 +2063,50 @@ "failed": "Database VACUUM failed.", "requestFailed": "The database could not be vacuumed." }, + "imageStorage": { + "adminOnly": "Only an administrator can change where images are stored.", + "layoutLabel": "Folder layout for new images", + "layoutHelp": "Applies to images saved from now on. Existing images stay where they are until you run Start Move.", + "layoutLoadFailed": "The storage layout could not be loaded.", + "layoutSaveFailed": "The storage layout could not be saved.", + "layouts": { + "flat": "Flat (one folder)", + "date": "By date (YYYY/MM/DD)", + "type": "By image category", + "hash": "By name prefix (hashed)", + "unknown": "Unknown layout ({{layout}})" + }, + "maintenanceHeading": "Move existing images", + "maintenanceDescription": "Moves every image and thumbnail into the current layout. Names, boards and metadata are unchanged. While the move runs, the gallery and the queue are paused.", + "statusLoadFailed": "The image move status could not be loaded.", + "statusIdle_zero": "All images are stored in the current layout.", + "statusIdle_one": "{{count}} image is stored outside the current layout.", + "statusIdle_other": "{{count}} images are stored outside the current layout.", + "statusMoving": "Moving images into the current layout…", + "statusRecovering": "Recovering an interrupted image move…", + "statusNeedsRecovery": "An image move was interrupted. Run Recover to finish it before starting another move.", + "latestJob": "Last move: {{state}}", + "jobStates": { + "planned": "planned", + "moving": "moving", + "moved": "moved", + "committed": "completed", + "error": "failed" + }, + "startMove": "Start Move", + "startConfirmTitle": "Move all images into the current layout?", + "startConfirmBody": "InvokeAI pauses image reads, uploads, deletions and generation for everyone on this server until the move finishes. It cannot start while queue work is active.", + "startFailed": "The image move could not be started.", + "recover": "Recover", + "recoverConfirmTitle": "Recover the image move journal?", + "recoverConfirmBody": "Recovery finishes moves that were recorded before the server stopped. Items in an ambiguous state are left for manual repair and reported here.", + "recoverFailed": "Image move recovery could not be started.", + "unavailableStatus": "Start Move and Recover become available once the move status has loaded.", + "unavailableRunning": "Start Move and Recover become available again when the current job finishes.", + "unavailableSavingLayout": "Start Move and Recover become available once the new layout is saved.", + "recoverNotNeeded": "Recover is only needed after a move is interrupted.", + "nothingToDo": "Nothing to move, and no interrupted move to recover." + }, "galleryMaintenance": { "heading": "Gallery maintenance", "removeMissing": "Remove records for missing images", @@ -2203,6 +2247,10 @@ "label": "Generation devices", "description": "Server-wide accelerator selection. Changes require a server restart." }, + "imageStorage": { + "label": "Image storage", + "description": "Where this server stores image files on disk, and the maintenance job that moves existing images into that layout." + }, "maxQueueHistory": { "label": "Queue history", "description": "How many finished queue items the server keeps on startup. Changes require a server restart." @@ -4846,6 +4894,18 @@ "missing": "This image has no workflow embedded.", "missingVideo": "This video has no workflow embedded." }, + "pbrMaps": { + "canceled": "PBR maps canceled", + "canceledMessage": "The queue item generating maps for {{name}} was canceled.", + "completed": "PBR maps are ready", + "completedMessage": "Normal, roughness and displacement maps for {{name}} were added to the gallery.", + "failed": "Could not generate PBR maps", + "label": "Generate PBR Maps", + "queued": "Generating PBR maps", + "queuedMessage": "Normal, roughness and displacement maps for {{name}} are queued.", + "removed": "PBR maps removed from the queue", + "removedMessage": "The queue item generating maps for {{name}} was removed from the queue." + }, "delete": { "failure_one": "Could not delete {{count}} item.", "failure_other": "Could not delete {{count}} items.", diff --git a/invokeai/frontend/webv2/scripts/mock-backend-fixtures.test.mjs b/invokeai/frontend/webv2/scripts/mock-backend-fixtures.test.mjs index 19288f5940b..04d6739b004 100644 --- a/invokeai/frontend/webv2/scripts/mock-backend-fixtures.test.mjs +++ b/invokeai/frontend/webv2/scripts/mock-backend-fixtures.test.mjs @@ -202,7 +202,7 @@ test('the HTTP reset contract selects profiles explicitly and restores the start ); assert.equal(projects.length, 40); assert.deepEqual(runtimeConfig, { - config: { generation_devices: 'auto', max_queue_history: null }, + config: { generation_devices: 'auto', image_subfolder_strategy: 'flat', max_queue_history: null }, set_fields: [], }); @@ -248,6 +248,16 @@ const getJson = async (backend, path, init) => { return body; }; +/** The representative queue holds pending and in-progress items, which refuse Start Move as on the real server. */ +const clearQueue = (backend) => getJson(backend, '/api/v1/queue/default/clear', { method: 'PUT' }); + +const enqueueBatch = (backend) => + fetch(`${backend.origin}/api/v1/queue/default/enqueue_batch`, { + body: JSON.stringify({ batch: { graph: { edges: [], id: 'pbr', nodes: {} }, runs: 1 }, prepend: false }), + headers: { 'content-type': 'application/json' }, + method: 'POST', + }); + test('queue item ids take the backend limit: the head of the order, counted as returned, bounded', async () => { await withRepresentativeBackend(async (backend) => { for (const orderDir of ['DESC', 'ASC']) { @@ -1211,6 +1221,243 @@ test('workflow thumbnails round-trip their bytes and refresh the listed URL on e }); }); +test('image storage: the layout is patched alone and the move count follows it', async () => { + await withRepresentativeBackend(async (backend) => { + assert.deepEqual(await getJson(backend, '/api/v1/image_moves/status'), { + active_job_id: null, + is_running: false, + last_error: null, + latest_job: null, + needs_move_count: 0, + operation: null, + }); + + const patched = await getJson(backend, '/api/v1/app/runtime_config', { + body: JSON.stringify({ image_subfolder_strategy: 'date' }), + headers: { 'content-type': 'application/json' }, + method: 'PATCH', + }); + + assert.deepEqual(patched, { + config: { generation_devices: 'auto', image_subfolder_strategy: 'date', max_queue_history: null }, + set_fields: ['image_subfolder_strategy'], + }); + assert.equal((await getJson(backend, '/api/v1/image_moves/status')).needs_move_count, 1_000); + + const invalid = await fetch(`${backend.origin}/api/v1/app/runtime_config`, { + body: JSON.stringify({ image_subfolder_strategy: 'by_user' }), + headers: { 'content-type': 'application/json' }, + method: 'PATCH', + }); + + assert.equal(invalid.status, 422); + }); +}); + +test('image storage: a move runs across status polls, commits, and refuses to overlap', async () => { + await withRepresentativeBackend(async (backend) => { + await getJson(backend, '/api/v1/app/runtime_config', { + body: JSON.stringify({ image_subfolder_strategy: 'hash' }), + headers: { 'content-type': 'application/json' }, + method: 'PATCH', + }); + await clearQueue(backend); + + const accepted = await fetch(`${backend.origin}/api/v1/image_moves/start`, { method: 'POST' }); + + assert.equal(accepted.status, 202); + assert.deepEqual(await accepted.json(), { + active_job_id: 1, + is_running: true, + last_error: null, + latest_job: { error_message: null, id: 1, state: 'moving' }, + needs_move_count: 1_000, + operation: 'move_all', + }); + + const overlapping = await fetch(`${backend.origin}/api/v1/image_moves/start`, { method: 'POST' }); + + assert.equal(overlapping.status, 409); + + const inProgress = await getJson(backend, '/api/v1/image_moves/status'); + + assert.equal(inProgress.is_running, true); + assert.equal(inProgress.operation, 'move_all'); + + // Like the real maintenance guard: image files and media mutations refuse while the move runs, listings do not. + const listed = await getJson(backend, '/api/v1/images/?limit=1&offset=0'); + const imageName = listed.items[0].image_name; + const thumbnail = await fetch(`${backend.origin}/api/v1/images/i/${imageName}/thumbnail`); + + assert.equal(thumbnail.status, 409); + assert.deepEqual(await thumbnail.json(), { detail: 'Image storage maintenance is active' }); + assert.equal((await fetch(`${backend.origin}/api/v1/images/i/${imageName}`)).status, 200); + assert.equal( + ( + await fetch(`${backend.origin}/api/v1/images/star`, { + body: JSON.stringify({ image_names: [imageName] }), + headers: { 'content-type': 'application/json' }, + method: 'POST', + }) + ).status, + 409 + ); + assert.equal((await fetch(`${backend.origin}/api/v1/board_images/`)).status, 409); + assert.equal((await fetch(`${backend.origin}/api/v1/boards/?all=true`)).status, 200); + assert.equal((await enqueueBatch(backend)).status, 409); + + assert.deepEqual(await getJson(backend, '/api/v1/image_moves/status'), { + active_job_id: null, + is_running: false, + last_error: null, + latest_job: { error_message: null, id: 1, state: 'committed' }, + needs_move_count: 0, + operation: null, + }); + assert.equal((await fetch(`${backend.origin}/api/v1/images/i/${imageName}/thumbnail`)).status, 200); + }); +}); + +test('image storage: an instant move commits inside the POST, and a failed job counts as finished', async () => { + await withRepresentativeBackend(async (backend) => { + await getJson(backend, '/api/v1/app/runtime_config', { + body: JSON.stringify({ image_subfolder_strategy: 'type' }), + headers: { 'content-type': 'application/json' }, + method: 'PATCH', + }); + await getJson(backend, '/__faults?imageMoves=instant', { method: 'POST' }); + await clearQueue(backend); + + const accepted = await fetch(`${backend.origin}/api/v1/image_moves/start`, { method: 'POST' }); + + assert.equal(accepted.status, 202); + assert.deepEqual(await accepted.json(), { + active_job_id: null, + is_running: false, + last_error: null, + latest_job: { error_message: null, id: 1, state: 'committed' }, + needs_move_count: 0, + operation: null, + }); + + await getJson(backend, '/__faults?imageMoves=failed', { method: 'POST' }); + + const failed = await getJson(backend, '/api/v1/image_moves/status'); + + assert.equal(failed.active_job_id, null); + assert.equal(failed.is_running, false); + assert.equal(failed.latest_job.state, 'error'); + assert.equal(failed.latest_job.error_message, 'Disk full while moving images'); + assert.equal(failed.last_error, 'Disk full while moving images'); + // A finished job needs no recovery, so another move may start. + assert.equal((await fetch(`${backend.origin}/api/v1/image_moves/start`, { method: 'POST' })).status, 202); + }); +}); + +test('image storage: Start Move waits for an idle queue, and recovery does not', async () => { + await withRepresentativeBackend(async (backend) => { + await getJson(backend, '/api/v1/app/runtime_config', { + body: JSON.stringify({ image_subfolder_strategy: 'date' }), + headers: { 'content-type': 'application/json' }, + method: 'PATCH', + }); + await clearQueue(backend); + assert.equal((await enqueueBatch(backend)).status, 201); + + // A queued item that has not run yet is active work, as `has_active_queue_work` counts it. + const refused = await fetch(`${backend.origin}/api/v1/image_moves/start`, { method: 'POST' }); + + assert.equal(refused.status, 409); + assert.deepEqual(await refused.json(), { detail: 'Cannot start image move while queue work is active' }); + assert.equal((await fetch(`${backend.origin}/api/v1/image_moves/recover`, { method: 'POST' })).status, 202); + await getJson(backend, '/api/v1/image_moves/status'); + assert.equal((await getJson(backend, '/api/v1/image_moves/status')).is_running, false); + + await clearQueue(backend); + + assert.equal((await fetch(`${backend.origin}/api/v1/image_moves/start`, { method: 'POST' })).status, 202); + }); +}); + +test('image storage: faults seed an interrupted job for recovery and a non-admin caller', async () => { + await withRepresentativeBackend(async (backend) => { + await getJson(backend, '/__faults?imageMoves=needs-recovery', { method: 'POST' }); + + const interrupted = await getJson(backend, '/api/v1/image_moves/status'); + + assert.equal(interrupted.active_job_id, 1); + assert.equal(interrupted.is_running, false); + assert.equal(interrupted.last_error, 'Server stopped during an image move'); + assert.equal((await fetch(`${backend.origin}/api/v1/image_moves/start`, { method: 'POST' })).status, 409); + + // A job awaiting recovery keeps maintenance active, so the guard still refuses image files and enqueueing. + const imageName = (await getJson(backend, '/api/v1/images/?limit=1&offset=0')).items[0].image_name; + + assert.equal((await fetch(`${backend.origin}/api/v1/images/i/${imageName}/thumbnail`)).status, 409); + assert.equal((await enqueueBatch(backend)).status, 409); + + const recovering = await fetch(`${backend.origin}/api/v1/image_moves/recover`, { method: 'POST' }); + + assert.equal(recovering.status, 202); + assert.equal((await recovering.json()).operation, 'recovery'); + await getJson(backend, '/api/v1/image_moves/status'); + + const recovered = await getJson(backend, '/api/v1/image_moves/status'); + + assert.equal(recovered.active_job_id, null); + assert.equal(recovered.is_running, false); + assert.equal(recovered.last_error, null); + assert.equal(recovered.latest_job.state, 'committed'); + assert.equal((await fetch(`${backend.origin}/api/v1/images/i/${imageName}/thumbnail`)).status, 200); + assert.equal((await enqueueBatch(backend)).status, 201); + + await getJson(backend, '/__faults?intermediatesCaller=user', { method: 'POST' }); + + assert.equal((await fetch(`${backend.origin}/api/v1/image_moves/status`)).status, 403); + assert.equal( + ( + await fetch(`${backend.origin}/api/v1/app/runtime_config`, { + body: JSON.stringify({ image_subfolder_strategy: 'date' }), + headers: { 'content-type': 'application/json' }, + method: 'PATCH', + }) + ).status, + 403 + ); + + await getJson(backend, '/__reset', { method: 'POST' }); + + assert.equal((await getJson(backend, '/api/v1/image_moves/status')).active_job_id, null); + }); +}); + +test('enqueue_batch accepts a utility graph as pending queue items with its origin', async () => { + await withRepresentativeBackend(async (backend) => { + const before = (await getJson(backend, '/api/v1/queue/default/item_ids')).total_count; + const response = await fetch(`${backend.origin}/api/v1/queue/default/enqueue_batch`, { + body: JSON.stringify({ + batch: { graph: { edges: [], id: 'pbr', nodes: {} }, origin: 'utility:pbr-1', runs: 1 }, + prepend: false, + }), + headers: { 'content-type': 'application/json' }, + method: 'POST', + }); + + assert.equal(response.status, 201); + + const enqueued = await response.json(); + + assert.equal(enqueued.enqueued, 1); + assert.equal(enqueued.item_ids.length, 1); + + const item = await getJson(backend, `/api/v1/queue/default/i/${String(enqueued.item_ids[0])}`); + + assert.equal(item.origin, 'utility:pbr-1'); + assert.equal(item.status, 'pending'); + assert.equal((await getJson(backend, '/api/v1/queue/default/item_ids')).total_count, before + 1); + }); +}); + test('shared boards answer as the backend role table does', async () => { const backend = await startMockBackend(0, { profile: 'representative' }); const call = (path, init) => @@ -1492,10 +1739,10 @@ test('control model seed sets cover both preprocessor id forms, and a PATCH chan }); }); -test('runtime config PATCH keeps the other writable field and validates the queue history limit', async () => { +test('runtime config PATCH preserves other writable fields and validates the queue history limit', async () => { await withRepresentativeBackend(async (backend) => { await getJson(backend, '/api/v1/app/runtime_config', { - body: JSON.stringify({ generation_devices: ['cpu'] }), + body: JSON.stringify({ generation_devices: ['cpu'], image_subfolder_strategy: 'date' }), method: 'PATCH', }); const withHistory = await getJson(backend, '/api/v1/app/runtime_config', { @@ -1503,7 +1750,11 @@ test('runtime config PATCH keeps the other writable field and validates the queu method: 'PATCH', }); - assert.deepEqual(withHistory.config, { generation_devices: ['cpu'], max_queue_history: 0 }); + assert.deepEqual(withHistory.config, { + generation_devices: ['cpu'], + image_subfolder_strategy: 'date', + max_queue_history: 0, + }); const invalid = await fetch(`${backend.origin}/api/v1/app/runtime_config`, { body: JSON.stringify({ max_queue_history: -1 }), diff --git a/invokeai/frontend/webv2/scripts/mock-backend.mjs b/invokeai/frontend/webv2/scripts/mock-backend.mjs index 39f64511940..8ddbf05ae95 100644 --- a/invokeai/frontend/webv2/scripts/mock-backend.mjs +++ b/invokeai/frontend/webv2/scripts/mock-backend.mjs @@ -79,12 +79,130 @@ const ARCHITECTURE_CAPABILITIES = JSON.parse( * multiuser-admin, until POST /__reset; independent of workload profile. `user` and `multiuser-admin` also turn on * multi-user auth, with a non-admin or an admin session: sign in with any email and password (or reload a page that * already holds the token). `sharedBoards: true` adds boards other accounts own (`SHARED_BOARD_FIXTURES`), which the - * caller sees through general access, a membership, or admin rights. + * caller sees through general access, a membership, or admin rights. `user` also makes the admin-only image storage + * routes answer 403. `imageMoves` seeds the image move journal: `idle` (the + * default), `needs-recovery` (an interrupted job that only Recover can finish), `failed` (a job that ended in error, + * which counts as finished) or `instant` (a move so small it commits before the POST answers, so `is_running` is + * already false). Like the real service, Start Move is refused while any queue item is pending or in progress (the + * representative profile seeds some; clear the queue first). While a move runs or a job awaits recovery, image files, + * media mutations and enqueueing answer 409 like the real maintenance guard; listings stay readable. */ const CAPABILITY_FAULTS = new Set(['ok', 'error', 'empty']); const INTERMEDIATES_CALLERS = new Set(['admin', 'user', 'multiuser-admin']); +const IMAGE_MOVE_FAULTS = new Set(['idle', 'needs-recovery', 'failed', 'instant']); +const IMAGE_MOVE_MAINTENANCE_DETAIL = 'Image storage maintenance is active'; +/** The job states the real service counts as finished; any other journaled job still needs recovery. */ +const FINISHED_IMAGE_MOVE_JOB_STATES = new Set(['committed', 'error']); +const IMAGE_STORAGE_LAYOUTS = new Set(['flat', 'date', 'type', 'hash']); +/** Status reads a running move or recovery takes to finish, so polling clients observe it in progress. */ +const IMAGE_MOVE_TICKS = 2; -const createFaults = () => ({ capabilities: 'ok', intermediatesCaller: 'admin' }); +const createFaults = () => ({ capabilities: 'ok', imageMoves: 'idle', intermediatesCaller: 'admin' }); + +/** + * Every fixture image is stored flat, so the move count is the whole library whenever the configured layout + * differs from the one the last committed move left the files in. + */ +const createImageStorage = () => ({ + layout: 'flat', + storedLayout: 'flat', + job: null, + lastError: null, + nextJobId: 1, + running: null, +}); + +const imageMoveStatus = (state) => { + const storage = state.imageStorage; + + return { + active_job_id: + storage.job !== null && !FINISHED_IMAGE_MOVE_JOB_STATES.has(storage.job.state) ? storage.job.id : null, + is_running: storage.running !== null, + last_error: storage.lastError, + latest_job: storage.job === null ? null : { ...storage.job }, + needs_move_count: storage.layout === storage.storedLayout ? 0 : state.images.size, + operation: storage.running?.operation ?? null, + }; +}; + +const seedInterruptedImageMove = (state) => { + const storage = state.imageStorage; + + storage.job = { error_message: null, id: storage.nextJobId, state: 'moving' }; + storage.nextJobId += 1; + storage.lastError = 'Server stopped during an image move'; +}; + +const seedFailedImageMove = (state) => { + const storage = state.imageStorage; + + storage.job = { error_message: 'Disk full while moving images', id: storage.nextJobId, state: 'error' }; + storage.nextJobId += 1; + storage.lastError = 'Disk full while moving images'; +}; + +const commitImageMove = (state) => { + const storage = state.imageStorage; + + if (storage.running.operation === 'move_all') { + storage.storedLayout = storage.layout; + } + if (storage.job !== null) { + storage.job = { ...storage.job, state: 'committed' }; + } + storage.running = null; +}; + +/** An `instant` move commits before the POST answers, as a tiny library does on the real server. */ +const startImageMoveOperation = (state, operation, { instant = false } = {}) => { + const storage = state.imageStorage; + + if (operation === 'move_all') { + storage.job = { error_message: null, id: storage.nextJobId, state: 'moving' }; + storage.nextJobId += 1; + } + storage.lastError = null; + storage.running = { operation, ticks: IMAGE_MOVE_TICKS }; + if (instant) { + commitImageMove(state); + } +}; + +/** Each status read advances a running operation; it commits on the last tick. */ +const tickImageMove = (state) => { + const storage = state.imageStorage; + + if (storage.running === null) { + return; + } + storage.running.ticks -= 1; + if (storage.running.ticks === 0) { + commitImageMove(state); + } +}; + +/** Mirrors `is_maintenance_active`: a running operation, or a journaled job that has not finished. */ +const isImageMoveMaintenanceActive = (state) => + state.imageStorage.running !== null || imageMoveStatus(state).active_job_id !== null; + +/** Mirrors `has_active_queue_work`, which gates Start Move: any queue item pending or in progress. */ +const hasActiveQueueWork = (state) => + [...state.queueItems.values()].some((item) => item.status === 'pending' || item.status === 'in_progress'); + +/** + * The real maintenance guard covers image uploads, deletions, edits, star changes, copies, downloads, image files + * and board-image membership, plus deleting a board with its images; listings and DTO reads stay open. + */ +const isImageMoveMaintenanceRoute = (method, path, url) => { + if (path.startsWith('/api/v1/board_images')) { + return true; + } + if (path.startsWith('/api/v1/images')) { + return method !== 'GET' || /^\/api\/v1\/images\/i\/[^/]+\/(full|thumbnail|workflow)$/.test(path); + } + return method === 'DELETE' && path.startsWith('/api/v1/boards/') && url.searchParams.get('include_images') === 'true'; +}; const isAdminCaller = (faults) => faults.intermediatesCaller !== 'user'; const isMultiuserCaller = (faults) => faults.intermediatesCaller !== 'admin'; @@ -346,6 +464,7 @@ const createState = (profile) => { ]), externalProviders: createExternalProviderConfigs(fixture.models), images: new Map(fixture.images.map((image) => [image.image_name, clone(image)])), + imageStorage: createImageStorage(), models: new Map(fixture.models.map((model) => [model.key, clone(model)])), /** What re-identify fills in per model key; only `POST /__models` seeds entries. */ modelProbes: new Map(), @@ -1111,6 +1230,7 @@ export const startMockBackend = async (port, { profile = 'empty' } = {}) => { const body = await readJsonBody(request); const requested = url.searchParams.get('capabilities') ?? body.capabilities; const caller = url.searchParams.get('intermediatesCaller') ?? body.intermediatesCaller; + const imageMoves = url.searchParams.get('imageMoves') ?? body.imageMoves; if (body.sharedBoards === true && !state.boards.has(SHARED_BOARD_FIXTURES[0].board_id)) { seedSharedBoards(state); @@ -1131,6 +1251,21 @@ export const startMockBackend = async (port, { profile = 'empty' } = {}) => { faults.intermediatesCaller = caller; } + + if (imageMoves !== undefined && imageMoves !== null) { + if (!IMAGE_MOVE_FAULTS.has(imageMoves)) { + return json(400, { detail: `Unknown image moves fault: ${String(imageMoves)}` }); + } + + faults.imageMoves = imageMoves; + state.imageStorage = createImageStorage(); + if (imageMoves === 'needs-recovery') { + seedInterruptedImageMove(state); + } + if (imageMoves === 'failed') { + seedFailedImageMove(state); + } + } } return json(200, { ok: true, faults: { ...faults } }); @@ -1256,26 +1391,85 @@ export const startMockBackend = async (port, { profile = 'empty' } = {}) => { } if (path === '/api/v1/app/runtime_config') { - if (method === 'PATCH') { - const changes = await readJsonBody(request); + const runtimeConfig = () => ({ + config: { ...state.runtimeConfig, image_subfolder_strategy: state.imageStorage.layout }, + set_fields: state.imageStorage.layout === 'flat' ? [] : ['image_subfolder_strategy'], + }); - if ('generation_devices' in changes) { - state.runtimeConfig.generation_devices = changes.generation_devices; + if (method === 'GET') { + return json(200, runtimeConfig()); + } + if (method === 'PATCH') { + if (!isAdminCaller(faults)) { + return json(403, { detail: 'Admin privileges required' }); } - if ('max_queue_history' in changes) { - const value = changes.max_queue_history; + + const body = await readJsonBody(request); + + if ('max_queue_history' in body) { + const value = body.max_queue_history; if (value !== null && !(Number.isInteger(value) && value >= 0)) { return json(422, { detail: 'max_queue_history must be a non-negative integer or null' }); } - state.runtimeConfig.max_queue_history = value; } + if ('image_subfolder_strategy' in body) { + if (!IMAGE_STORAGE_LAYOUTS.has(body.image_subfolder_strategy)) { + return json(422, { detail: 'image_subfolder_strategy must be one of flat, date, type, hash' }); + } + + state.imageStorage.layout = body.image_subfolder_strategy; + } + if ('generation_devices' in body) { + state.runtimeConfig.generation_devices = body.generation_devices; + } + if ('max_queue_history' in body) { + state.runtimeConfig.max_queue_history = body.max_queue_history; + } + + return json(200, runtimeConfig()); + } + } + + if (path.startsWith('/api/v1/image_moves/')) { + if (!isAdminCaller(faults)) { + return json(403, { detail: 'Admin privileges required' }); + } + + const storage = state.imageStorage; + + if (method === 'GET' && path === '/api/v1/image_moves/status') { + tickImageMove(state); + return json(200, imageMoveStatus(state)); + } + if (method === 'POST' && path === '/api/v1/image_moves/start') { + if (storage.running !== null) { + return json(409, { detail: 'An image move job is already running' }); + } + if (imageMoveStatus(state).active_job_id !== null) { + return json(409, { detail: 'An image move job is already active' }); + } + if (hasActiveQueueWork(state)) { + return json(409, { detail: 'Cannot start image move while queue work is active' }); + } + + startImageMoveOperation(state, 'move_all', { instant: faults.imageMoves === 'instant' }); + return json(202, imageMoveStatus(state)); } - if (method === 'GET' || method === 'PATCH') { - return json(200, { config: { ...state.runtimeConfig }, set_fields: [] }); + if (method === 'POST' && path === '/api/v1/image_moves/recover') { + if (storage.running !== null) { + return json(409, { detail: 'An image move job is already running' }); + } + + startImageMoveOperation(state, 'recovery', { instant: faults.imageMoves === 'instant' }); + return json(202, imageMoveStatus(state)); } } + if (isImageMoveMaintenanceActive(state) && isImageMoveMaintenanceRoute(method, path, url)) { + return json(409, { detail: IMAGE_MOVE_MAINTENANCE_DETAIL }); + } + if (path.startsWith('/api/v1/app/invocation_cache')) { const cache = state.invocationCache; @@ -1734,6 +1928,55 @@ export const startMockBackend = async (port, { profile = 'empty' } = {}) => { const queueMatch = /^\/api\/v1\/queue\/[^/]+\/(.+)$/.exec(path); if (queueMatch) { const action = queueMatch[1]; + + if (action === 'enqueue_batch' && method === 'POST') { + if (isImageMoveMaintenanceActive(state)) { + return json(409, { detail: IMAGE_MOVE_MAINTENANCE_DETAIL }); + } + + const { batch = {} } = await readJsonBody(request); + const runs = Number.isInteger(batch.runs) && batch.runs > 0 ? batch.runs : 1; + const now = timestamp(state); + const batchId = `mock-batch-${String(state.mutationClock)}`; + const itemIds = []; + + // Items stay pending: the mock has no processor and no socket, so nothing ever completes them. + for (let run = 0; run < runs; run += 1) { + const itemId = Math.max(0, ...state.queueItems.keys()) + 1; + + itemIds.push(itemId); + state.queueItems.set(itemId, { + batch_id: batchId, + completed_at: null, + created_at: now, + destination: batch.destination ?? null, + error_message: null, + error_traceback: null, + error_type: null, + field_values: null, + item_id: itemId, + origin: batch.origin ?? null, + retried_from_item_id: null, + session: { prepared_source_mapping: {}, results: {} }, + session_id: `mock-session-${String(itemId)}`, + started_at: null, + status: 'pending', + updated_at: now, + user_display_name: 'Fixture User', + user_email: 'fixture@example.test', + user_id: MOCK_USER_ID, + }); + } + + return json(201, { + batch: { ...batch, batch_id: batchId }, + enqueued: itemIds.length, + item_ids: itemIds, + priority: 0, + queue_id: 'default', + }); + } + const scopedItems = queueItemsForScope(state, url); if (action === 'status') { diff --git a/invokeai/frontend/webv2/src/architecture/dependencyPolicy.test.ts b/invokeai/frontend/webv2/src/architecture/dependencyPolicy.test.ts index 812832d1ce7..d19c4dc5450 100644 --- a/invokeai/frontend/webv2/src/architecture/dependencyPolicy.test.ts +++ b/invokeai/frontend/webv2/src/architecture/dependencyPolicy.test.ts @@ -225,6 +225,7 @@ describe('feature public-interface registry', () => { expect(checkDependency('workbench/palette/paletteProviders.ts', '@features/workflow/queries')).toEqual([]); expect(checkDependency('workbench/palette/paletteProviders.ts', '@features/workflow/paletteSearch')).toEqual([]); expect(checkDependency('workbench/palette/paletteProviders.ts', '@features/gallery/paletteSearch')).toEqual([]); + expect(checkDependency('workbench/settings/ImageStorageSettings.tsx', '@features/gallery/storage')).toEqual([]); expect(checkDependency('workbench/palette/paletteProviders.ts', '@features/queue/queries')).toEqual([]); expect(checkDependency('workbench/palette/WorkbenchCommandPaletteDialog.tsx', '@features/queue/queries')).toEqual( [] diff --git a/invokeai/frontend/webv2/src/architecture/featureInterfaces.ts b/invokeai/frontend/webv2/src/architecture/featureInterfaces.ts index e76f3d9bd87..4fae8e20539 100644 --- a/invokeai/frontend/webv2/src/architecture/featureInterfaces.ts +++ b/invokeai/frontend/webv2/src/architecture/featureInterfaces.ts @@ -12,6 +12,7 @@ export const FEATURE_PUBLIC_INTERFACES: Readonly