Skip to content

feat: Round-robin session queue scheduling across users#127

Open
Copilot wants to merge 5 commits intomainfrom
copilot/enhancement-round-robin-job-scheduling
Open

feat: Round-robin session queue scheduling across users#127
Copilot wants to merge 5 commits intomainfrom
copilot/enhancement-round-robin-job-scheduling

Conversation

Copy link
Copy Markdown

Copilot AI commented Mar 10, 2026

In multiuser mode, a single user could monopolize the queue by enqueueing large batches, forcing other users to wait indefinitely. This adds a round_robin queue mode that interleaves jobs across users so each gets a turn before any user gets a second slot.

Changes

  • New config field session_queue_mode ("FIFO" | "round_robin", default "round_robin"): controls dequeue ordering. Configurable via invokeai.yaml, env var (INVOKEAI_SESSION_QUEUE_MODE), or CLI.
  • Single-user mode always uses FIFOsession_queue_mode is ignored when multiuser=False.
  • Round-robin dequeue() SQL: uses two CTEs — user_last_served tracks MAX(started_at) per user; user_next_item selects each user's best pending item (priority DESC, item_id ASC). Rows are ordered by COALESCE(last_served_at, '1970-01-01') ASC so the least-recently-served user always goes next.
  • Tests: 10 new tests covering FIFO and round-robin behavior, including the exact interleaving example from the issue:
Queued Processed
A1, A2, B1, C1, C2, A3 A1, B1, C1, A2, C2, A3

QA Instructions

  1. Run with multiuser: true in invokeai.yaml (default session_queue_mode: round_robin).
  2. Enqueue several batches as two different users — confirm jobs alternate per user rather than draining one user's queue fully before moving to the next.
  3. Set session_queue_mode: FIFO and confirm strict insertion-order is restored.
  4. Run with multiuser: false — confirm FIFO is used regardless of session_queue_mode.

Run the new unit tests:

pytest tests/app/services/session_queue/test_session_queue_dequeue.py -v

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)
Original prompt

This section details on the original issue you should resolve

<issue_title>[enhancement]: Round-robin generation sessions across users</issue_title>
<issue_description>### Is there an existing issue for this?

  • I have searched the existing issues

Contact Details

No response

What should this feature add?

Right now, when running in multiuser mode, the session manager takes jobs off the queue in a FIFO manner. However, when multiple users are working with the same backend, a greedy user can enqueue 100 rendering jobs, forcing all other users to wait their turn.

This feature would change the queue logic such that the jobs for each active user are dequeued in such a way that each user is served in turn. That is, if there are users A, B and C, and their jobs are queued like this:

A job 1
A job 2
B job 1
C job 1
C job 2
A job 3

They will be processed in this order:

A job 1
B job 1
C job 1
A job 2
C job 2
A job 3

The dequeueing behavior should be controlled by a configuration variable session_queueing with one of the values 'FIFO' (traditional behavior) or round_robin (new behavior).

If multiuser mode is active, then round_robin is the default. If in single user mode, use FIFO and ignore the round_robin option.

Alternatives

No response

Additional Content

No response</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

- Add SESSION_QUEUE_MODE type and session_queue_mode config field
- Modify dequeue() to support round-robin ordering when multiuser mode
  is active, serving each user in turn based on last-served timestamp
- Add tests for FIFO and round-robin dequeue behavior

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Copilot AI changed the title [WIP] Add round-robin generation sessions across users feat: Round-robin session queue scheduling across users Mar 10, 2026
@lstein lstein marked this pull request as ready for review April 25, 2026 16:53
@lstein lstein self-requested a review as a code owner April 25, 2026 16:53
Three regressions from the multiuser isolation work in 33ec16d were
preventing non-admin users from seeing the broader queue:

1. The "X/Y" pending badge collapsed to a single number because the
   backend stopped returning per-user counts and the frontend dropped the
   X/Y formatting. Restored user_pending/user_in_progress on
   SessionQueueStatus and the X/Y formatter; get_queue_status now takes
   an explicit is_admin flag for current-item visibility.

2. The queue list only showed the caller's own jobs because
   get_queue_item_ids filtered by user. Per-item field redaction already
   happens in list_all_queue_items / get_queue_items_by_item_ids, so the
   id list itself can be returned unfiltered.

3. After enqueue or status change in another user's batch, A's queue
   list, badge totals, and item statuses stayed stale until reload because
   QueueItemStatusChangedEvent and BatchEnqueuedEvent went only to
   user:{owner} + admin rooms. Now the full event still goes to those
   rooms, and a sanitized companion (user_id="redacted", identifiers and
   error fields stripped) is broadcast to the queue room with the owner
   and admin sids in skip_sid so they don't receive a clobbering
   duplicate. The frontend handler short-circuits the redacted variant to
   tag invalidation only, skipping per-session side effects.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Run via `pnpm run generate-docs-data`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[enhancement]: Round-robin generation sessions across users

2 participants