Fix graph execution edge cases

invokeai/app/services/shared/README.md

@@ -24,10 +24,12 @@ fan-out. Five or more nested `If`s, other fan-out, mixed graphs outside the boun
 topology, loop-containing, saved-workflow, legacy, and five-or-more sibling shapes remain on the compatibility
 fallback. Legacy snapshots retain their generic compatibility projection and legacy skipped-state metadata. Legacy
 skipped-state projection remains only for old snapshots. Fresh `If` materialization prepares the condition boundary
-first, resolves the activation token, and attaches only the selected branch input. Unselected branch nodes are never
-prepared, skipped, or added to fresh execution history. Both scheduler adapters consume these dependency decisions
-without pruning execution edges or calling a type-specific branch scheduler. Legacy skipped-state metadata remains only
-for snapshots that already contain the old projection. Branch-membership analysis remains an internal author-graph
+first, resolves the activation token, and attaches only the selected branch input. Nodes exclusive to the unselected
+branch are not materialized or invoked. `executed_history` records source-level completion and retirement: a branch
+source retired as empty can appear there without an invocation or an execution result. It is bookkeeping, not an
+invocation trace. Both scheduler adapters consume the same dependency decisions without pruning execution edges or
+calling a type-specific branch scheduler. Legacy skipped-state metadata remains only for snapshots that already contain
+the old projection. Branch-membership analysis remains an internal author-graph
 admission decision. The focused fresh-shape ownership matrix also verifies that supported flat, nested, bounded
 fan-out, and four-sibling graphs do not instantiate the compatibility controller or project fresh skipped nodes in
 either scheduler route. This proves the current ownership boundary; it does not remove the controller, which remains
@@ -347,6 +349,13 @@ mutation helpers. Those helpers reject changes once the affected nodes have alre
   - source node id
   - iteration path
   - runtime state such as pending, ready, executed, or skipped
+- _engine_frame() stores canonical immutable frames in the owning state's private runtime, keyed by state ID,
+  workflow-call depth, and iteration path. The cache is process-local, is not serialized, and retains only the distinct
+  frames requested while that state is alive. State copies get their own cache dictionary; a copy with a new ID starts
+  with no frames for the old ID.
+- If activation dependency lookup has a reverse index from If owner, selected branch, and frame to prepared exec IDs.
+  A restored state rebuilds the index once; newly prepared nodes extend it. The index is derived runtime state, not
+  snapshot data.
 - `execution_refs: dict[str, ExecutionReference]` - in-memory stable references for prepared execution nodes, including
   their source node and execution frame; version-2 snapshots rebuild these references instead of storing them.
 - `execution_tokens: dict[str, ExecutionToken]` - in-memory output tokens produced by applied execution results;
@@ -437,7 +446,10 @@ same adapter now projects ordinary readiness and calls the graph-state generic c
 prepared iteration when needed, and finalizes `output_collection` and `final_state`. For an empty literal collection,
 the generic path completes through the existing synthetic terminal `For` result without running the body or
 `ForReturn`. The generic scheduler receives only opaque node IDs and dependencies; it never receives a literal next
-node ID. The compatibility continuation bridge remains available only for unsupported loop shapes and explicitly
+node ID. Compatibility nested-For expansion also checks whether any prepared node is the synthetic empty
+terminal before finalizing empty contexts; non-empty sibling contexts still run their bodies. This handles mixed outer
+collections such as [[], [1]] as well as all-empty collections. The compatibility continuation bridge remains available
+only for unsupported loop shapes and explicitly
 legacy-loaded snapshots. The exact four-node/four-edge empty input-driven shape with one inputless static-list producer
 is also generic-routed. Other input-driven outer collections, five-level-or-deeper or sibling nested loops outside the
 exact two-sibling `CollectionConcat` fan-in contract, or unsupported mixed loop shapes remain compatibility-owned; the
@@ -653,11 +665,29 @@ materialization.
   iterator ancestors because the source path crosses a collector, prepared parent iteration paths are still used to
   materialize one downstream exec node for each preserved collector path.
 
-- For a fresh **IfInvocation**: prepare only its condition inputs first. After the condition resolves, graph-state
-  activation dependencies admit selected branch contexts and attach only the selected branch input to the pending `If`
-  execution. Unsupported fresh shapes and legacy snapshots use the compatibility controller. Rejected branch sources
-  are not materialized. Pending `If` state is reconstructed from its durable execution edges, prepared mappings, and
-  activation token after rehydration.
+- For an IfInvocation with a wired condition, prepare condition inputs first. Readiness is constrained to condition
+  source nodes and their ancestors; iterator ancestry on a branch cannot hold the gate open. After the condition resolves,
+  activation dependencies admit selected branch contexts and attach only the selected branch input. Literal conditions
+  use their known value and selected-branch ancestry. Unsupported fresh shapes and legacy snapshots use the compatibility
+  controller. Rejected branch sources are retired empty without invocation. Pending If state is reconstructed from durable
+  execution edges, prepared mappings, and the activation token after rehydration.
+
+- A branch `Collect` whose `item` source combines exactly two sibling `Iterate` axes can close the axis beside the
+  `If` condition axis. The compatibility materializer expands that sibling `Iterate` beneath each resolved `If` frame
+  selecting the `Collect` branch, pairs only the matching condition frame, and closes the branch interval empty when no
+  frame selects it. Independent iterator axes follow destination input-port order; node insertion order and node IDs do
+  not reorder distinct ports. The condition and branch axes must be nested; divergent inner axes are rejected even when
+  they share an outer axis. An uncollected deeper branch axis that cannot be selected from the condition frame raises
+  `UnsupportedIfIterationAxesError` before invocation. One sibling `Iterate` can belong to only one `If` owner. A
+  `Collect` may feed both branch ports of that same `If`; if it would serve different `If` owners, materialization
+  raises `UnsupportedSharedIfCollectBranchesError` before invocation rather than dropping frames.
+
+- For a root-conditioned `If` whose selected branch input is a completed direct `Collect`, compatibility
+  materialization preserves each non-root `Collect` iteration path through ordinary descendants. It runs one `If`
+  execution per preserved group, with a matching activation frame; the already-resolved root activation gate and
+  token are aligned to the first group. Ordinary descendants of `If.value` use those same frames, and a downstream
+  `Collect` gathers every completed group through the standard grouping behavior. This bounded case does not imply
+  propagation through further nested controls or loops.
 
 - For each new exec node:
 
@@ -669,6 +699,56 @@ materialization.
   - Try to resolve any `If`-specific scheduling state.
   - If the node is ready and not deferred by an unresolved `If`, enqueue it into its class queue.
 
+`next()` applies `_GenericForPlanner.prepare()` and If-bearing `_ExecutionMaterializer.prepare()` calls as state
+transactions. If one of those expansions raises, journaled execution-graph, prepared-mapping, frame,
+activation-token, gate, and scheduler mutations are rolled back; derived caches are reset and ready queues are rebuilt
+from the restored state before the exception propagates. A later `next()` can retry from the pre-expansion state
+without retaining partially materialized nodes or activation state. If-free `_ExecutionMaterializer.prepare()` runs
+without this transaction wrapper to avoid overhead on ordinary graphs; root-If completed-Collect expansion retains
+its own transaction.
+
+This transaction boundary does not cover pending-If attachment (`_attach_pending_if_inputs`),
+`prepare_nested_iterate_sequence()`, or direct Iterate/Collect preparation. Those operations run outside the wrapper
+and must be considered separately when reasoning about rollback behavior.
+
+The execution-node builder retains a filtered iterator graph for the current source-graph identity. It builds this
+graph when compatibility preparation needs sibling-Collect relationships or iterator ancestry. The graph replaces
+Collect item inputs with its collection inputs for traversal; edge-scoped ancestry retains item axes needed for grouping.
+Final For outputs do not carry an open iteration axis.
+
+Iterator ancestry owns ordered ancestor and edge-axis analysis plus effective-path memoization for one graph traversal.
+The builder shares that object across candidate admission and materialization. Its separate per-source axis cache adds
+sibling-Collect relationship ordering and is cleared when that relationship graph is established; it does not repeat
+the ancestor-list cache. Independent parent paths merge in destination input-port order, with source node ID used only
+to break ties on the same input port. A standalone path lookup owns a separate context. These caches are process-local
+and rebuilt when graph identity changes; they are not serialized.
+
+Each pending-If attachment sweep also builds a local index from source edge and prepared iteration path to execution
+node ID. It enumerates each prepared source edge once per sweep, then resolves the deepest matching path prefix for each
+condition-ready If frame. This preserves deterministic attachment while avoiding a repeated full prepared-source scan
+for every If frame. The index is discarded at the end of the call; a later sweep rebuilds it from the current prepared
+mapping, so it is not retained across graph edits, rollback, or state restore. This optimization bounds candidate lookup
+for attachment; other preparation and branch-context work has separate costs.
+
+Inactive branch-source retirement uses an event queue owned by one materializer. A rejected branch queues only its
+activation-dependent source IDs when the corresponding `If` resolves; an empty condition `Iterate` queues both branch
+source sets. Each preparation pass checks the queued sources in topological order rather than rescanning every
+unprepared source. The index is tied to source-graph identity, rebuilt from durable activation state after graph changes
+or state restore, and is not serialized. Queue mutations participate in execution-state rollback. A process-local
+completed-source ID cache avoids rescanning the full `executed` set for every inactivity check; For/Iterate completion
+invalidates it because axis completion can change branch activity. Registering a new prepared execution evicts its
+source ID from the cached set; re-registering an existing execution does not. Graph edits or transaction rollback clear
+the derived cache. This bounds source-inactivity checks for serial If chains, but does not bound other preparation or
+branch-context costs.
+
+Literal-condition If fallback path inference uses the same ancestry context, including nested selected branches.
+
+Ordinary source nodes are traversed with an explicit stack, but path inference recurses when a selected input comes from
+another unprepared `If`. A direct lookup over a 400-node literal-`If` chain raises `RecursionError` with CPython 3.12's
+default recursion limit of 1,000; serial execution of the equivalent 1,100-node graph completes. This is a pathological
+depth limit in the unresolved compatibility fallback, not a general graph-size limit. Supporting arbitrary nested-`If`
+depth requires converting that path inference to an explicit stack.
+
 ### 4.5 Readiness and class ordering
 
 - `_enqueue_if_ready(nid)` applies generic readiness: `indegree == 0`, not executed or claimed, and, for an `If`
@@ -716,12 +796,17 @@ Run `C` -> `D:0` -> enqueue `D`. Run `D` -> done.
 - Nodes that are exclusive to the true or false branch remain unmaterialized until their branch is admitted, even when
   their indegree would otherwise be zero.
 - Once the condition resolves, graph-state activation logic records the selected activation token and attaches only the
-  selected branch input to the pending `If` execution. Branch-exclusive ancestors of the unselected branch are never
-  prepared, executed, or added to fresh execution history. Unsupported fresh shapes and legacy snapshots use the
+  selected branch input to the pending `If` execution. Branch-exclusive ancestors of the unselected branch are not
+  invoked or materialized into execution results. Their source IDs can appear in `executed_history` after empty
+  retirement; the history is not an invocation trace. Unsupported fresh shapes and legacy snapshots use the
   compatibility controller. Both scheduler adapters use append-only execution edges; no type-specific branch scheduler
   performs pruning or skip propagation.
 - Legacy snapshots that already contain skipped prepared nodes retain their compatibility metadata and scheduler
   projection. Fresh execution does not create that projection.
+- If an `Iterate.item` stream feeding `If.condition` closes empty, no `If` execution is created. The branch sources are
+  retired empty, the downstream `Collect` produces an empty collection, and the state completes without an error.
+  Retirement may add the branch source IDs to `executed_history`; neither branch invocation runs and no branch result
+  is created.
 - The SQLite queue/processor path has evidence for cancellation before and after `If` resolution and retry from each
   boundary for both condition polarities. A canceled attempt keeps its activation ledger and cannot resume; retry
   starts a fresh state and must emit a fresh matching activation token before selected-only completion.

invokeai/app/services/shared/execution_engine/primitives.py

@@ -51,6 +51,8 @@ class _InternalModel(BaseModel):
     def __eq__(self, other: object) -> bool:
         """Compare durable fields, excluding process-local synchronization locks."""
 
+        if self is other:
+            return True
         if not isinstance(other, type(self)):
             return NotImplemented
         return self.model_dump(mode="python") == other.model_dump(mode="python")