Python: a lone recipe bundle runs in the facade's own process

[knutwannheden]

A mod run of a pip recipe bundle uses two Python processes and sends the whole LST between them. The facade deserializes the tree from Java and re-serializes it to a child that runs the recipes — a hop that produces no edits.

Measured on jd/tenacity (20 files, 5,896 LOC) with org.openrewrite.python.migrate.UpgradeToPython313, three interleaved pairs:

              wall   fix.patch   python CPU   procs   inbound msgs   facade→child
main-1         94s   c0a2a2ab        66.0s      2       4,032,543      1,303,024
main-2        120s   c0a2a2ab        77.0s      2       4,133,014      1,303,024
main-3         86s   c0a2a2ab        61.7s      2       4,157,577      1,303,024
branch-1       68s   c0a2a2ab        49.8s      1       2,806,383              0
branch-2       51s   c0a2a2ab        37.5s      1       2,434,323              0
branch-3       68s   c0a2a2ab        49.6s      1       2,599,951              0

c0a2a2ab… is the recorded baseline hash, so the patch is byte-identical in all six runs. Another session's RPC server competed for CPU throughout, so wall and CPU are directional; facade→child is deterministic.

Why one bundle is the normal case

The facade came from #8275: recipes from different bundles shared one environment, so their dependencies could conflict. Serving each child a diff over that child's ref table means the facade holds the tree as live objects, so every tree is materialized twice.

But a mod run executes one recipe, so normally one bundle participates — the multi-bundle case is a YAML composite spanning two pip packages — and a lone bundle has nothing to be isolated from.

What changed

BundleChildren._discover is the one place that decides. A bundle installed while no other exists is activated in this process:

• its venv joins sys.path via site.addsitedir, which appends, so the engine keeps the precedence _child_env gives it in a child
• discover_root_recipes scopes discovery to that distribution, as --child-bundle does for a child
• a second bundle moves the first behind a child and spawns one for the newcomer

handle_request follows that decision. InstallRecipes and GetMarketplace always go to the facade; SetDataTableStore and Evict go to the facade and the local handler; everything else only when Facade.routes_to_children().

With one bundle that predicate is false, so PrepareRecipe, Visit, BatchVisit, Generate, Print and GetObject reach the plain handlers — the same code a child runs — and no _hub_* state is touched.

What bounds the hosting

• A venv built by another interpreter is left to a child. is_usable_venv checks only that pyvenv.cfg's home still exists, never the version, and a 3.11 site-packages appended to a 3.12 path breaks any compiled dependency. built_by_running_interpreter reads the recorded version.
• A process takes one bundle's imports, ever. Imports are permanent, so _imported is sticky where _hosted is not: a bundle installed after the hosted one is uninstalled starts behind a child.

activate is injected, not imported

bundle_children first reached the hook with from rewrite.rpc.server import activate_bundle_in_process. The server runs as python -m rewrite.rpc.server, so it is __main__ and that binds a second module object with its own empty marketplace.

Activation filled that copy with 134 recipes while the running server's stayed empty, and every run died at PrepareRecipe — through the delegation path, so it looks nothing like an import fault:

java.lang.IllegalStateException: No available resolver for 'pip' ecosystem
  org.openrewrite.marketplace.RecipeListing.resolve(RecipeListing.java:93)
  org.openrewrite.rpc.RewriteRpc.recipeFromPrepareResponse(RewriteRpc.java:524)
  org.openrewrite.python.marketplace.PipRecipeBundleReader.prepare(PipRecipeBundleReader.java:47)

activate is therefore a required keyword: missing wiring is a TypeError, not a silent fall back to the two-process path this change removes. java_rpc_client.py:137 has the same shape and is untouched.

Tests

Seven, each pinning one line:

• the hosting branch, and the eviction branch a second install takes
• the interpreter check, and _imported outliving _hosted across an uninstall
• the routes_to_children guard, asserting _hub_tree stays empty
• the missing return that gets a hosted bundle its data-table store
• the facade-scoped marketplace

Scope

The _hub_* block, child_connection and the facade's routing are now dead code on the path a mod run takes. They stay: they are the multi-bundle fallback, and deleting them would drop multi-bundle support rather than simplify it.

Eviction rests on bundle installs completing before any recipe is prepared, which PipRecipeBundleResolver does by running during marketplace resolution. If that stopped holding, a Visit fails with No child owns visitor — loudly.