set_react_page: automatically deliver dependencies for renderers registered after page load

[schloerke]

Follow-up to #87. Builds on the automatic dependency discovery shipped there (synchronously-mounted top-level + @module.server renderers) and the @render.ui dynamic-UI path (delivered natively by Shiny).

Goal

Make the server-registers-a-renderer-after-page-load + React-supplied placeholder pattern work automatically, with no user ceremony. Example:

@reactive.effect
@reactive.event(input.add)
async def _():
    @render_plotly
    def scatter():
        return px.scatter(...)
    get_current_session().output(scatter)   # registered after the page is built

…with the React client mounting <ShinyOutput id="scatter" .../>. Today the chart never renders: the renderer's HTMLDependency (the binding JS) was never delivered, so the element can't bind.

What we already ruled in / out (investigation summary)

All verified with Playwright spikes against shinywidgets render_plotly:

1. Value delivery is NOT the problem. A dynamically-registered output sends its value fine; Shiny's bindOutput even replays a stored $values[id] when the element later binds. A dynamically-registered render.text (binding JS in core shiny.js) renders correctly.
2. The problem is the missing dependency. dep_scripts=0 → bound=0 → no render. Proof: a ui.hold()-ed warm-up render_plotly (so Layer A injects the binding JS at startup) makes the dynamically registered output render (bound=1, plotly=1).
3. Naive late-push does NOT work for widgets. Pushing the dep after registration + renderDependenciesAsync + bindAll fails with Error: No model found for id …. shinywidgets sends the widget model via a comm_open custom message alongside the value; because the dep (which registers the comm handler) loads asynchronously after comm_open arrives, the message is dropped and the model is never created. This is a fundamental ordering problem, not a timing tweak (retrying bindAll for 2.4s did not help — the model is gone).
4. A re-trigger handshake DOES work (bound=1, plotly=1): push deps → client loads them → client signals server → the output recomputes → shinywidgets re-sends comm_open with its handler now present.
5. But auto-triggering that recompute is the open problem. The handshake only worked when the renderer depended on a reactive that the deps-ready signal bumped (one line of user cooperation). Two no-cooperation triggers failed:
   • re-registering the renderer (session.output(renderer) again) → bound=0;
   • manually invalidating the output effect's context (effect._ctx.invalidate()) → bound=0 (invalidation without Shiny's flush-scheduling pipeline doesn't drive the recompute).

What "automatic" needs (remaining work)

A way to force a registered output to recompute through Shiny's normal reactive pipeline — without the user's renderer explicitly depending on a deps-ready signal. Candidate directions to explore:

• A supported force-invalidate-and-flush primitive (invalidate the output context and request a session flush), rather than the private effect._ctx poke that didn't drive the recompute.
• A suspend_when_hidden-based dance: register the dynamic output suspended/hidden, push deps, then reveal it so Shiny's resume-on-show recomputes it after deps load.
• A shinyreact-owned reactive that dynamic outputs implicitly depend on (would likely require wrapping the renderer at registration).

Interim opt-in fallback (proven, ~1 line): have the dynamic renderer depend on a shinyreact-provided "deps-ready" reactive so the existing reactive path recomputes it.

Related

• set_react_page: HTMLDependency harvesting misses renderers inside Shiny modules #87 — automatic dep discovery for synchronously-mounted renderers (Layer A).
• Design/investigation notes: docs/superpowers/specs/2026-06-11-dynamic-renderer-dep-delivery-design.md.

[schloerke]

Status update — half of this is now automatic; the hard half is not.

Dep delivery is solved, in all three page modes. _dep_discovery.py (Python, #249) and pkg-r/R/dep-discovery.R (R, #221) install a per-session post-flush hook that diffs the session's registered outputs, extracts each new output's auto_output_ui(), and pushes any not-yet-sent HTMLDependency as a shinyreact-deps custom message; the bundle loads them and re-runs bindAll. It diffs on every flush, not just the first, so an output registered after page load — including a module server mounted inside an observer — does get its dependency. The hook is a .shinyreact_init client ping, so it needs no user wiring.

What this issue was really about is still open: the recompute. Finding 3 above (shinywidgets drops comm_open because the dep loads asynchronously after the message arrives) is an ordering problem that late-pushing the dep does not fix, and finding 5 — forcing a registered output to recompute through Shiny's normal reactive pipeline, without the renderer explicitly depending on a deps-ready signal — has no answer yet. So a dynamically registered render_plotly still needs the one-line reactive-dependency cooperation.

Retitling territory: the remaining work is "re-trigger a dynamic output after its deps load", and it is no longer set_react_page-specific.