Orbit as a Galaxy analysis workbench (standalone + Galaxy Interactive Tool)#330
Open
dannon wants to merge 30 commits into
Open
Orbit as a Galaxy analysis workbench (standalone + Galaxy Interactive Tool)#330dannon wants to merge 30 commits into
dannon wants to merge 30 commits into
Conversation
dannon
force-pushed
the
feat/orbit-gxit-page-sync
branch
from
June 19, 2026 09:33
8cb8a86 to
cae0ebe
Compare
dannon
marked this pull request as ready for review
June 19, 2026 09:59
Import initGalaxyPageSync and flushNotebookToGalaxy into the lifecycle, and call them at the appropriate hooks: init resumes the per-history page on launch, and flush guarantees one final push (including any session-summary block) on shutdown.
Live testing against test.galaxyproject.org showed resume-on-launch silently
failed: a fresh container only knows the derived slug (orbit-<historyId>), but
Galaxy's GET /pages/{id} 400s on a slug. List the history's pages, match the
slug, and resume by the real page id. Without this a fresh container also can't
update the existing page -- the next push would try to create a duplicate slug.
Tool XML (scoped api_key + $__galaxy_url__ injection, port 3000, ALLOW_INSECURE trust model, explicit server start command) and a deploy README covering image build, registration, the admin-key-via-job_conf path, a custom-provider option, and the trust model. Claude-Session: https://claude.ai/code/session_015LfwTStrqyxDTCv19PbTkT
The container never booted: web/server.ts imports ./auth.js and ./rpc-guard.js (added by the auth-hardening commit), but the runner stage's COPY list was never updated to include them -> ERR_MODULE_NOT_FOUND /app/web/auth.js at startup. Caught by the Plan 3 container smoke test; verified the image now boots, serves the prebuilt renderer, and the agent responds.
Live GxIT testing on a local Galaxy surfaced two things: Galaxy injects the host TMPDIR into the container (a macOS /var/folders path that tsx can't use -> EACCES), so the command forces TMPDIR=/tmp; and node:22-slim has no uvx, so galaxy-mcp (tool/workflow execution) fails in-container while notebook->Page persistence (direct API) still works. Documented both in the deploy guide.
The prior commit documented that node:22-slim has no Python or uvx, so the agent's Galaxy tool/workflow surface (galaxy-mcp, launched via `uvx galaxy-mcp>=1.8.0`) never started in the container and Galaxy tools silently vanished. Fix it: copy uv from Astral's published image, point its cache, python-install, and tool dirs at a node-owned /opt/uv, and pre-warm `uv tool install galaxy-mcp>=1.8.0` at build so the package and a managed Python are baked in. Verified the server now launches and answers an MCP initialize fully offline (--network none), resolving entirely from the baked cache -- so it works even in a network-locked job container. Notebook->Page persistence was already fine (direct API); this restores the execution half.
Live GxIT testing surfaced a second blocker after uvx: the agent could
connect to galaxy-mcp but couldn't invoke any of its tools. On a fresh
container pi-mcp-adapter never registers the galaxy_*/brc_analytics_* direct
tools -- that needs a warm per-server metadata cache, and the per-user scoped
GALAXY_API_KEY changes the cache's config hash every launch, so it's always
cold. The only path to those servers is then the single `mcp` proxy tool,
which the gate denied by default ("mcp is not available in remote mode").
Allow the `mcp` proxy, but scope it: a tool call (or connect) must target one
of the curated servers (galaxy, brc-analytics), mirroring what ALLOWED_PREFIXES
already permits for direct tools; read-only discovery (search/describe/list/
status/ui-messages) passes through. A call with no server is unverifiable, so
it's denied with a message telling the agent to set it -- which the model then
does on its own. Verified live in the GxIT: the agent retrieved 42 histories
and ran an upload_file_from_url job that landed a dataset, both through the
scoped proxy.
bgruening's Docker refactor compiles web/server.ts to web/build/server.js and runs that (CMD node web/build/server.js), which shifts __dirname one level deeper than the dev layout. Every __dirname-relative path in server.ts was silently thrown off: the brain binary (../bin/loom.js), the remote lockdown gate (extensions/web-mode-gate.ts), and the built renderer (dist) all resolved into web/build/ instead of web/. The gate one is security-relevant -- a missing --extension means the remote lockdown may not load. Anchor those paths at a WEB_ROOT that points at the web dir whether we're running from web/ (dev) or web/build/ (container). Also point the GxIT tool command at the compiled node web/build/server.js instead of tsx web/server.ts -- tsx is a devDependency the image prunes and server.ts isn't copied to the runner, so the old command couldn't launch.
The image never built end to end, and it turns out there were four separate reasons rather than the one we thought. Working outward: the vite build couldn't resolve marked from the renderer, because vite roots at app/src/renderer and marked is declared in the root package.json -- node walks up to /app/node_modules looking for it, which the builder stage never installed. That one hid everything behind it, so the builder now installs root deps before building. Next up was the tsc step, which is what we'd originally pinned this on: web pins TypeScript 6, and TS6 refuses to compile named files while a tsconfig.json is sitting there (TS5112) unless you pass --ignoreConfig. And even with that added it only got us to the next problem -- tsc emits flat into web/build/, so the emitted server.js kept its `../shared/brain-env.js` import, which resolves to web/shared/ at runtime rather than the /app/shared the runner ships. Module load would have died at boot for both the CMD and the GxIT command. So rather than paper over that with another copy of shared/, esbuild now bundles server.ts: every relative import gets inlined and bare packages stay external for the runner's npm ci to supply. That kills the whole class of path fragility, and WEB_ROOT still lands on /app/web since the output is still web/build/server.js. Last one, in the runner: npm ci --omit=dev still runs the root prepare script, which shells out to husky -- a devDep it was just told not to install -- and dies with "husky: not found". Dropped the script before installing; git hooks don't mean much in an image with no .git. Verified by hand: docker build succeeds, and the image boots offline (--network none) with no module-resolution errors.
The image pre-warmed galaxy-mcp>=1.8.0 into its uv cache while bin/loom.js wrote >=1.9.0 into mcp.json. That's less broken than it looks -- uv resolves >=1.8.0 to the newest release, which is 1.9.0 today, so the cache happened to satisfy the runtime spec and offline launch worked. But it works by luck: the two specs are independent, and any bump on either side (or a day when the newest release predates the runtime floor) quietly sends uv back to PyPI at job-launch, which is the one thing baking the cache is supposed to prevent. That failure would show up as a slow or dead GxIT on a restricted-network deploy, nowhere near this line. So there's now one constant, GALAXY_MCP_SPEC in shared/galaxy-mcp-spec.js, that bin/loom.js writes into mcp.json, and the Dockerfile takes the same string as a build arg. Nothing in CI builds the image, so a test asserts the two sites agree and that neither has drifted back to a hardcoded literal. Verified in the built image: uvx "galaxy-mcp>=1.9.0" --help resolves from the baked cache with --network none.
The remote key gate only recognized ${PROVIDER}_API_KEY names, so a container set
up the way gxit/README.md describes -- baseUrl in config.json, key injected as
LOOM_ACTIVE_LLM_API_KEY -- looked keyless. The connection handler then declined
to spawn the brain and the user got the BYO-key overlay despite the operator
having configured everything correctly. The brain resolves custom keys from
LOOM_ACTIVE_LLM_API_KEY (resolveActiveLlmApiKey), so that var now counts as a key
being present, regardless of the provider label: the server defaults
activeProvider() to "anthropic" when LOOM_LLM_PROVIDER is unset, while the real
provider name only exists in a config.json remote mode never reads.
The BYO path had the mirror-image problem. providerKeyVar() returns null for
anything not built in -- including "openai-compatible", which the overlay itself
can send -- and the handler injected nothing, reported ok:true and hid the
overlay anyway, leaving an unauthenticated brain and no way back to the prompt.
Keys now route the way Orbit desktop routes them (agent.ts): built-in providers
to their own var, everything else to LOOM_ACTIVE_LLM_API_KEY, so a supplied key
always lands somewhere. The renderer also stops ignoring the response and keeps
the overlay up on a rejection.
Restart and reset-session fired SIGTERM, nulled the process and immediately spawned a replacement. The brain's shutdown hook is what flushes the notebook to its Galaxy Page, so the outgoing brain's final push could still be in flight when the incoming one resumed and pushed: either the new brain resumes a page the old one hasn't finished writing, or the two pushes land last-writer-wins. Both handlers now await stopLoomGracefully first, the same drain the signal path already used. Pulled the grace-window resolution out into shutdown-grace.ts while I was in there, since three call sites want it now. It keeps the NaN guard honest with tests -- setTimeout(NaN) fires immediately, which would turn the SIGKILL backstop into an instant kill and drop the very flush the window exists to protect.
In GxIT mode the tool XML sets LOOM_WEB_ALLOW_INSECURE=1 and no token, since a Galaxy-issued entry-point URL can't carry one -- gx-it-proxy is the trust boundary. But authorizeWsUpgrade skipped the origin check entirely when the Origin header was absent, and with no token to check either, that left nothing at all between the control socket and any process that can reach port 3000. On the default Docker bridge that's every other container on the host, and the socket drives a live agent holding the user's scoped Galaxy key. Browsers always send Origin on a WebSocket upgrade, so requiring one costs the real client nothing: this only narrows the accepted set, and every upgrade that passes the existing origin match still passes. The bit that was doing no work was the origin-less path, which is exactly the attacker. Left it off for loopback dev and token deployments so wscat/curl workflows keep working. Worth noting for review: this can't regress the gx-it-proxy path relative to today. If the proxy rewrote Host such that Origin no longer matched, the existing same-origin check would already be rejecting browsers and the GxIT would be broken now -- so any browser connection that works today is unaffected. I haven't been able to put this in front of a real Galaxy instance, so that reasoning is the argument, not a live test.
If Galaxy was briefly unreachable when init() ran, the history lookup came back empty, init returned early, and sync was off for the rest of the session -- no retry, no reinit, while the UI happily kept showing Galaxy as connected. The notebook then only ever existed under the container's /tmp, which dies with the job. Silent and permanent, from a blip. Arming (resolving the history, adopting any prior page) is now separate from init, and the push path retries it, so the first change after Galaxy comes back recovers sync. Attempts are capped so a genuinely dead Galaxy doesn't mean a round trip on every change event forever. The init-time calls also get a timeout -- init runs inside session_start, so a wedged Galaxy was hanging the session's whole startup rather than just costing us sync. Two things that had to move with it. The baseline body is now only seeded from disk after an actual resume: seeding it unconditionally meant a recovered session compared the notebook against itself and flushed nothing, losing exactly the work the recovery was for. And since the baseline starts empty, pushes of an empty body are skipped, so an untouched notebook doesn't publish a blank page. Push failures are still stderr-only, and the shell has no way to show sync status -- that one needs a UI surface and is worth its own issue rather than smuggling it in here.
Two bits of the guide were describing a different branch than the one it's on. It told operators the user gets prompted for their own key "once BYO-key (Plan 2) lands" -- BYO-key lands in this same PR, so that's stale on arrival; said what actually happens instead, including that the key stays in server memory. The trust model also leaned on "the container is not otherwise reachable", which is a property of the deployment and not of this container: on a default Docker bridge every neighbour can hit port 3000, and that socket drives an agent holding the user's scoped key. Wrote down the Origin requirement that now backstops it, and pointed operators at LOOM_WEB_TOKEN for deployments where the port is really shared.
Codex caught a real one in the previous key-gate commit. providerKeyVar built the var name by uppercasing the provider, so google resolved to GOOGLE_API_KEY -- but Loom's var is GEMINI_API_KEY. A GxIT launched with LOOM_LLM_PROVIDER=google and a perfectly good admin-injected GEMINI_API_KEY was told it had no key and shown the BYO overlay; hand it that same key through the overlay and it went into LOOM_ACTIVE_LLM_API_KEY, which built-in google never reads, so the brain came up unauthenticated. The map now lives next to PROVIDER_API_KEY_NAMES in shared/brain-env.js, and that set is derived from it so the two can't drift. The second one is mine from the same commit: I let a non-empty LOOM_ACTIVE_LLM_API_KEY vouch for any provider, on the theory that the server couldn't know the container's real provider. It can -- it just wasn't looking. activeProvider() now falls back to config.json's llm.active (what the brain actually reads) instead of defaulting to "anthropic" behind its back, and "is this custom" comes from that provider's baseUrl, the same discriminator everything else uses. So a stale LOOM_ACTIVE_LLM_API_KEY no longer suppresses the BYO prompt for a built-in provider that can't use it, and a configured custom container still starts without one. That also gives the BYO handler a real answer to a question it was guessing at: if a key has nowhere to go -- "openai-compatible" with no endpoint baked into the image, say, which the overlay will happily offer -- it says so and the prompt stays up, rather than reporting success and handing back an agent that can't authenticate.
Fallout from the drain I added a few commits back, both found by Codex. Waiting for the old brain to exit opened a window that didn't exist when restart was synchronous, and two things walked into it. Prompts kept going to the dying brain. loomProcess still points at it during the drain and its stdin is still writable, so a prompt sent right after hitting restart -- or after a /resume, which the renderer fires and forgets -- was accepted, acknowledged, and then died with the process, leaving the UI waiting on a turn nobody was running. Messages that arrive mid-drain now queue and go to the replacement once it's up. And nothing serialized the swaps themselves. Two of them could await the same dying process and both spawn: reset's continuation would bring up a fresh-session brain, then restart's would SIGKILL it through the non-graceful path and spawn a normal one -- the reset the user asked for, silently undone. Swaps now run one at a time on a chain.
Five findings against my page-sync commit, all fair, all in the same code path. The retry budget was the worst of them: a hard cap of five attempts sounds conservative until you notice attempts are driven by change events, so five debounced edits during one short outage exhaust it and sync is off for good -- the exact silent data loss the retry existed to prevent. It's a cooldown now, not a budget: retry no more than once per interval, forever, and flush() ignores the cooldown because shutdown is the last chance to save anything. Committing historyId before page discovery meant one failed listing left the engine armed but blind to a page that already existed -- the next push would take the create path and duplicate it, and discovery never re-ran. Nothing commits now until discovery actually succeeds, and a failed listing or resume leaves the engine unarmed to try the whole adoption again. Refusing to push beats adopting a page we couldn't read or growing a second one for the same history. The resume call escaped the timeout I added to its neighbours, so a page GET that accepts the connection and never answers could still hang session_start -- the symptom the timeout was for. Wrapped. Skipping every empty body was too blunt: a user who deliberately clears the notebook could never sync that, and Galaxy would restore the stale content on the next launch. Only refuse to CREATE a page from an empty body; clearing one we have a baseline for is a real edit. And an arm() already in flight when the engine is disposed used to land anyway. resume() resolves the notebook path globally, so a stale engine could overwrite the *next* session's notebook with its own page. It bails after each await now.
Codex pointed out the ARG I added undercut the test that was supposed to backstop it: --build-arg GALAXY_MCP_SPEC=galaxy-mcp==1.8.0 bakes a non-matching spec into the cache while the test happily passes against the ARG's default. A guarantee with an override flag isn't one. It's a literal now, and the test asserts the install line matches the shared constant and that it's the only galaxy-mcp install in the image.
dannon
force-pushed
the
feat/orbit-gxit-page-sync
branch
from
July 15, 2026 20:29
70d7350 to
9e9e7e1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Runs the existing
LOOM_MODE=remoteOrbit web shell as a per-user agentic analysis workbench, from one image, two ways: standalone (docker run) and as a Galaxy Interactive Tool -- Galaxy launches a per-user container, injects the server URL + a scoped API key, and proxies the user in.The point is that durable artifacts live in Galaxy: tools/workflows run as Galaxy jobs (outputs are history datasets), and the analysis notebook persists as a per-history Galaxy Page that resumes on relaunch. Meant as a proving ground for ideas that can fold back into Galaxy's own assistant, not a replacement.
What's in here
Notebook -> Galaxy Page persistence (
extensions/loom/galaxy-page-sync.ts)LOOM_GALAXY_PAGE_SYNC=auto), brain-side (pi's rpc dispatch is closed, so the shell just sets the env at spawn and drains the brain on shutdown)orbit-<historyId>), current history viamost_recently_usedGET /pages/{id})session-lifecycle.ts; enabled + SIGTERM-drained inweb/server.tsGalaxy Interactive Tool wrapper (
gxit/) -- tool XML (scopedinject="api_key"+$__galaxy_url__,requires_domainentry point) + a deploy guide.Container fixes (
Dockerfile)web/auth.ts+web/rpc-guard.tsin the runner -- theLOOM_MODE=remoteimage didn't actually boot without them (ERR_MODULE_NOT_FOUND)TMPDIR=/tmp(Galaxy injects the host's TMPDIR, whichtsxcan't use)uv+ pre-warmgalaxy-mcpso the Galaxy tool/workflow execution surface loads in-container (node:slimhas no Python/uvx)Remote-mode tool gate (
web/extensions/web-mode-gate.ts) -- on a fresh container thegalaxy_*MCP tools don't register as direct tools (needs a warm per-server cache, which the per-user key invalidates each launch), so the only path is themcpproxy. Allow it, scoped to the curated servers (galaxy / brc-analytics); discovery passes, an unscoped call is denied with a message the agent self-corrects on.Bring-your-own LLM key (
web/llm-credentials.ts,web/server.ts, web shim + renderer) -- when the remote shell starts with no provider key in its env, the renderer prompts for a provider + key, the server holds it in memory for the session (never persisted, logged, or echoed -- only ahasApiKeyboolean crosses to the renderer) and spawns the brain with it. An admin-injected env key still auto-spawns exactly as before. So the workbench can spread without an admin-baked key. (Standard providers; custom/openai-compatibleBYO is a follow-up.)Testing
New unit tests for the page-sync engine, the gate's proxy scoping, the history helper, and the LLM-credential helpers; full suite green, typechecks clean. Verified live as a real Galaxy Interactive Tool (Galaxy 26.1, interactive tools + gx-it-proxy + Docker): the notebook persisted to a per-history Page, and the agent -- running entirely in the container -- invoked Galaxy MCP through the scoped proxy and ran an
upload_file_from_urljob that landed a dataset. The BYO-key flow was verified at the WebSocket-protocol level: no env key -> the server reportshasApiKey:falseand holds off spawning until the key is provided; an admin env key ->hasApiKey:true+ auto-spawn; the key never reaches the logs orconfig.json.Notes
LOOM_MODE=remoteshell; branch is based on an oldermainand will want a rebase. Follow-ups: wire BYO for custom/openai-compatibleproviders (baseUrl + key), and a browser-level eyeball of the BYO overlay.