improvement(desktop): Electron E2E for desktop tool lifetimes against a live local Sim by waleedlatif1 · Pull Request #8697 · simstudioai/sim

waleedlatif1 · 2026-10-06T22:15:34Z

Summary

New Electron E2E spec runs the real desktop app against a live local Sim (Postgres, Redis, realtime). A recording proxy sits at the app origin and a scripted stand-in worker sits at SIM_AGENT_API_URL.
Covers the ungated desktop fixes:
- a browser call issued after switching chats fails as not started after the 15 s pickup grace (fix(mothership): fail unclaimed desktop calls fast and stop dropping them silently #8652)
- leaving the chat doesn't end a running read_local_file, and an import survives chat switches (fix(mothership): keep local file tools running when the chat view changes #8666)
- Stop from the reopened chat ends the running import, recorded exactly as cancelled with no competing report (fix(mothership): keep local file tools running when the chat view changes #8666, fix(desktop): leave a stopped native file tool's outcome to Stop #8732)
- signing out cancels a running desktop tool at sign-out itself, before the login reload (fix(mothership): end desktop tools at sign-out and settle held reports as final #8673)
- unapproved reads and terminal runs, and a claim that reaches Sim after Stop, are refused (fix(mothership): refuse desktop claims for unapproved or stopped calls #8643)
Flag-off dormancy, checked layer by layer: the registration answer, device records, the turn's binding, the desktop routes the app calls, and the inbox doorbell.
Assertions are on observable outcomes: the result the model is resumed with, workspace files and folders, persisted call rows, the requests that reach Sim, and Redis MONITOR.
A named warm-up runs the tests' own flows once and waits until every request the app made is answered, so no route the tests reach compiles mid-test. A marker request lets the dev log show anything compiled after it.
New desktop-live-e2e CI job: dev app, realtime, Postgres and Redis, Xvfb, --retries=0, per-run E2E_APP tags. On pull requests it's skipped only when every change is clearly unrelated (docs, other apps, published content). If the diff can't be worked out, it runs.

Type of Change

Tests

Testing

Base: staging with fix(desktop): leave a stopped native file tool's outcome to Stop #8732.
8/8 green in the desktop-live-e2e CI job.
Cold starts: 28/28 in each of two independent runs (4 parallel slices × 7 iterations, each pinned to 8 cores in its own network namespace, retries: 0, no resends). After rebasing onto staging with fix(desktop): leave a stopped native file tool's outcome to Stop #8732, a 1 × 7 cold check (head merged with staging f83d977d4) passed 7/7.
Each fix's revert turns its test red, and each dormancy layer's mutation turns only its own assertion red.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-10-06T22:15:40Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Oct 7, 2026 9:51am UTC

cubic-dev-ai

All reported issues were addressed across 5 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Turn on auto-fix | Re-trigger cubic}

greptile-apps · 2026-10-06T22:23:32Z

Confidence Score: 5/5

[High risk] Adds CI workflow and test infrastructure for desktop E2E testing.

The PR appears safe to merge; no new actionable issue was established.

What we checked:

Chat IDs cannot inject patterns: The checks use chat IDs generated by the fixture, not the supplied chat titles.
Original delayed claim is checked: The proxy delivers the held claim even after Electron abandons it. The test requires that original request to return 410 before replaying it.

Summary

Adds eight Electron E2E tests against a live local Sim, with a recording proxy and scripted agent.

Covers chat switches, Stop, sign-out, approval checks, and background-executor dormancy.
Adds a dedicated CI job with Postgres, Redis, realtime, Xvfb, and failure artifacts.
Previous helper, Redis acknowledgment, delayed-claim, and CI-filter findings are addressed. The earlier Electron-install finding was correctly disputed by waleedlatif1 and conceded by Greptile.
No new actionable issues found. This review inspected code; it did not run the suite.

Diagram

sequenceDiagram
  participant Electron
  participant Proxy as Recording proxy
  participant Sim as Local Sim
  participant Agent as Scripted agent
  participant DB as Postgres and Redis
  Electron->>Proxy: Send chat and native-tool requests
  Proxy->>Sim: Forward or hold requests
  Sim->>Agent: Start or resume turn
  Agent-->>Sim: Issue tools or finish turn
  Sim->>DB: Save calls and outcomes
  Sim-->>Proxy: Return authorization and results
  Proxy-->>Electron: Deliver responses

_{Reviews (8) · Last reviewed commit: "improvement(ci): run the live desktop su..." · Reviewed by Greptile}

waleedlatif1 · 2026-10-07T01:27:01Z

@greptileai review

waleedlatif1 · 2026-10-07T01:34:30Z

@greptileai review

waleedlatif1 · 2026-10-07T04:20:45Z

@greptileai review

waleedlatif1 · 2026-10-07T04:32:21Z

@greptileai review

waleedlatif1 · 2026-10-07T08:00:13Z

@greptileai review

waleedlatif1 · 2026-10-07T08:20:22Z

@greptileai review

… a live local Sim Runs the real Electron app against a local Sim (Postgres, Redis, realtime) with a recording proxy and a scripted stand-in worker, covering chat switches, Stop, sign-out, approval-gated and stopped claims, and the flag-off foreground round trip. The spec skips unless a live Sim is configured; the http-e2e job provides one.

…e desktop build on the workspace disk

…in the Linux E2E step

…nner memory

…rger runner

…tcomes instead of time - Compile every route, page and client chunk the tests use in a named warm-up step before any timed hold - Send retries only while no turn reached Sim; every held request and poll is bounded - Stop and sign-out tests wait for the import's own report and assert one exact terminal state - Clicks fail fast with the dev overlay's error instead of a blocked click

…ser dropped

…kground executor - The dormancy check accepts the app's own registrations, refused while signed out and answered as not enabled once signed in, and still forbids inbox, doorbell, claim, lease and completion - The app and realtime server run in their own sessions and stop through stop-session.sh

…ts go, and report the call's error on a wrong terminal state

…toRecord

… dev console notices, and keep failure diagnostics

…if Electron gave up on it, and require its 410

…ds honest, and check dormancy layer by layer - Warm-up runs the tests' own flows (read, import, a chat switch during a live turn, Stop, login) and waits until every request the app made has been answered - send() resends only when the UI did not take the message; a taken message whose turn never reaches Sim fails as a lost send - Stop and sign-out tests assert exact outcomes without holding the tool's report (the stopped tool reports nothing since #8732) - Dormancy checks registration, device records, binding, routes and the doorbell as separate soft assertions - CI: per-run E2E_APP tags, retries 0, and a path filter for pull requests

… app settles

… chat pages and auth routes

…early unrelated, and fail open - The change check skips only docs, the other apps and published content; any change elsewhere, and any failure to fetch or diff the base, runs the suite - settled() discounts only requests a hold is keeping from Sim right now - The warm-up's Stop turn waits for its stream to close instead of on a promise that never settles

waleedlatif1 · 2026-10-07T10:11:21Z

@greptileai review

waleedlatif1 requested a review from a team as a code owner October 6, 2026 22:15

vercel Bot deployed to Preview October 6, 2026 22:19 View deployment

cubic-dev-ai Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread .github/workflows/test-build.yml

Comment thread apps/desktop/e2e/fixtures/live-sim.ts Outdated

greptile-apps Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread .github/workflows/test-build.yml Outdated

Comment thread apps/desktop/e2e/fixtures/live-sim.ts Outdated

Comment thread apps/desktop/e2e/fixtures/live-sim.ts Outdated

Comment thread apps/desktop/e2e/fixtures/live-sim.ts

vercel Bot deployed to Preview October 6, 2026 22:37 View deployment

vercel Bot deployed to Preview October 6, 2026 22:55 View deployment

This comment has been minimized.

Sign in to view

vercel Bot deployed to Preview October 6, 2026 23:24 View deployment

vercel Bot deployed to Preview October 6, 2026 23:41 View deployment

vercel Bot temporarily deployed to Preview October 7, 2026 00:12 Inactive

This comment has been minimized.

Sign in to view

vercel Bot temporarily deployed to Preview October 7, 2026 00:27 Inactive

waleedlatif1 force-pushed the test/desktop-ungated-e2e branch from 349e0ba to e5717b0 Compare October 7, 2026 00:46

vercel Bot deployed to Preview October 7, 2026 00:49 View deployment

vercel Bot temporarily deployed to Preview October 7, 2026 01:10 Inactive

greptile-apps Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread apps/desktop/e2e/desktop-tools-live-sim.spec.ts Outdated

vercel Bot temporarily deployed to Preview October 7, 2026 01:34 Inactive

vercel Bot temporarily deployed to Preview October 7, 2026 02:14 Inactive

waleedlatif1 force-pushed the test/desktop-ungated-e2e branch from 74fa742 to 2163653 Compare October 7, 2026 04:03

vercel Bot deployed to Preview October 7, 2026 04:06 View deployment

greptile-apps Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread apps/desktop/e2e/desktop-tools-live-sim.spec.ts

vercel Bot temporarily deployed to Preview October 7, 2026 04:32 Inactive

waleedlatif1 mentioned this pull request Oct 7, 2026

fix(desktop): leave a stopped native file tool's outcome to Stop #8732

Merged

6 tasks done

waleedlatif1 force-pushed the test/desktop-ungated-e2e branch from ab8109f to 6074c04 Compare October 7, 2026 07:26

vercel Bot deployed to Preview October 7, 2026 07:29 View deployment

vercel Bot temporarily deployed to Preview October 7, 2026 07:44 Inactive

greptile-apps Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread .github/workflows/test-build.yml Outdated

Comment thread .github/workflows/test-build.yml Outdated

vercel Bot deployed to Preview October 7, 2026 08:10 View deployment

waleedlatif1 added 17 commits October 7, 2026 02:48

improvement(desktop): use shared helpers, confirm Redis MONITOR, stag…

5f452e2

…e desktop build on the workspace disk

improvement(desktop): bundle the desktop app without macOS prebuilds …

691096d

…in the Linux E2E step

improvement(desktop): report desktop E2E failures inline and trace ru…

b09c458

…nner memory

improvement(desktop): run the live desktop E2E in its own job on a la…

35c01b1

…rger runner

improvement(desktop): retype and resend a message the hydrating compo…

6e3a4ec

…ser dropped

improvement(desktop): release the held read at the moment its view le…

f890834

…ts go, and report the call's error on a wrong terminal state

improvement(desktop): read the chat body's desktop capabilities with …

a2099ff

…toRecord

improvement(desktop): hold the read's own report across Stop, dismiss…

93e866d

… dev console notices, and keep failure diagnostics

improvement(desktop): stop the live E2E apps by their E2E_APP tags

a23ab4e

improvement(desktop): deliver the claim held across Stop to Sim even …

ea86d97

…if Electron gave up on it, and require its 410

improvement(desktop): wait with the shared sleep helper while the dev…

da77efd

… app settles

improvement(ci): detect live desktop suite changes with git, covering…

aa313ff

… chat pages and auth routes

waleedlatif1 force-pushed the test/desktop-ungated-e2e branch from c1fccc0 to 5370c79 Compare October 7, 2026 09:49

vercel Bot deployed to Preview October 7, 2026 09:52 View deployment

waleedlatif1 merged commit af5501b into staging Oct 7, 2026
39 checks passed

waleedlatif1 deleted the test/desktop-ungated-e2e branch October 7, 2026 16:32

waleedlatif1 mentioned this pull request Oct 7, 2026

v0.9.15: mship improvements, memory improvements, nextjs bump, plane integration #8748

Open

Conversation

waleedlatif1 commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

waleedlatif1 commented Oct 7, 2026

Uh oh!

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

waleedlatif1 commented Oct 6, 2026 •

edited

Loading

vercel Bot commented Oct 6, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

greptile-apps Bot commented Oct 6, 2026 •

edited

Loading