Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
9ad339d
feat(claude): add desktop manifest and packaging workflow
grabbou Oct 23, 2026
c0058c8
fix(claude): ask for ambiguous devices in chat
grabbou Oct 23, 2026
fb9f259
fix(claude): disable codex updater and polling
grabbou Oct 23, 2026
45535fe
feat(ui): support fullscreen in claude desktop
grabbou Oct 23, 2026
b41f98b
fix(ui): gate openai chat routing by host capability
grabbou Oct 23, 2026
a3ce405
feat(ui): support standard mcp context attachments
grabbou Oct 23, 2026
92babc3
fix(ui): use host theme tokens without openai styles
grabbou Oct 23, 2026
aa487ad
fix(ui): respect structured context capability
grabbou Oct 23, 2026
9f142b8
fix(ui): report usable inline workspace height
grabbou Oct 23, 2026
fcc0438
fix(logs): expose session handles to mcp models
grabbou Oct 23, 2026
9cfac3a
fix(ui): declare clipboard permission for mcp hosts
grabbou Oct 23, 2026
b50c45b
test(claude): verify packaged host compatibility
grabbou Oct 23, 2026
e243493
chore(claude): merge latest main and preserve host compatibility
grabbou Oct 23, 2026
5f7c3d7
docs(claude): document manual installation and local development
grabbou Oct 23, 2026
6495896
fix(claude): keep annotation context behind readable chat drafts
grabbou Oct 23, 2026
1c73cbe
feat(claude): manage annotation drafts in the attachment bar
grabbou Oct 23, 2026
3325582
fix(claude): direct annotation requests to widget context
grabbou Oct 23, 2026
ce7eef6
fix(claude): identify the widget for native context reads
grabbou Oct 23, 2026
4026b99
fix(claude): include selection locators in annotation drafts
grabbou Oct 23, 2026
6110d83
refactor: restructure the repository as an npm workspace
lauridskern Oct 23, 2026
322210d
fix(ui): apply standard MCP host style tokens and fonts in every host
grabbou Oct 23, 2026
e768b95
refactor(claude): replace host flag with capability checks
grabbou Oct 23, 2026
66cba97
refactor(ui): add a host bridge and share the device workspace
lauridskern Oct 23, 2026
8435b4e
feat(mac): add a standalone Mac app
lauridskern Oct 23, 2026
9835a86
fix(mac): link against the current macOS SDK
lauridskern Oct 23, 2026
0ebe40b
fix(mac): remove the title bar so the page reaches the top edge
lauridskern Oct 23, 2026
2646542
fix(mac): keep the top toolbar clear of the window controls
lauridskern Oct 23, 2026
9a483b2
fix(mac): keep the page beside the sidebar when it reopens
lauridskern Oct 23, 2026
3e4ad2d
fix(mac): set the minimum window size to 1280×720
lauridskern Oct 23, 2026
563731e
chore: merge main into the workspace restructure
lauridskern Oct 23, 2026
b3805d5
feat(claude): merge Claude Desktop support into the workspace layout
lauridskern Oct 23, 2026
d8d0de6
refactor: move host features and packaging into their apps
lauridskern Oct 23, 2026
62623f4
refactor(ui): name host context attachments instead of OpenAI extensions
lauridskern Oct 23, 2026
c0b1bc3
refactor(skills): share product skills across hosts
lauridskern Oct 23, 2026
97bb9c3
refactor(shared): keep pinned runtime constants inside shared
lauridskern Oct 23, 2026
1dd60c7
test: move tests beside the packages and apps they cover
lauridskern Oct 23, 2026
8b55146
chore(telemetry): keep Sentry project slugs in one place
lauridskern Oct 23, 2026
e68c8e2
chore(telemetry): rename Sentry projects to mobile-dev-*
lauridskern Oct 23, 2026
bae51f3
chore: point links at the renamed mobile-dev repository
lauridskern Oct 23, 2026
752c03a
refactor: share workspace tools, card bootstrap and host build steps
lauridskern Oct 23, 2026
b77e980
docs: update the README for every host and the workspace layout
lauridskern Oct 23, 2026
b2a187c
chore(release): bump version to 0.1.166
lauridskern Oct 23, 2026
0b383fd
fix(ui): size the panel's corners from the host's md radius
lauridskern Oct 23, 2026
9fe4fd8
fix(codex): leave out Back to chat in the Codex panel
lauridskern Oct 23, 2026
cb06a5a
fix(core): declare JSON Schema 2020-12 for tool schemas
lauridskern Oct 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
12 changes: 6 additions & 6 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,16 +43,16 @@ jobs:
cache: npm
cache-dependency-path: |
package-lock.json
runtimes/agent-device/package-lock.json
runtimes/serve-emu/package-lock.json
packages/runtimes/agent-device/package-lock.json
packages/runtimes/serve-emu/package-lock.json

- name: Validate release version and tag
id: release
env:
RELEASE_TAG: ${{ inputs.tag || github.ref_name }}
run: |
git show-ref --verify "refs/tags/$RELEASE_TAG"
node scripts/release-metadata.mjs
node tooling/release-metadata.mjs

- name: Check Sentry configuration
env:
Expand Down Expand Up @@ -88,7 +88,7 @@ jobs:
id: native
env:
GH_TOKEN: ${{ github.token }}
run: node scripts/release-native.mjs
run: node tooling/release-native.mjs

- name: Run tests
run: |
Expand Down Expand Up @@ -209,7 +209,7 @@ jobs:
run: |
gh api "repos/$GH_REPO/releases" --paginate \
--jq '.[] | select(.draft == false) | .tag_name' > "$RUNNER_TEMP/published-release-tags.txt"
node scripts/release-notes.mjs \
node tooling/release-notes.mjs \
"$RUNNER_TEMP/published-release-tags.txt" "$RUNNER_TEMP/release-notes.md"
gh release create "$RELEASE_TAG" \
"release/mobile-dev-$RELEASE_VERSION-darwin-arm64.zip" \
Expand Down Expand Up @@ -248,4 +248,4 @@ jobs:
run: |
mkdir "$RUNNER_TEMP/mobile-dev-marketplace"
unzip -q "release/mobile-dev-$RELEASE_VERSION-darwin-arm64.zip" -d "$RUNNER_TEMP/mobile-dev-marketplace"
node scripts/publish-release.mjs "$RUNNER_TEMP/mobile-dev-marketplace"
node apps/codex-plugin/tooling/publish-release.mjs "$RUNNER_TEMP/mobile-dev-marketplace"
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
node_modules/
.husky/_/
dist/
vendor/baguette/
packages/native/vendor/baguette/
release/
.DS_Store
*.log
.local-dev/
.sentry/
.env.sentry-build-plugin
native/ios-mirror/target/
packages/native/ios-mirror/target/
22 changes: 16 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Project instructions

## Repository layout

This is an npm workspace: shared code lives in `packages/` (`shared`, `core`, `ui`,
`mcp-app`, `skills`, `native`, `runtimes`) and each shipped product in `apps/` (`codex-plugin`,
`claude-desktop`, `mac`). Keep host chrome out of `packages/ui`: chrome every MCP Apps
host shares goes in `packages/mcp-app`, chrome for one host in its app. Drive host
differences from advertised capabilities rather than host names. See
[CONTRIBUTING.md](CONTRIBUTING.md#repository-layout).

## Commit messages

Use Conventional Commits for new source commits and pull request titles, for
Expand All @@ -26,16 +35,17 @@ new telemetry when existing coverage already answers the relevant questions.
monitored device/app's data. Browser frame intervals are callback pacing, and
device input round trips end at acknowledgement; neither is device rendered FPS
or touch-to-photon latency.
- Reuse `src/ui/telemetry.ts`: `recordUiTiming`, `setUiGauge`, `countUiEvent`,
- Reuse `packages/ui/src/telemetry.ts`: `recordUiTiming`, `setUiGauge`, `countUiEvent`,
`setUiSurface`, `setUiTelemetryContext`, and `markUiSurfaceReady`. Use
`captureUiError` and `src/server/telemetry.ts`'s `captureServerError` for relevant
`captureUiError` and `packages/core/src/telemetry.ts`'s `captureServerError` for relevant
handled failures. Keep SDK initialization centralized in the existing UI and
server telemetry modules, with shared sampling and scrubbing in
`src/shared/telemetry.ts`.
- Native helpers share `native/telemetry/telemetry.c` and its Rust wrapper. Reuse
`packages/shared/src/telemetry.ts`.
- Native helpers share `packages/native/telemetry/telemetry.c` and its Rust wrapper. Reuse
their bounded timing windows and resource sampler. Rebuild affected helpers
after changing shared telemetry, retain matching symbols in `.sentry/native`,
and upload them to `codex-mobile-dev-native` with the same plugin release.
and upload them to the native project in `tooling/sentry-projects.mjs` with the same
plugin release.
- Use sampled traces for meaningful operations spanning UI and server work. The
existing MCP wrapper propagates trace context; preserve it when changing the
bridge. Use bounded aggregate metrics for frequent frame, polling, processing,
Expand All @@ -62,7 +72,7 @@ new telemetry when existing coverage already answers the relevant questions.
Node feature change into native Sentry setup incidentally.
- Verify instrumentation on the changed execution path, including surface
attribution and observer/timer cleanup where relevant. Use the existing
`tests/telemetry.test.ts` and `tests/ui-telemetry.test.ts` for meaningful telemetry
`packages/core/test/telemetry.test.ts` and `packages/ui/test/ui-telemetry.test.ts` for meaningful telemetry
behavior checks. In the completion summary, state what coverage was added or
preserved, or why existing coverage is sufficient.

Expand Down
124 changes: 117 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,74 @@
This guide covers building the plugin, installing it locally, and working on its UI
and native helpers. To use the prebuilt plugin, see the [README](README.md#install).

## Repository layout

The repository is an npm workspace. Host-independent code lives in `packages/`; each
product that ships it lives in `apps/`. Adding a host means adding an app; the shared
packages know no host by name.

| Path | Contents |
| --- | --- |
| `packages/shared` (`@mobile-dev/shared`) | Protocol types and pure logic used by the server and every UI host. |
| `packages/core` (`@mobile-dev/core`) | The MCP server, device runtimes and tool handlers. A host adds to it with `ServerFeature`s passed to `runStdioServer`. |
| `packages/ui` (`@mobile-dev/ui`) | Host-independent React views, panel controllers, theme and UI telemetry. Views reach the host only through `HostBridge`. |
| `packages/mcp-app` (`@mobile-dev/mcp-app`) | The MCP Apps panel every MCP Apps host loads. Host differences follow advertised capabilities; a host adds chrome through `startMcpApp`. |
| `packages/native` | Native helper sources; `packages/native/vendor` holds their verified prebuilt binaries. |
| `packages/runtimes` | Pinned Node runtimes bundled into the server (agent-device, serve-emu). |
| `packages/skills` | Agent skills for the shared tools; each host's package step includes the ones its host supports. |
| `apps/codex-plugin` | Codex: manifest, launcher, assets, and its own server (`server/`, the self-updater) and panel (`ui/`, the update banner). |
| `apps/claude-desktop` | Claude Desktop: the MCPB manifest. It runs the shared server and panel unchanged. |
| `apps/mac` | The Mac app: a SwiftUI shell (`Sources/`), the page it hosts (`web/`) and its server (`server/`). |
| `packages/test-support` | Fixtures shared by tests across packages and apps. |
| `tooling` | Shared build, native, release and device smoke-test scripts. Run them from the repository root. |

Each app's `tooling/` holds that host's packaging and smoke tests.

Tests live beside the code they cover, in each package's or app's `test/` folder;
`tooling/test` covers the release and build scripts. Fixtures several packages share
are in `packages/test-support`. `npm test` runs them all from the repository root,
which test paths such as `packages/ui/src/...` are relative to.

Import across packages with the workspace name and an explicit extension, for example
`@mobile-dev/shared/logs.ts`. `packages/ui` never imports from `packages/mcp-app` or an
app. Chrome every MCP Apps host shares belongs in `packages/mcp-app`; chrome for one
host, such as the Mac sidebar or Codex's update banner, belongs in that host's app.

`npm run build` writes the host-neutral runtime to `dist/`: the shared server, its
services and natives, and the MCP Apps panel. Each host's package step copies that
runtime and builds the host's own pieces into the copy: Codex its server and panel,
the Mac app its server and page. No host packages through another, and the shared
build knows no host. The release version comes
from the root `package.json`; each package step writes it into its host's manifest.
The published `release/latest` layout is unchanged.

## Mac app

The Mac app runs the same server and shared UI as the Codex plugin. A SwiftUI
window provides the native sidebar (tools and visible device platforms); a web
view renders `apps/mac/web`, the shared views without the Codex toolbar.

The app starts the bundled server over stdio exactly as Codex does and relays
JSON-RPC lines between it and the page. Before the page connects, the app reads
`ui://mobile-dev/mac.html` from the server, so the page gets the server's telemetry
metadata. The page implements `HostBridge` (`packages/ui/src/host-bridge.ts`) with
an MCP client and offers no chat capabilities, so chat-only actions stay hidden.

```sh
npm run build # shared runtime in dist/
npm run build:mac # release/mac/Mobile Dev.app, signed ad hoc
npm run test:mac # launches the app, waits for the page to connect and draw, quits
open "release/mac/Mobile Dev.app"
```

`build:mac` copies the runtime from `dist/`, builds the Mac server and page into it, and
adds the Node executable running the build
(pass `--node <path>` for another self-contained build) into the app. Distribution
still needs a Developer ID signature, hardened-runtime entitlements for Node, and
notarization. `test:mac` uses the app's `MOBILE_DEV_SMOKE_SNAPSHOT` hook to save a
page snapshot because it cannot rely on Screen Recording permission; pass a path to
keep it. Restart the server from **View › Restart Server** (⇧⌘R).

## Development requirements

- An Apple Silicon Mac with Xcode 27 and Swift 6.4 or later selected as the active
Expand Down Expand Up @@ -63,13 +131,13 @@ changing native code or shared native telemetry; the build validates source and
binary hashes. For UI or server edits with unchanged native sources, run the build,
relevant checks, and packaging steps again.

`vendor:baguette` downloads the pinned official release and checks its SHA-256. It keeps the full resource bundle and rebuilds the same v0.2.1 source with Swift 6.4 or later because the official binary crashes in Swift task allocation on macOS 27. Preparing Baguette requires Xcode 27, Git, and network access to fetch its pinned source and Swift dependencies. `rebuild:baguette` repeats the source build. `vendor/baguette/release.json` records the source commit, compiler version, and rebuilt binary SHA-256.
`vendor:baguette` downloads the pinned official release and checks its SHA-256. It keeps the full resource bundle and rebuilds the same v0.2.1 source with Swift 6.4 or later because the official binary crashes in Swift task allocation on macOS 27. Preparing Baguette requires Xcode 27, Git, and network access to fetch its pinned source and Swift dependencies. `rebuild:baguette` repeats the source build. `packages/native/vendor/baguette/release.json` records the source commit, compiler version, and rebuilt binary SHA-256.

`vendor:agent-device` installs the exact npm packages in `runtimes/agent-device/package-lock.json` with package scripts disabled. `build` copies this full runtime into `dist/agent-device`, records its integrity and lockfile hash in `release.json`, and saves its version-matched help in the control skill. The installed plugin needs no npm install or runtime download. Xcode compiles the shipped Apple runner source on its first use.
`vendor:agent-device` installs the exact npm packages in `packages/runtimes/agent-device/package-lock.json` with package scripts disabled. `build` copies this full runtime into `dist/agent-device`, records its integrity and lockfile hash in `release.json`, and saves its version-matched help in the control skill. The installed plugin needs no npm install or runtime download. Xcode compiles the shipped Apple runner source on its first use.

`build` also bundles JavaScript and CSS with esbuild and copies Baguette into `dist/baguette`. It does not run TypeScript, lint, Biome, visual checks, or React Doctor.

`package` copies built files into `release/marketplace/plugins/mobile-dev` and creates the ZIP. It includes agent-device's runtime `node_modules`, licenses, and Apple runner source. The Codex compatibility manifest lives at `.codex-plugin/plugin.json` and declares `.mcp.json`. The MCP entry invokes `scripts/launch-mcp.sh` through `/bin/sh` and explicitly forwards `HOME` and the telemetry settings. The launcher directly executes Codex's bundled Node at `$HOME/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node`. `npm start` uses the same launcher. The source repo's existing build-codex-native-plugins skill stays outside that release package.
`package` copies built files into `release/marketplace/plugins/mobile-dev` and creates the ZIP. It includes agent-device's runtime `node_modules`, licenses, and Apple runner source. The Codex compatibility manifest lives at `apps/codex-plugin/.codex-plugin/plugin.json` and declares `.mcp.json`. The MCP entry invokes `apps/codex-plugin/scripts/launch-mcp.sh` (`scripts/launch-mcp.sh` inside the package) through `/bin/sh` and explicitly forwards `HOME` and the telemetry settings. The launcher directly executes Codex's bundled Node at `$HOME/.cache/codex-runtimes/codex-primary-runtime/dependencies/node/bin/node`. `npm start` uses the same launcher. The `build-codex-native-plugins` skill in `.agents/skills` helps agents working on this repository and is not shipped.

`test` checks MCP contracts, frame reads, input validation, and capture cleanup against a local fixture, plus runtime selection and argument forwarding with Node absent from PATH or another Node present. `test:package` copies the release package into a temporary directory and uses its actual MCP launch configuration with a system-only PATH. The smoke-test caller supplies an isolated home with the Codex runtime cache layout and its test Node executable; the MCP process has a system-only PATH. Run these checks with the Codex bundled Node executable to verify the packaged runtime. It checks native addon loading, sidebar/chat entrypoints, UI resources, and telemetry attribution, starts bundled Baguette, reads the real device list, and checks shutdown. It does not boot or change a simulator. Run it on an Apple Silicon Mac with Xcode.

Expand All @@ -93,6 +161,8 @@ not included in the plugin package.

## Install the local build

### Codex

From this project directory, after packaging:

```sh
Expand All @@ -104,9 +174,48 @@ Open a new chat after installing, then open Mobile Dev from the sidebar.

`npm run package` writes the local ZIP to `release/mobile-dev-<version>-darwin-arm64.zip`. Install through the local marketplace above. The New Plugin archive dialog uploads to the workspace plugin service; it is a separate install route. This package has not gone through public directory review or publication.

### Claude Desktop local development

Complete the first-time runtime and native-helper setup under [Build and package](#build-and-package),
then run from this checkout with Node.js 22.19 or later:

```sh
npm run build
npm run claude:config
```

1. Open Claude Desktop **Settings > Developer > Edit Config**. On macOS, this is
`~/Library/Application Support/Claude/claude_desktop_config.json`.
2. Merge the `mobile-dev` entry from `.local-dev/claude-desktop-config.json` into
the existing `mcpServers` object, preserving other servers and settings. The
snippet uses absolute paths to this checkout and the Node executable running
the generator.
3. Fully quit and reopen Claude, start a new conversation, and ask **“Open Mobile Dev.”**
Approve the connector when prompted; use **Expand** when available.

After UI or server edits, run `npm run build` and fully restart Claude. This
workflow uses the checkout directly and needs no repackaging. Regenerate the
config if the checkout or Node path changes. For connection failures, check
**Settings > Developer** for the server status and logs.

To create and check a bundle for manual installation instead:

```sh
npm run package:claude
npm run test:claude
```

This writes `release/mobile-dev-<version>-claude-desktop-darwin-arm64.mcpb`.
Follow the [manual installation steps](README.md#claude-desktop-manual-installation).
Remove the `mobile-dev` development entry before installing the bundle to avoid
duplicate servers. Rebuild, repackage, and reinstall to update a bundled copy.

Local builds retain the `development` telemetry environment. Public releases use
`npm run build:release` and `npm run package:claude -- --release`.

## Live React UI development

Run `npm run dev` after installing the plugin. It watches the UI sources and
In Codex, run `npm run dev` after installing the plugin. It watches the UI sources and
rebuilds a self-contained HTML bundle. Open panels read updates through MCP,
then reload and reconnect their streams. Edits reset UI state.

Expand All @@ -124,7 +233,7 @@ last working UI. Set `MOBILE_DEV_PLUGIN_ROOT` for a nonstandard plugin install.

## UI conventions

The panel uses React and shadcn/ui with preset `b0`, Nova controls, neutral colors, Inter, and Lucide icons. The build bundles the SVGs and font files into the HTML; the panel needs no external asset requests. Use the preset components in `src/ui/components/ui` for controls, forms, notices, and empty states. Compose the views with Tailwind utilities. `src/ui/style.css` only covers device frames, Codex layout, and base rules; preset tokens live in `src/ui/theme.css`. The log list uses the [Legend List React DOM entrypoint](https://www.legendapp.com/open-source/list/v3/react/getting-started/).
The panel uses React and shadcn/ui with preset `b0`, Nova controls, neutral colors, Inter, and Lucide icons. The build bundles the SVGs and font files into the HTML; the panel needs no external asset requests. Use the preset components in `packages/ui/src/components/ui` for controls, forms, notices, and empty states. Compose the views with Tailwind utilities. `packages/ui/src/style.css` only covers device frames, Codex layout, and base rules; preset tokens live in `packages/ui/src/theme.css`. The log list uses the [Legend List React DOM entrypoint](https://www.legendapp.com/open-source/list/v3/react/getting-started/).

Panels, toolbars, settings areas, and footers use transparent backgrounds so they blend with the host. Add a panel background only when the user asks to highlight that area.

Expand All @@ -136,8 +245,9 @@ helpers. Local builds use the `development` environment; public packages require
`npm run build:release` and `npm run package:release`. Keep credentials, source maps,
and native symbols outside the package.

Bump the plugin version for plugin changes and keep `.codex-plugin/plugin.json`, `package.json`,
both root versions in `package-lock.json`, and `src/shared/version.ts` in sync.
Bump the version in the root `package.json` for product changes and keep both root
versions in `package-lock.json` and `packages/shared/src/version.ts` in sync. Host
manifests carry no version; each package step writes it.
See [GitHub releases](docs/releases.md) for tagging, CI packaging, symbol uploads,
and marketplace publishing.

Expand Down
Loading
Loading