Skip to content

Company-wide agent comms: central hub server and web console #37

Description

@Mearman

Every agent in the company should be able to see, and talk to, every other agent
in the company — without anyone hand-wiring a mesh per team, per office, or per
laptop. And a human should be able to open a browser and see the whole thing:
who is working on what, in which room, right now.

Today the mesh stops at localhost. This issue is the epic for taking it
company-wide.

Where we actually are

Worth stating precisely, because parts of this are further along than they look
and one part is further behind:

  • Local mesh works. First bridge to bind 19876 becomes coordinator, peers
    connect and then hold direct data connections with every other peer. State is
    in-memory and replicated to all peers.
  • Identity is Syncthing-style cert pinning. Agent ID = fingerprint of a
    locally generated, self-signed ECDSA P-256 cert, persisted per bridge slot.
    No CA, no PKI.
  • A WebSocket transport already exists (src/core/ws-transport.ts) and is a
    drop-in for TcpTransport — same wire protocol, MeshStore can't tell the
    difference. This is the thing that makes a remote hub cheap: wss:// needs no
    new protocol work.
  • Discovery exists and is opt-in: mDNS and Tailscale backends behind
    mesh_advertise / mesh_discover.
  • Federation is half-built. FederationManager can dial out
    (mesh_fed_connect), handshake, and forward presence, room joins and messages
    for rooms explicitly flagged federated. But:
    • handleInbound() has no production caller — it is referenced only from
      src/test/federation.integration.test.ts. Nothing in the shipped product
      accepts an inbound federation link, so today federation cannot actually be
      established between two machines.
    • There is no authentication or authorisation on the federation path.
      tlsConnect sets rejectUnauthorized: false and, unlike TlsTransport,
      FederationManager never checks the presented fingerprint against anything.
      fed_handshake is accepted from whoever sends it. This is fine while it is
      unreachable; it is a hole the moment we bind it to a routable interface.
  • The web UI is local-only. src/bridges/user/web/server.ts binds
    127.0.0.1 and has no auth of any kind. There is already a project tree, a
    room browser, and browser peers over WS — so the UI groundwork for the
    console exists; the server it talks to is a single laptop's mesh.

So: the transport is ready, the UI shell is ready, and the two missing pieces are
an authenticated node that legitimately holds cross-machine state, and a story
for who is allowed to see what.

Two architectures, and they are not the same choice

The original issue listed "central server" and "cascade's announce server" as two
routes to the same goal. They aren't — they buy different things, and only one of
them gets us the web console.

Announce / rendezvous (the cascade
model).
A stateless announce server holds only soft, expiring, signed
candidate sets (cascade: a Cloudflare Worker with KV soft state, signed
candidates, HMAC write auth), plus a relay for NAT traversal and a rendezvous
broker for live pairing. Peers find each other through it and then talk
directly. No message content ever touches the server.

  • Keeps the current trust model and privacy posture intact.
  • Scales beautifully — the server does almost nothing.
  • Needs NAT traversal, hole punching, and a relay fallback for the cases that
    can't punch. That is a lot of machinery.
  • Gives us no observability. A web console can't "peek into rooms" if the
    server never sees a room. The console would have to become a full peer that
    joins everything, which is a worse version of the hub.

Hub / relay (what this issue actually asks for). One deployed node is a
first-class mesh participant. Every session dials out to it, it holds
company-scoped state, and the web console is a view over that node.

  • Every session dials outbound to one address, so remote work, home networks,
    and NAT all stop mattering. No hole punching.
  • One place to enrol identity, enforce authorisation, and keep history — which
    is exactly what a company-wide console needs.
  • Central point of failure and a central pile of sensitive data. Both need
    answering rather than hand-waving.

Recommendation: build the hub. It is the only one that delivers the console,
it reuses WebSocketTransport and the existing fed_* protocol, and it removes
the NAT problem instead of solving it. Keep rendezvous as a later optimisation
for direct peer links between two sessions that discover each other via the hub
(P5 below) — the hub can hand out candidates without becoming a relay for the
bulk traffic.

Design sketch

A new agent-comms hub entrypoint: a long-lived process that runs a MeshStore
with no local agents of its own and an authenticated inbound listener.

  • Transport: wss://comms.exadev.io, reusing WebSocketTransport. TLS is
    terminated normally (real cert, not pinned), and agent identity is carried
    above it, so no changes to the wire protocol beyond version negotiation.
  • Sessions connect by config, not by command. AGENT_COMMS_HUB=wss://… in
    the plugin/MCP config, or a hub block in ~/.agent-comms/config.json. A
    bridge with a hub configured joins its local mesh and the hub. The local
    coordinator keeps doing what it does; the hub is an additional link, so
    same-laptop comms stay on localhost and never round-trip.
  • The hub is a coordinator-of-coordinators, not a router for everything.
    Only the existing federated room flag crosses the link, and it stays
    opt-in. A room is local until someone marks it federated.
  • Scoping. Rooms and the agent directory get an org/team dimension so
    general on one laptop isn't general company-wide. Suggest namespacing:
    exadev/eng/code-review.
  • Fan-out, not full replication. See "scale" below — this is the design
    constraint most likely to bite.

Problems that need deciding, with proposed defaults

  1. Identity and enrolment. Cert pinning doesn't scale to N employees × M
    machines × K sessions; nobody is going to approve fingerprints by hand.
    Proposal: Google Workspace OIDC device-code flow on first run. The hub
    issues a short-lived token bound to the bridge's existing cert
    fingerprint
    , so the on-disk key stays the agent's identity and SSO only
    authorises it. Agent ID stays stable across restarts as it does now.

  2. Authorisation and visibility. The existing visible / hidden / ghost
    levels and public / private / secret room types must be honoured
    company-wide, not bypassed by the console. Proposal: the console sees
    exactly what a member agent would see, plus an explicit org-admin role for
    the directory view. No god mode by default.

  3. Privacy and consent — the one to get right before writing code. "Peek
    into any session in the company" is, read uncharitably, employee monitoring.
    AgentIdentity already carries cwd (project paths) and pid, and room
    messages carry whatever agents said to each other about client work.
    Publishing all of that to a company server by default is a policy decision,
    not a config default. Proposal: publishing to the hub is opt-in per
    session
    (or per repo via project config); cwd is published as a repo name
    rather than an absolute path unless the session opts in to detail; a session
    attached to the hub says so in whoami and in the local UI so it is never a
    surprise. Worth a team conversation before P1 lands, and worth checking
    whether client contracts constrain what can leave a developer's machine.

  4. Persistence and retention. Mesh state is in-memory today; a console
    showing "what happened this morning" needs durable history. Proposal:
    durable store on the hub only (laptops stay in-memory), with an explicit
    retention window — 30 days as a starting point — and retention applied to
    room messages, not just presence.

  5. Topology and scale. Today every peer holds a data connection to every
    other peer and full replicated state. That is correct for four bridges on a
    laptop and wrong for ~40 people × several sessions each: we'd be pushing the
    whole company's state to every laptop. Proposal: the hub link is
    subscription-based — a session receives presence for the scopes it belongs to
    and messages for rooms it has joined, not the full state set. This is the
    biggest single piece of engineering in the epic.

  6. Wire protocol versioning. Blocked on Wire protocol has no version negotiation; mixed fleets break state sync during rolling upgrades #31. A company-wide hub is a mixed
    fleet by definition — people upgrade the plugin on different days — so
    version negotiation stops being nice-to-have and becomes a prerequisite.

  7. Close the federation auth hole first. rejectUnauthorized: false plus an
    unchecked fed_handshake plus an unwired handleInbound means the first
    thing that binds federation to a routable port inherits an unauthenticated
    listener. This must be fixed before anything is exposed, not alongside it.

  8. Hosting. Long-lived WebSocket connections and shared mutable state fit
    Cloudflare Durable Objects (one object per org scope) or a plain Node service
    behind a load balancer with sticky sessions. Cascade's announce server is a
    stateless Worker + KV, which suits candidates but not live sockets — so this
    is a genuinely different deployment shape, not a copy of it.

The web console

What makes this worth building rather than just a nice diagram:

  • Directory: every agent, grouped person → repo/project → session, with
    harness, status (active/idle/busy/offline), and uptime. "Who is working
    on what, right now."
  • Room browser: live message stream per room, with the existing read-receipt
    and presence events rendered as they arrive.
  • Join from the browser. Browser peers already work over WS, so a human can
    be a first-class participant in an agent room rather than a spectator. This is
    the part that turns observability into collaboration — noticing two sessions
    fighting the same bug and introducing them.
  • Search across rooms, scoped to what the viewer is allowed to see.
  • Read-only by default; participation behind the same authorisation as any agent.

Phasing

Each phase is independently shippable and probably its own sub-issue:

  • P0 — Close the hole. Fingerprint/allowlist verification on federation
    links, an authenticated inbound listener wired to handleInbound(), and
    version negotiation (Wire protocol has no version negotiation; mixed fleets break state sync during rolling upgrades #31).
  • P1 — agent-comms hub. Hub process over wss://, SSO enrolment bound to
    cert fingerprint, org room namespacing, opt-in publish, session config.
  • P2 — Durability. Persistent store and retention on the hub.
  • P3 — Console, read-only. SSO-gated org directory and room browser.
  • P4 — Console, participate. Join rooms and send from the browser; search
    and presence timeline.
  • P5 — Optional rendezvous. Signed candidates and hole punching so two
    sessions can talk directly, cascade-style, with the hub as the broker.

Open questions

  • Does the hub need message content at all, or only presence and metadata? A
    presence-only hub — who exists, which repo, which room, what status, no message
    bodies — is dramatically cheaper, far less sensitive, and might deliver most of
    the collaboration value. Worth deciding deliberately rather than defaulting to
    full relay.
  • Opt-in or opt-out for publishing a session to the hub?
  • Do DMs cross the hub, and are they ever visible to an org admin? (Proposed
    answer: they cross if both parties are on the hub, and they are never
    admin-readable.)
  • Retention window, and who can read history they weren't present for.
  • Self-hosted for ExaDev only, or is this a product surface for the plugin's
    external users too? That changes the identity design substantially.

Acceptance criteria for the epic

  • Two sessions on different machines, on different networks, discover each other
    and exchange messages with no manual mesh setup on either side.
  • No unauthenticated path into the mesh from a routable interface.
  • A session's participation in the company hub is explicit and visible to its
    operator.
  • A human can open the console, authenticate with their work account, and see
    live org-wide agents and rooms within their authorisation.

View original Slack conversation

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions