A production-grade real-time collaboration backend — WebSockets, Yjs CRDT, awareness protocol, Redis pub/sub for horizontal scaling, and persistent document state.
Architecture · Quick Start · Protocol · Scaling · API Reference
Collab Engine is the backend infrastructure for building Google Docs-style collaborative applications. It handles the hardest parts of real-time collaboration so you can focus on your product.
The core challenge in collaborative editing is the concurrent edit problem: when two users edit the same document simultaneously on different machines, how do you merge their changes without losing either edit and without requiring a central lock? Collab Engine solves this using Yjs, a battle-tested CRDT (Conflict-free Replicated Data Type) library used in production by Notion, Linear, and hundreds of other applications.
| Feature | Description |
|---|---|
| CRDT Sync | Yjs-based conflict-free document merging — no operational transforms, no central lock |
| Awareness Protocol | Real-time cursor positions, selections, and user presence |
| Horizontal Scaling | Redis pub/sub distributes updates across multiple server instances |
| Document Persistence | Automatic periodic snapshots to Redis with 7-day TTL |
| JWT Authentication | Secure WebSocket connections with JWT verification |
| Reconnection Support | Late-joining clients receive full document state via sync protocol |
| Ping/Pong Heartbeat | Automatic stale connection detection and cleanup |
| Room Management | Dynamic room creation and destruction with configurable limits |
┌─────────────────────────────────────────────────────────────────────┐
│ Client Applications │
│ (Browser, Mobile, Desktop — any Yjs client) │
└──────────────────────┬──────────────────────────────────────────────┘
│ WebSocket (ws://)
▼
┌─────────────────────────────────────────────────────────────────────┐
│ Collab Engine (Fastify + WS) │
│ │
│ ┌─────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │
│ │ Auth │ │ Room Manager │ │ Room Sessions │ │
│ │ Middleware │→ │ │→ │ (one per active room) │ │
│ │ (JWT/WS) │ │ getOrCreate │ │ │ │
│ └─────────────┘ │ destroy │ │ ┌────────────────────┐ │ │
│ │ persist │ │ │ Connections │ │ │
│ └──────────────┘ │ │ (one per client) │ │ │
│ │ └────────────────────┘ │ │
│ └──────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ Yjs CRDT Layer │ │
│ │ CollabDocument → Y.Doc + Awareness + Sync Protocol │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────┬──────────────────────────────────────┘
│
┌────────────────┴────────────────┐
│ │
┌──────▼──────┐ ┌──────▼──────┐
│ Redis │ │ Redis │
│ Pub/Sub │ │ Document │
│ (scaling) │ │ Persistence │
└─────────────┘ └─────────────┘
Traditional collaborative editing uses Operational Transformation (OT) — the algorithm behind Google Docs. OT requires a central server to serialize and transform all operations, which creates a bottleneck and makes offline editing difficult.
Yjs uses a CRDT approach. Each operation is self-describing and carries enough information to be merged with any other operation in any order. The merge is mathematically guaranteed to produce the same result on all clients, regardless of network delays or reordering. This means:
- No central lock or serialization point
- Offline editing works natively — sync when reconnected
- Horizontal scaling is straightforward — any server can handle any client
- Merge conflicts are impossible by construction
- Node.js 22+
- Redis 7+
git clone https://github.com/josenevado/collab-engine
cd collab-engine
npm install
cp .env.example .env
npm run devThe server starts on ws://localhost:4000.
import * as Y from "yjs";
import { WebsocketProvider } from "y-websocket";
const ydoc = new Y.Doc();
const provider = new WebsocketProvider(
"ws://localhost:4000",
"my-room-id",
ydoc,
{
params: { token: "your-jwt-token" },
}
);
provider.on("status", ({ status }: { status: string }) => {
console.log("Connection status:", status);
});
// Collaborative text
const text = ydoc.getText("content");
text.insert(0, "Hello, world!");
// Collaborative map (for whiteboard objects, etc.)
const objects = ydoc.getMap("objects");
objects.set("rect-1", { x: 100, y: 200, width: 300, height: 150 });The sync protocol follows the Yjs sync specification:
Client connects
│
├─── Server sends: sync.step1 (server state vector)
│
├─── Client sends: sync.step1 (client state vector)
│ Server replies: sync.step2 (updates client is missing)
│
├─── Client sends: sync.step2 (updates server is missing)
│
└─── Both sides synchronized
│
├─── Client sends: update (on every local edit)
│ Server broadcasts to all other clients
│
├─── Client sends: awareness (cursor/selection changes)
│ Server broadcasts to all other clients
│
└─── Server sends: ping every 30s
Client replies: pong
No pong → connection closed
All messages are JSON-encoded with the following base structure:
{
type: "sync.step1" | "sync.step2" | "update" | "awareness" | "ping" | "pong" | "room.joined" | "room.left" | "error",
roomId: string,
timestamp: number,
// type-specific fields...
}Binary Yjs updates are serialized as number[] arrays within the JSON envelope for transport compatibility.
Collab Engine scales horizontally using Redis pub/sub. Each server instance publishes document updates to a per-room Redis channel. All other instances subscribed to that channel apply the update to their local Yjs document and broadcast to their local clients.
Server 1 (clients A, B) Server 2 (clients C, D)
│ │
Client A edits │
│ │
Server 1 applies update │
Server 1 broadcasts to B │
Server 1 publishes to Redis ──────────────►│
Server 2 receives from Redis
Server 2 applies update
Server 2 broadcasts to C, D
Because Yjs updates are CRDT-based, applying the same update multiple times is idempotent. There is no risk of double-application causing corruption.
To run multiple instances:
SERVER_ID=server-1 PORT=4000 npm start &
SERVER_ID=server-2 PORT=4001 npm start &
# Put a load balancer (nginx, Caddy) in front with sticky sessionsDocuments are persisted to Redis automatically:
- Periodic snapshots: Every 5 seconds (configurable via
PERSISTENCE_INTERVAL_MS) - On graceful shutdown: All active rooms are flushed before exit
- TTL: 7 days — rooms that have been empty for 7 days are automatically evicted
- Restoration: When a room is created, the server checks Redis for a persisted state and restores it before accepting connections
For long-term persistence, implement a PostgreSQL adapter that snapshots documents to a database on a longer interval (e.g., every 60 seconds or on room close).
Connect to a room:
ws://host/rooms/:roomId?token=<jwt>
The JWT payload must include:
{
"sub": "user-id",
"name": "User Name",
"email": "user@example.com"
}| Method | Path | Auth | Description |
|---|---|---|---|
GET |
/health |
None | Server health and active room count |
GET |
/rooms |
JWT | List all active rooms with stats |
GET |
/rooms/:roomId/stats |
JWT | Stats for a specific room |
PORT=4000
HOST=0.0.0.0
NODE_ENV=production
JWT_SECRET=your-secret-minimum-32-chars
REDIS_URL=redis://localhost:6379
LOG_LEVEL=info
MAX_CONNECTIONS_PER_ROOM=50
PERSISTENCE_INTERVAL_MS=5000
SERVER_ID=server-1- Set a strong
JWT_SECRET(minimum 32 characters, randomly generated) - Configure Redis with AOF persistence to survive restarts
- Use sticky sessions at the load balancer (or implement Redis-based session routing)
- Set
NODE_ENV=productionto disable development logging - Monitor
/healthwith your uptime service - Set up log aggregation for structured JSON logs
- Configure
MAX_CONNECTIONS_PER_ROOMbased on your expected load
MIT License. See LICENSE for details.