OpsForge — Event-Sourced Infrastructure Automation Platform

Build Doctrine: Everything is an event. Execution is stateless. Truth is reconstructable.

Core Principles

This system is built following a strict A→Z build doctrine to ensure zero rework and production-grade reliability:

1. Event Sourcing: All state changes are immutable events (append-only)
2. Stateless Workers: Workers process jobs independently, no shared state
3. Idempotency: Same job cannot run twice (deduplication guard)
4. Traceability: Correlation IDs propagate through entire system
5. Pluggable Executors: Support for SSH, Docker, Kubernetes
6. Projections: Read models derived from events for fast queries

Architecture

API → Event Store → Queue → Worker → Executor
                              ↓
                        Observability (Logs, Traces)
                              ↓
                         Projections (Read Models)
                              ↓
                              UI

Phases

Phase 0: Environment

• Docker Compose with PostgreSQL, Redis, API, Worker

Phase 1-5: Event Core

• Immutable event store
• Job model with idempotency
• BullMQ queue
• Pluggable executors (SSH)
• Event-driven job lifecycle

Phase 6-7: Security

• JWT authentication
• RBAC (admin/operator/viewer)
• Correlation ID propagation
• Structured logging

Phase 8-10: Observability

• Retry engine with exponential backoff
• Structured JSON logs
• Projections (read models)

Phase 11-12: UI & Deployment

• React dashboard
• Docker Compose
• Kubernetes Helm charts

Phase 13-15: Hardening

• Vault integration
• Audit logging
• Testing & validation

Quick Start

# Install dependencies
pnpm install

# Start services
docker-compose up -d

# Run migrations
docker-compose exec api node scripts/migrate.js

# Create job
curl -X POST http://localhost:3000/jobs \
  -H "Content-Type: application/json" \
  -d '{
    "idempotencyKey": "job-1",
    "target": "user@host:22",
    "executor": "ssh",
    "command": "echo hello"
  }'

# Check status
curl http://localhost:3000/jobs/{jobId}

API Endpoints

Authentication

• POST /login - Get JWT token

Jobs

• POST /jobs - Create job (with idempotency)
• GET /jobs/:jobId - Get job status (from projection)
• GET /jobs/:jobId/events - Get job event history

Event Types

• JOB_CREATED - Job created
• JOB_STARTED - Execution started
• JOB_SUCCESS - Completed successfully
• JOB_FAILED - Execution failed
• JOB_RETRY - Scheduled for retry
• JOB_FINAL_FAILED - Max retries exceeded

Key Files

server/
├── api.ts                 # Express API server
├── worker.ts              # Worker entry point
├── db/
│   ├── schema.sql         # Event store schema
│   ├── events.ts          # Event store service
│   └── jobs.ts            # Job store service
├── queue/
│   └── queue.ts           # BullMQ queue
├── executors/
│   ├── executor.ts        # Executor interface
│   └── ssh.executor.ts    # SSH executor
├── worker/
│   └── processor.ts       # Job processor
├── auth/
│   └── jwt.ts             # JWT authentication
├── logging/
│   └── logger.ts          # Structured logging
└── projections/
    └── builder.ts         # Projection builder

Development

# Watch mode
pnpm dev

# Worker
pnpm worker

# Tests
pnpm test

# Lint
pnpm lint

Production Deployment

# Build Docker images
docker build -f Dockerfile.api -t opsforge-api:latest .
docker build -f Dockerfile.worker -t opsforge-worker:latest .

# Deploy to Kubernetes
helm install opsforge ./infra/helm

Design Decisions

Why Event Sourcing?

• Complete audit trail
• Full state reconstruction
• Debuggable failures
• Compliance-ready

Why Stateless Workers?

• Horizontal scaling
• Crash recovery
• No shared state
• Independent processing

Why Idempotency?

• Safe retries
• Duplicate prevention
• Exactly-once semantics

Why Correlation IDs?

• Request tracing
• Debugging support
• Observability

Common Failure Points (Avoid These)

1. ❌ Skipping event model → system collapses later
2. ❌ Mixing execution into API → kills scalability
3. ❌ No idempotency → duplicate chaos
4. ❌ No correlation → impossible debugging
5. ❌ Adding UI too early → wrong architecture

License

MIT