OpenCoroutineProxy

A production-grade API Gateway and Traffic Shadowing Proxy built with Kotlin Coroutines and Spring WebFlux (Netty).

Its defining feature is dark launching: every request is forwarded to a live primary backend and simultaneously mirrored to a staging backend via a fire-and-forget coroutine, with zero latency impact on the client response.

---

Why this exists

Problem	Solution
Testing staging changes against real traffic without risking users	Shadow (mirror) every request asynchronously
Proving the shadow is truly non-blocking	Integration test: primary = 50 ms, shadow = 5 000 ms → client receives response in < 1 000 ms
Resilience when upstream is down	Per-backend Resilience4j circuit breakers
Observability out of the box	Micrometer + Prometheus metrics, Grafana dashboard

---

Tech stack

Layer	Technology
Language	Kotlin 1.9
Framework	Spring Boot 3.3 (WebFlux / Netty)
Routing	coRouter functional endpoints
HTTP client	Spring WebClient
Concurrency	Kotlin Coroutines (Dispatchers.IO, SupervisorJob)
Resilience	Resilience4j Circuit Breaker
Observability	Micrometer + Prometheus + Grafana
Testing	JUnit 5, MockK, WireMock, Awaitility
Load testing	Gatling 3.10
Container	Docker (multi-stage, JRE 21, non-root, ZGC)

---

Quick start

Prerequisites

• Java 21+
• Docker and Docker Compose
• Gradle (only needed once for setup.sh)

First-time setup

git clone <repo-url>
cd OpenCoroutineProxy
bash setup.sh        # generates Gradle wrapper

Run tests

./gradlew test

Run locally

./gradlew bootRun
# Proxy starts on http://localhost:8080

Full Docker pipeline (build + start + smoke test + teardown)

make docker-test

All Makefile targets

make setup        First-time: generate Gradle wrapper
make build        Compile and package the fat JAR
make test         Run all unit + integration tests
make run          Start the proxy on localhost:8080
make docker-build Build the Docker image
make docker-up    Start proxy + mock backends + Prometheus + Grafana
make docker-down  Tear down the Docker stack
make smoke-test   Run curl-based smoke tests (set PROXY_URL= to override)
make docker-test  End-to-end Docker pipeline
make load-test    Run Gatling simulation (proxy must be running)

---

Configuration

All routing configuration lives in src/main/resources/application.yml.

proxy:
  routes:
    # Route with shadowing enabled
    - path-match: "/api/v1/payments/**"
      primary-url: "http://prod-payment-service:8080"
      shadow-url:  "http://staging-payment-service:8080"   # optional

    # Route without shadowing
    - path-match: "/api/v1/users/**"
      primary-url: "http://prod-user-service:8080"

    # Catch-all fallback
    - path-match: "/**"
      primary-url: "http://prod-default-service:8080"

  webclient:
    primary-timeout-ms: 5000   # exceeded -> HTTP 504 to client
    shadow-timeout-ms:  10000  # exceeded silently

Zero-downtime config for Kubernetes / Docker: override any property via environment variable using Spring Boot's relaxed binding:

PROXY_ROUTES_0_PRIMARY_URL=http://new-service:8080 ./gradlew bootRun

---

Architecture

Client
  │
  ▼
┌─────────────────────────────────────────────┐
│  ProxyRouter (coRouter catch-all)           │
│  ┌──────────────────────────────────────┐   │
│  │  ProxyHandler (suspend fun handle)   │   │
│  │                                      │   │
│  │  1. Read body bytes (once)           │   │
│  │  2. ShadowingService.fire()  ──────► │──►│─── shadow coroutine (Dispatchers.IO)
│  │     (returns immediately)            │   │         │ fire-and-forget
│  │  3. Forward to primary ─────────────►│──►│ Primary Backend
│  │     (circuit-broken, awaited)        │   │
│  └──────────────────────────────────────┘   │
│            │                                │
└────────────│────────────────────────────────┘
             ▼
          Client response (~primary latency only)

Traffic shadowing — how it works

ShadowingService.fire() is a regular (non-suspend) function. It schedules a coroutine on a dedicated CoroutineScope(Dispatchers.IO + SupervisorJob()) and returns in microseconds. The handler thread is never blocked waiting for the shadow backend.

Key design properties:

• SupervisorJob: one shadow failure does not cancel other in-flight mirrors
• DisposableBean.destroy(): the scope is cancelled cleanly on application shutdown
• Circuit breaker on the shadow path (more lenient thresholds than primary)
• All shadow errors are caught, logged at WARN, and swallowed — never propagated to the client

---

Injected headers

Every forwarded request carries:

Header	Value
X-Proxy-Trace-Id	Random UUID, same value sent to both primary and shadow
X-Forwarded-For	Original client IP
X-Forwarded-By	OpenCoroutineProxy
X-Shadow-Request	true (shadow requests only)

Hop-by-hop headers (Connection, Transfer-Encoding, Host, etc.) are stripped before forwarding.

---

Observability

Endpoint	Description
GET /actuator/health	Liveness / readiness
GET /actuator/prometheus	Prometheus scrape endpoint

Key metrics:

Metric	Type	Tags
proxy_primary_request_duration_ms	Timer	route, outcome
proxy_shadow_request_duration_ms	Timer	outcome
proxy_shadow_requests_fired_total	Counter	—
proxy_shadow_requests_total	Counter	outcome

When running with make docker-up, Grafana is available at http://localhost:3000 (admin / admin) with Prometheus pre-configured as a datasource.

---

Project structure

OpenCoroutineProxy/
├── src/main/kotlin/com/openproxy/
│   ├── OpenCoroutineProxyApplication.kt   Entry point
│   ├── config/
│   │   ├── ProxyProperties.kt             @ConfigurationProperties binding
│   │   └── WebClientConfiguration.kt     WebClient bean
│   ├── routing/
│   │   ├── ProxyRouter.kt                 coRouter (catch-all)
│   │   └── ProxyHandler.kt                Core proxy logic + header injection
│   ├── shadow/
│   │   └── ShadowingService.kt            Fire-and-forget coroutine engine
│   ├── resilience/
│   │   └── ResilienceConfiguration.kt     Circuit breaker beans
│   └── metrics/
│       └── ProxyMetrics.kt                Micrometer facade
├── src/test/kotlin/com/openproxy/
│   ├── config/ProxyPropertiesTest.kt       YAML binding unit tests
│   ├── routing/ProxyIntegrationTest.kt     Full-stack WireMock integration tests
│   └── shadow/ShadowingServiceTest.kt      Shadow service unit tests
├── load-tests/                             Gatling subproject (separate Gradle module)
│   └── src/gatling/kotlin/.../
│       └── ProxyShadowSimulation.kt        4-phase load profile + assertions
├── docker/
│   ├── wiremock/primary/mappings/          50 ms response stubs
│   ├── wiremock/shadow/mappings/           5 000 ms response stubs
│   ├── prometheus/prometheus.yml           Scrape config
│   └── grafana/provisioning/              Datasource auto-provisioning
├── scripts/
│   ├── smoke-test.sh                       6-point curl validation
│   └── docker-test.sh                      Full Docker pipeline script
├── docker-compose.yml                      Proxy + mocks + Prometheus + Grafana
├── Dockerfile                              Multi-stage build (JDK builder + JRE runtime)
├── Makefile                                Developer task runner
└── setup.sh                                First-time Gradle wrapper setup

---

Load testing

The load-tests/ Gradle subproject contains a Gatling simulation with a 4-phase injection profile:

1. Warm-up — ramp from 10 to 80% of targetRps over 30 s
2. Sustained — hold at 80% for 60 s
3. Spike — burst to 200% for 10 s (tests coroutine back-pressure)
4. Cool-down — ramp to 0 over 10 s

Built-in Gatling assertions fail the run if:

• Mean response time > 200 ms
• p99 response time > 1 000 ms
• Error rate > 1%

# proxy must be running first
make docker-up
make load-test              # default 200 rps
make load-test ARGS=-DtargetRps=500

Reports are saved to load-tests/build/reports/gatling/.

---

Running in production

The proxy is configured entirely through environment variables — no file changes required:

docker run -p 8080:8080 \
  -e PROXY_ROUTES_0_PATH_MATCH="/api/**" \
  -e PROXY_ROUTES_0_PRIMARY_URL="http://prod-service:8080" \
  -e PROXY_ROUTES_0_SHADOW_URL="http://staging-service:8080" \
  -e PROXY_WEBCLIENT_PRIMARY_TIMEOUT_MS=3000 \
  open-coroutine-proxy:latest

JVM flags in the Dockerfile are tuned for containerised, latency-sensitive workloads:

• -XX:+UseZGC — low-pause GC
• -XX:MaxRAMPercentage=75.0 — respects container memory limits
• -Djava.security.egd=file:/dev/./urandom — avoids /dev/random entropy blocking