Local Development Cluster

Local Development Cluster

This page is the contributor onramp for the cluster-based TMI development environment. make start-dev deploys the TMI server and the full component platform (Redis, NATS JetStream, KEDA, the TMIComponent controller, and the extractor/chunk-embed workers) into a local Kubernetes cluster, talking to a developer-owned external database — exactly the shape TMI runs in production.

⚠️ Breaking change: the host-process dev path is retired

make start-dev no longer runs the server and workers as host Go processes. It now builds container images, pushes them to a local registry, and deploys them into whatever cluster your current kubectl context targets. There is no longer a host tmi-server process or a make start-server step in the dev loop.

If you have an old workflow that expected start-dev to launch a bare Go binary, that path is gone. The standalone manage-database.py local Postgres container is retained — but only as one of several database options you may point your config at (see Database configuration).

---

How it works

make start-dev is cluster-agnostic: it deploys into the cluster your active kubectl context points at. Two backends are supported from the same path:

Backend	Use	Cluster lifecycle
kind (laptop / Docker Desktop)	The default laptop path	make dev-cluster-up / make dev-cluster-down create and delete it for you
k3s (dedicated desktop/server)	A persistent dev cluster	You manage it; start-dev only deploys into it

The database is external and config-defined — the server pod dials whatever database.url your config-development.yml specifies (local Postgres container, cloud Postgres, or OCI Autonomous Database). There is no Kubernetes Service for the database; this mirrors production, where the server connects to RDS/ADB directly.

Container images reach the cluster through a local registry at localhost:5000 that both kind and k3s pull from.

host: curl / tmi-ux / oauth-stub / postman / wstest
        │  http://localhost:8080  (managed kubectl port-forward → svc/tmi-server)
        ▼
   tmi-server (pod)                 config: ConfigMap (your config-development.yml)
        │  database.url from config ─────► external DB (host PG / cloud PG / OCI ADB)
        │                                   (Oracle: wallet Secret mounted in the pod)
        │  redis (in-cluster svc) ◄─ TMI_REDIS_HOST=redis (baked into server.yml)
        │  TMI_NATS_URL=nats://nats.tmi-platform.svc:4222
        ▼
   NATS JetStream ──► KEDA scales ──► tmi-extractor / tmi-chunk-embed (0→N)

Everything lands in the tmi-platform namespace.

---

Prerequisites

Tool	Required?	Notes
docker	Yes	Builds images; hosts the local registry and (on kind) the cluster
kubectl	Yes	start-dev deploys through your active context
A local cluster	Yes	A kind cluster (make dev-cluster-up) or a dedicated k3s cluster
config-development.yml	Yes	Your developer-owned config (gitignored); see Configuring Local Development
tilt	Optional	Only for the fast inner loop; never required
go	For Tilt only	The Tilt loop cross-compiles the binary on the host

Create your config first if you haven't:

cp config-example.yml config-development.yml
# then edit database.url, OAuth, JWT secret, etc. — see Configuring Local Development

---

Cluster setup

Option A — kind (laptop)

make dev-cluster-up does everything in one step: it starts the local registry container (tmi-dev-registry on localhost:5000), creates a two-node kind cluster named tmi-dev, and wires the cluster's containerd registry mirror so localhost:5000 resolves to the registry. No manual registry config is required on kind.

make dev-cluster-up      # create the kind cluster + registry (idempotent)
# ... develop ...
make dev-cluster-down    # delete the kind cluster (registry container is left)

The kind context is named kind-tmi-dev, which start-dev recognizes as a safe local context automatically.

Option B — k3s (dedicated cluster)

start-dev does not create or delete a k3s cluster — you manage that. You do need to teach k3s to pull from the local registry once. Add a registry mirror to /etc/rancher/k3s/registries.yaml:

# /etc/rancher/k3s/registries.yaml
mirrors:
  "localhost:5000":
    endpoint:
      - "http://localhost:5000"

Then restart k3s so it picks up the change:

sudo systemctl restart k3s

The local registry container itself is started for you by start-dev (and dev-cluster-up) if it isn't already running; it listens on 127.0.0.1:5000.

Prod-context guard. start-dev refuses to deploy into a context it doesn't recognize as local, to avoid accidentally targeting a real cluster. Contexts auto-accepted: anything starting with kind- or k3d-, and the exact names k3s, default, rancher-desktop, docker-desktop, minikube. For any other context name (e.g. a custom k3s context), pass --yes to override — e.g. uv run scripts/start-dev.py --yes, or rename your context to one of the recognized names.

---

Bringing the environment up

# Default: Postgres database target
make start-dev

# Oracle Autonomous Database target (see Oracle section below)
make start-dev DB=oracle

make start-dev runs, in order:

1. Preflight — checks docker and kubectl, and that a cluster is reachable.
2. Context guard — refuses non-local contexts unless --yes.
3. Registry — starts the tmi-dev-registry container if absent.
4. Build + push four images to localhost:5000: the server (tmi-server for Postgres, or tmi-server-oracle for Oracle), the tmi-component-controller, and the tmi-extractor / tmi-chunk-embed workers.
5. Platform base — applies NATS, KEDA, the CRD, and the in-cluster controller (ServiceAccount + RBAC + Deployment); waits for the controller rollout.
6. Config + secrets — renders your current config-development.yml into the tmi-server-config ConfigMap, creates the embedding Secret, and (for Oracle) the wallet Secret.
7. Deploy — applies the dev overlay (Redis, server) and the worker component CRs.
8. Expose — waits for the tmi-server rollout, starts a managed kubectl port-forward svc/tmi-server 8080:8080, and prints http://localhost:8080.

Verify it's up:

curl http://localhost:8080/
# returns the server version JSON. (There is no /health endpoint — use /.)

For authentication, use the built-in tmi OAuth provider with the callback stub — see Configuring Local Development → Authenticating in Development.

Skipping the workers

make start-dev accepts a flag to skip the extractor/chunk-embed component CRs (the server, Redis, and controller still deploy):

uv run scripts/start-dev.py --no-workers

---

Database configuration (your responsibility)

The server reads database.url from config-development.yml and connects directly to that endpoint. The dev environment adds no database plumbing — choosing and reaching the database is yours to configure, exactly as in production.

Local Postgres container

make start-database runs a local Postgres container via manage-database.py. The one gotcha: from inside a pod, localhost points at the pod, not your host. Set the host to a pod-reachable address:

Cluster	Use this host
kind / Docker Desktop	host.docker.internal
k3s (same machine)	the host / node IP

# config-development.yml
database:
  url: "postgres://tmi_dev:dev123@host.docker.internal:5432/tmi_dev?sslmode=disable"

localhost in database.url will not work from inside a pod — this is the first thing to check if the server can't reach the database.

Cloud Postgres

Point database.url at the cloud endpoint. This works with the default static server image (the Postgres driver is pure Go):

database:
  url: "postgres://user:pass@db.example.com:5432/tmi?sslmode=require"

OCI Autonomous Database (DB=oracle)

Oracle ADB needs the Oracle server image variant (CGO + Instant Client) and the connection wallet:

1. Set an Oracle DSN in database.url:

   database:
     url: "oracle://user:pass@adb-dsn-name"

2. Point TMI_ORACLE_WALLET_ZIP at your downloaded ADB wallet .zip file. start-dev delivers it as the tmi-oracle-wallet Secret, and the Oracle image extracts it at startup:

   export TMI_ORACLE_WALLET_ZIP=/path/to/Wallet_TMIDEV.zip
   make start-dev DB=oracle

DB=oracle fails fast (before deploying) if TMI_ORACLE_WALLET_ZIP is unset or doesn't point at a file.

Swapping database targets

Config is delivered dynamically on every deploy — the image carries code only. To switch targets, edit database.url (and DB= if changing Postgres↔Oracle) and re-run:

make restart-dev            # or: make restart-dev DB=oracle

A content hash on the ConfigMap forces the pod to roll and pick up the change.

---

Day-to-day commands

Command	What it does
make dev-cluster-up	Create the local kind cluster + registry (laptop path)
make dev-cluster-down	Delete the local kind cluster (registry container left running)
make start-dev [DB=oracle]	Build, push, and deploy the full environment; port-forward to :8080
make restart-dev [DB=oracle]	Rebuild + push the server, redeliver config, roll the server pod (~20–60 s)
make stop-dev	Tear down everything start-dev deployed; leaves the cluster intact
make tilt-up	Optional fast server-only loop (Postgres path; requires tilt)
make tilt-down	Stop Tilt and restore the prod-shaped server

What stop-dev removes

make stop-dev deletes the server, Redis, controller (+ RBAC), the worker component CRs, the config ConfigMap, and the embedding/wallet Secrets; kills the managed port-forward; and stops the local registry container. It does not delete your cluster (use make dev-cluster-down for an ephemeral kind cluster) and does not stop a local Postgres container or the OAuth stub if you started those separately.

---

Optional: Tilt fast inner loop

make tilt-up gives a sub-5-second server-only inner loop on top of an environment you already brought up with make start-dev. It takes over only deploy/tmi-server: on each save it cross-compiles the binary on the host (CGO_ENABLED=0 GOOS=linux go build -tags=dev), builds a one-COPY image on the prod static base, pushes it to localhost:5000, and rolls the server in seconds — versus make restart-dev, which rebuilds all four images.

make start-dev      # bring the full environment up first
make tilt-up        # foreground; edit Go sources and watch the server roll
# Ctrl-C, then:
make tilt-down      # stop Tilt and restore the prod-shaped server

Notes:

• Postgres path only. The Tilt loop always builds/restores the Postgres server.yml. The Oracle CGO image is out of fast-loop scope — if you brought the environment up with DB=oracle, make tilt-down restores the Postgres server, so re-run make start-dev DB=oracle to get the Oracle server back.
• tilt-down re-applies the canonical server.yml, so the server returns to its prod-shaped, restart-dev-managed form.
• The config ConfigMap is untouched by Tilt.

---

Troubleshooting

Symptom	Likely cause / fix
No reachable cluster	Run make dev-cluster-up (kind) or start your k3s cluster
Context '…' is not a recognized local dev cluster	The prod-context guard fired — re-run with --yes or use a recognized context name
Server can't reach the database	database.url uses localhost — switch to host.docker.internal (kind) or the node IP (k3s)
ImagePullBackOff with a localhost:5000/... image	Registry not reachable from the cluster — on k3s, check /etc/rancher/k3s/registries.yaml and restart k3s; on kind, re-run make dev-cluster-up
DB=oracle exits immediately	TMI_ORACLE_WALLET_ZIP is unset or not a file
tilt: command not found on make tilt-up	Install Tilt (https://docs.tilt.dev/install.html) — it's optional
Lost connection to localhost:8080	The managed port-forward dropped; make restart-dev re-establishes it, or restart kubectl port-forward -n tmi-platform svc/tmi-server 8080:8080

Server logs are inside the pod:

kubectl -n tmi-platform logs deploy/tmi-server -f

---

See Also

• Configuring Local Development — config file layout, secrets, and authenticating with the tmi OAuth provider
• Getting Started with Development — overall dev onramp (server + web app)
• Database Setup — Postgres container details and schema migration
• OCI Container Deployment — Oracle ADB and the Oracle image variant in production
• Component Integration — the TMIComponent platform that runs in the cluster