Repository navigation
Conversation
|
Coverage of the acceptance run against the meshStack backend, on
Uncovered functions
|
|
Coverage of the terraform-provider-meshstack acceptance run against the meshStack backend, on
Uncovered functions
|
5749f1c to
c41637f
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Failure reporting can be permanently lost, and debug HTTP logging can expose sensitive output values.
3 open findings
What changed in this PR
Consolidates the Go runners into one configurable binary supporting in-process and Kubernetes Job dispatch.
Changes:
- Adds shared configuration, HTTP, crypto, and dispatcher packages.
- Migrates Terraform and manual execution to the unified runner.
- Updates containers, CI, tests, and documentation for both build modes.
| File | Description |
|---|---|
cmd/runner/* |
Adds the unified runner entry point and build-tag dispatchers. |
internal/config/* |
Adds unified and legacy-compatible configuration handling. |
internal/crypto/* |
Adds run decryption and redaction. |
internal/http/* |
Adds authenticated HTTP, retry, logging, and timeout handling. |
internal/http/meshapi/* |
Adds typed runner API DTOs and operations. |
internal/dispatcher/* |
Adds claiming, registration, status reporting, and run-file execution. |
internal/dispatcher/inprocess/* |
Adds in-process Terraform/manual dispatch. |
internal/dispatcher/k8s/* |
Adds Kubernetes Job dispatch and informer recovery. |
internal/build/build.go |
Centralizes build version metadata. |
tf-block-runner/tfrun/* |
Migrates Terraform execution to shared APIs and configuration. |
tf-block-runner/Dockerfile |
Builds the unified runner for Terraform. |
tf-block-runner/README.md |
Documents unified runner usage. |
tf-block-runner/main.go |
Removes the former Terraform entry point. |
tf-block-runner/build/build.go |
Removes runner-specific build metadata. |
tf-block-runner/resources/test.pem |
Removes obsolete crypto fixture. |
tf-block-runner/.gitignore |
Removes package-local ignore rules. |
manual-block-runner/manual/* |
Converts manual execution into an in-process executor. |
manual-block-runner/Dockerfile |
Builds the unified runner for manual runs. |
manual-block-runner/main.go |
Removes the former manual entry point. |
run-controller/controller/* |
Removes the superseded controller implementation. |
run-controller/inprocess/* |
Removes the former in-process dispatcher. |
run-controller/build/* |
Removes controller-specific build and dispatcher types. |
run-controller/main.go |
Removes the former controller entry point. |
run-controller/Dockerfile |
Builds the Kubernetes-tagged unified runner. |
run-controller/k8s/rbac.yaml |
Adds Job patch permission. |
run-controller/README.md |
Documents Kubernetes-tagged runner usage. |
go-meshapi-client/* |
Removes the superseded client and crypto implementation. |
Taskfile.yml |
Runs builds and checks with both dispatcher modes. |
.github/workflows/ci.yml |
Builds, lints, and tests both build-tag variants. |
go.mod / go.sum |
Updates configuration and YAML dependencies. |
README.md |
Documents the single-binary architecture. |
meshstack-satellite.gradle |
Updates runner terminology. |
.gitignore |
Preserves the centralized build package. |
.agents/skills/backend-go/* |
Updates repository Go development guidance. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
1e5eca3 to
ec806a3
Compare
ec806a3 to
edf06e0
Compare
ab7e76e to
c66d00e
Compare
c66d00e to
aa082e3
Compare
malhussan
left a comment
There was a problem hiding this comment.
I've checked config and mostly higher level, great work!
Regarding legacy, is there a plan when we will phase it out?
ed59a23 to
4f41956
Compare
4f41956 to
27837ec
Compare
27837ec to
ad999af
Compare
e92102b to
412aa16
Compare
bb41651 to
656f9e8
Compare
- ignore the direnv files - take over the golangci-lint linter set of meshstack-cli; tf-block-runner, go-meshapi-client, run-controller and manual-block-runner stay excluded until they are reworked or deleted - update golangci-lint to v2.15.0 and golang.org/x/crypto to v0.58.0 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- internal/crypto holds the runner's private keys, inline or from files, and decrypts with any of them; its test keeps a value the former client encrypted - internal/config loads one runner-config.yml in the new or a legacy layout, resolves Spring placeholders and RUNNER_* env overrides and validates per mode; config.Api answers the meshstack-cli settings, and internal/config/tf holds the tf settings - crypto.DecryptRun decrypts the sensitive inputs and the implementation secret of a meshstack-cli run go.mod pins meshstack-cli at the head of meshstack-cli#33 until it merges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- internal/executor runs the run in RUNNER_RUN_YML with the executor of its type and reports through a Reporter's steps; exit code 0 means it delivered a final or handed-over status, 3 that it delivered FAILED; closing stdin stops the run with the reason written before it - internal/poller claims and decrypts runs up to maxConcurrentJobs and reports FAILED for a run that did not deliver its status; the process dispatcher starts the binary again per run with an allowlisted env, a run dir of its own and stdin as lifeline, and kills the process group killGrace after it closed stdin - one main in cmd/runner for every image; tf and manual register as executors, tfrun clones with the git CLI and SIGTERM fails a run with the signal as its reason - an asynchronous Terraform run reports one "Prepare Run" step and is handed over to the pipeline its code starts, as the tf runner on main did; an async destroy applies first, so the destroy sees this run's inputs - a run builds only the runner API of the meshStack client, and logs each finished tofu command with its duration and exit code - dead tf-block-runner code goes, and go-meshapi-client is deleted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- internal/dispatcher/k8s starts a Job per run on the image of its type, with the run yml and run token in a Secret and an emptyDir as work dir; a podFailurePolicy maps exit code 3 to a Job condition and an informer keeps the capacity count across restarts - a Job gets backoffLimit 0 and dispatcher.killGrace as termination grace - k8s.jobTtl (default 60 minutes) keeps a finished Job, unless the job spec template sets ttlSecondsAfterFinished - every Job becomes an owner of its run Secret and of the per-definition ServiceAccount through a strategic merge patch, so both go with their last Job; the runner needs patch on secrets and serviceaccounts - creating the service account, Secret and Job retries transient Kubernetes API errors Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dispatcher.keycloakWif takes over the run-controller WIF: the poller mints one token per TERRAFORM run from a Keycloak client of its own, with sub bbd:<workspace>:<definition uuid>, and registers Keycloak as the runner's issuer. A run whose token cannot be issued fails with a fixed message and does not start. - the process dispatcher writes the token below the runner dir, the k8s dispatcher puts it into the run's Secret - the tf executor links it as .meshstack/wif-token and passes its path in meshstack_keycloak_wif_token_file Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pace of the building block - a flock per version under the install dir keeps two run processes from installing the same version at once - the install goes to a temp dir that is renamed when complete, so an interrupted install no longer counts as complete - the install logs through slog with the version and path - a destroy deletes the workspace of the building block Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- /metrics and /healthz on RUNNER_METRICS_PORT, or PORT when unset; a port in use fails before the runner claims a run - run_duration_seconds counts a run as failed when its exit code is not 0 - each finished run logs "run finished" with the labels of run_duration_seconds and its exit code - a k8s build also exposes the run-controller metrics under their old names, except those that active_runs and max_concurrent_runs replace Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The debug config points at http://localhost:8080, as the CLI accepts http only for a loopback host, and each Job gets a socat sidecar that forwards that port to meshfed-api on the host. The config moves to the current layout, as only job spec templates can declare the sidecar. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- a Reporter adds steps after it registered, and Step.SetStatus sets a step's status without failing the run - the gitlab, azuredevops and github executors; a sync run reports the jobs or stages as steps and cancels the external run when it stops; GitHub authenticates as the GitHub App and finds the run by time when a dispatch has no run details; GitLab's MESHSTACK_RUN leaves out the trigger token - GitLab gets an input that is no string as JSON text, which it parses into the type of a number, boolean or array spec:inputs entry - executor.NewPipelineClient, which follows no redirects and retries a trigger only on 429 and 503 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rt:runner Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- containers/Dockerfile builds every image from the Go binary, with the target runner or tf and the build tag of the image, including the all-types building-block-runner image without a build tag - dev/ holds the runner configs of task start:runner and task debug:minikube - Gradle and the Kotlin runners are removed - the k8s Job no longer sets RUN_JSON_FILE_PATH and RUNNER_API_URL, and the configs drop SPRING_PROFILES_ACTIVE - no image sets SSL_CERT_DIR, which the manual image did: git ignores it, and the entrypoint adds custom CA certificates to the system bundle that git and Go read - the images build with VCS stamping, so the runner reports the module version Go derives from the tags and the git revision BREAKING CHANGE: the github, gitlab and azure-devops images run the Go binary, which reads runner-config.yml from /app/runner-config.yml or the path in RUNNER_CONFIG_FILE. Spring's discovery of /app/config and the Spring environment no longer apply. Every image serves metrics on RUNNER_METRICS_PORT=2112; tf and the other runner images used PORT=8080. No image contains a runner-config.yml any more, so a deployment must mount /app/runner-config.yml or set RUNNER_CONFIG_FILE. The Go module path is github.com/meshcloud/building-block-runner/v2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…estacc rules The name matches the development skill of meshstack-cli. .claude/settings.json enables the modern-go-guidelines plugin, which the skill builds on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- each TestAcc<Case> deploys the folder testdata/<case> with the meshStack provider, starts the runner binary built with coverage and asserts its JSON log and the run in meshStack - it replaces the root testacc/ package - tf runs: inputs, secrets, timeout, abort, plan approval, stale plan, killed child, workspaces, Terraform 1.5.5, failing init, git ref as tag and commit, asynchronous Terraform; manual and noop runs - GitHub, GitLab and Azure DevOps runs against mocks of their APIs - SSH clone of the module and of a child module from an in-process git server - Keycloak workload identity against the stack's Keycloak - network faults with toxiproxy - the harness, the runner process, the mocks, the SSH server, toxiproxy and Keycloak are subpackages, which depguard keeps out of non-test code - task test:acc:coverage merges the unit and acceptance coverage Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ges current Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|





single-go-bin-runner
One Go binary replaces every building block runner: Terraform, manual, GitHub workflows, GitLab pipelines and Azure DevOps pipelines. Gradle, Kotlin and the JVM images are gone from this repo. This PR makes the next release v2.0.0.
The binary runs each run as a process from a run yml: it starts itself (process dispatcher), or the k8s dispatcher (
-tags k8s) starts a Job on the runner type's image. A build tag per type (tf,manual,github,gitlab,azuredevops) links only that executor; no tag links all of them. The runner talks to meshStack only through the meshstack-cli client, so it also runs locally with a CLI profile.The commits build on each other and read best one by one.
What changes for operators
containers/Dockerfilebuilds every image. The five type images keep their names, and the new imagebuilding-block-runnerruns every type.run-controlleris the binary with-tags k8s.backoffLimit: 0(main had 1), so a run never runs twice. Exit code 3 reaches the run-controller through apodFailurePolicy.k8s.jobTtl(default 60 minutes) keeps a finished Job. The run Secret and the per-definition ServiceAccount are owned by their Jobs and go with the last one.dispatcher.keycloakWiffor every dispatcher.building_block_runner_*; the k8s build also serves the oldrun_controller_*names.Breaking changes
/app/runner-config.ymlorRUNNER_CONFIG_FILE, and no image contains a config. Spring's discovery of/app/configand the Spring env are gone./healthzare served onRUNNER_METRICS_PORT=2112; the images usedPORT=8080.patchon Jobs, Secrets and ServiceAccounts.SSL_CERT_DIRany more; the entrypoint adds custom CA certificates to the system bundle.github.com/meshcloud/building-block-runner/v2.awscall installs awscli2 throughnix profile add(a few seconds, output on stderr), and later calls use it directly. The image has no nix channel, sonix-env -iandnix-shell -pno longer work;nix profile add nixpkgs#<pkg>andnix shell nixpkgs#<pkg>use the nixpkgs pinned incontainers/nix/flake.lockwithout downloading it, and dependabot keeps that pin current.Tests
internal/testaccstarts the binary built with-coveragainst a live meshStack: tf (also asynchronous), manual, GitHub, GitLab and Azure DevOps (against mocks of their APIs), SSH clone, Keycloak WIF and network faults.task test:acc:coveragemerges unit and acceptance coverage.Merge order
go.modpin from the fix: gradle rootProject name #33 head to that release.Cross-repo PRs
client/httpexported,client.Runnerrunner_refin the hub modules, and themeshstack/run-controllermodule🤖 Generated with Claude Code