Skip to content

feat(actions): add a hosted build mode, PR-env inputs and commit pins (ENG-2000) - #71

Open
lucas-koontz wants to merge 2 commits into
mainfrom
feat/eng-2000-hosted-build-mode
Open

lucas-koontz wants to merge 2 commits into
mainfrom
feat/eng-2000-hosted-build-mode

Conversation

@lucas-koontz

@lucas-koontz lucas-koontz commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

User story

As a maintainer of a public repository that builds and deploys with these actions
I want build-push-ecr to build on a GitHub-hosted runner, and argocd-pr-env-deploy to deploy a PR that another workflow names
So that my repository's builds run on GitHub-hosted runners and its PR environments keep deploying

Why this matters

Today a maintainer who builds a public repository's image with build-push-ecr has to run that job on mdb-dev, because the action builds on the in-cluster BuildKit service with the runner pod's own AWS role. GitHub recommends GitHub-hosted runners for public repositories, because anyone can open a pull request there. With builder: local, the same action builds on a GitHub-hosted runner with a role the job assumes through GitHub OIDC, and private callers keep today's commands.

argocd-pr-env-deploy used to work only inside a pull_request run. It now takes the PR as inputs, so a scheduled workflow in another repository can deploy PR environments. Because that caller, not GitHub, names the PR, the action deploys only a PR whose branch lives in the named repository.

Test plan

Each line is a test in tests/ unless it says otherwise.

  • Remote mode is unchanged: a caller that passes no builder runs the same aws and docker commands, argument for argument. A comparison of every caller found, in every build environment, showed no difference.
  • Local mode makes no aws call and creates no remote builder, and adds only the GitHub Actions cache flags, on a BuildKit image pinned by digest.
  • Negative: any other builder value fails before any aws or docker call.
  • A pull_request run deploys development-<merge SHA> as before. A scheduled run that names a PR deploys development-head-<head-sha>.
  • Negative: a repository, PR number or head SHA of the wrong shape stops the step before any call, and the log prints only the rule.
  • Negative: a PR whose branch lives in a fork, or in a deleted fork, stops before any ECR or ArgoCD call.
  • A PR image tag is checked only in mindsdb-<repo>-dev when that repository exists, and only in mindsdb-<repo> otherwise.
  • Every third-party action in the changed composites is pinned to a commit. FranzDiebold/github-env-vars-action stays on v2.9.0, although its v2 tag has since moved to v2.10.0.
  • Not run yet: a local-mode build on a GitHub-hosted runner, which needs the cloud roles first. The first cowork build after the merge shows it.

Rollback: revert the merge commit. Revert the callers' builder: local first, because the reverted action would send them to the in-cluster builder.

Ships with

Part of ENG-2000. Merge with a merge commit (gh pr merge 71 -R mindsdb/github-actions --merge), so this PR's head stays on main: a private deploy repository pins argocd-pr-env-deploy to it.

  1. Our infrastructure change creates the OIDC roles and the dev-tier image repositories, and the operator applies it.
  2. This PR merges.
  3. The private deploy repository opens with this PR's head pinned.
  4. mindsdb/anton#526, mindsdb/cowork#1117 and mindsdb/cowork-server#621 merge into staging.

build-push-ecr gains a `builder` input. The default, `remote`, runs the
same create-repository, set-repository-policy, buildx create and build
calls as before. `local` builds on the docker-container builder that
setup-buildx-action starts on the runner. It caches layers in the GitHub
Actions cache under module-name, and it never creates an ECR repository
or changes a repository policy. So a caller job can push with a role it
assumes through GitHub OIDC, and that role needs push rights only.

In local mode, crazy-max/ghaction-github-runtime first exports the
cache service's ACTIONS_* variables, which buildx needs and run steps do
not get. The BuildKit container runs moby/buildkit v0.33.1, pinned to
its multi-arch index digest, because it holds the ECR registry
credentials while it pushes.

argocd-pr-env-deploy gains `repository`, `pr-number` and `head-sha`
inputs. They default to the pull_request event's values, so today's
callers pass nothing new. A scheduled or dispatched workflow passes them
to deploy a named PR. Before any API, ECR or ArgoCD call, the action
refuses a value that is not owner/name, a number or a 40-character SHA,
and it never prints the refused value. Outside a pull_request run it
deploys the PR head's development-head-<sha> tag, because that run
built no image of its own.

The action's ECR existence check now uses one repository per repo. When
mindsdb-<repo>-dev exists, it checks PR image tags there and only there.
Otherwise it checks mindsdb-<repo>. It asks describe-images whether the
-dev repository exists, so ecr:DescribeImages stays the only ECR
permission the action needs.

Every third-party action in build-push-ecr, snyk-docker-scan and
setup-env is pinned to the commit its tag points at today, with the
version in a comment. snyk-docker-scan also pins the Snyk CLI to
v1.1307.4, the current release. stale-deploy-label's prune job runs on
ubuntu-latest instead of mdb-dev.

The README documents the local builder, the new inputs, the
one-repository check and how to bump the BuildKit pin. Its secret-source
table describes what an environment's deployment branch policy can do.

New pytest suites run the build and deploy step scripts under bash with
aws, docker, curl and argocd stubbed, and assert on the argv of each
aws, docker and argocd call. Another suite fails if a pin loses its
commit, digest or version.

Part of Lucas Koontz's ticket "Take the public repos off the
credentialed runner group and require devops to release a privileged
run".

Refs: ENG-2000
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant