Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,40 @@
Versions are git tags (`vX.Y.Z`); projects install one with
`uv tool install git+https://github.com/hybridinteract/deployctl@vX.Y.Z`.

## Unreleased

**Added**
- **A release branch: build on one branch, deploy from another.** `BUILD_BRANCH` (`ci init
--build-branch main`, or Configure → *Build branch*) says images are built on `main`, and
`DEPLOY_BRANCH=prod` that merging into `prod` ships. `build-image.yml` then publishes from
`main`, a new `ci.yml` checks and builds without deploying, and a third managed workflow,
`release.yml`, deploys the `main` commit `prod` matches once that commit's CI has passed.
This is herbally's hand-written `release.yml`, made a template — with it, the local edit to
`deploy.yml`'s branch check that `ci init --force` would drop is no longer needed. Empty
(the default), nothing changes.

**Faster** (both from herbally's own edits, measured on its production deploys)
- **CI deploys over one ssh connection per host.** `deploy.yml` sets `DEPLOYCTL_SSH_MUX=1`:
the engine opens a `ControlMaster` per host in a private `/tmp/dctl-mux.*` directory and
closes it on exit, error or cancel. Each handshake from a runner costs seconds and an
update makes dozens; herbally's deploy step went from ~350s to ~170s. herbally did this by
writing `~/.ssh/config` and `pkill`-ing the masters; the engine keeps it to its own
connections, so a self-hosted runner's ssh config is never touched.
- **A new `ci.yml` builds the image while the checks run** (no `needs: checks` on `publish`);
the deploy job waits for both. An existing `ci.yml` is yours and keeps its order.

**Fixed**
- **CI/CD said "Finish CI/CD: .github/workflows/ci.yml" for a pipeline that deployed.**
`ci doctor` looked for the text `deploy.yml` in `ci.yml` alone; herbally calls it from
`release.yml`, so the panel's stepper never reached *Live*. Every workflow is now scanned
for the job that calls `deploy.yml`, and that job is the one checked for `packages: read` —
which now also accepts the inline `permissions: { contents: read, packages: read }` that
doctor's own fix suggests, `read-all`, and a workflow-level grant the job inherits.
- **Deploys on merge were missing from `ci runs` and the panel's history** unless `ci.yml`
called the deploy. They are read from whichever workflow does, and the tag shown is the
image deployed (from the deploy job's name), not the commit pushed — the two differ for a
release, whose merge commit never had an image.

## 0.14.1

Fixes from taking a second project (herbally) to its first server: things that failed late,
Expand Down
6 changes: 3 additions & 3 deletions docs/15-FIRST-DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ validation re-runs after every Save.
| **PostgreSQL** | Single: `container` (default; the password is generated once into `config/secrets.<env>.env` and reused forever), or `external` with `POSTGRES_HOST`, `POSTGRES_PASSWORD` and `DB_SSL_MODE=require`. Cluster: the managed database's host, port, name, user, password and SSL mode — and every host must be in the database's trusted sources. |
| **Redis** | Single: `container`. Cluster: the managed instance's host, port and password, over TLS (`REDIS_SSL`). |
| **Runtime** | `API_WORKERS` and `CELERY_WORKERS` (processes), their memory limits, `LOG_LEVEL`, and `ENABLE_DOCS` (the API docs, behind a login — see [30-OPERATIONS.md](30-OPERATIONS.md#docs-access)). Validation warns when the processes cannot fit in the memory limit. |
| **Continuous deployment** | `DEPLOY_BRANCH`: the branch whose pushes deploy, once CI/CD is on. |
| **Continuous deployment** | `DEPLOY_BRANCH`: the branch whose pushes deploy, once CI/CD is on. `BUILD_BRANCH`, only when images are built on another branch that is then merged into it (see [a release branch](35-CONTINUOUS-DEPLOYMENT.md#a-release-branch)). |
| **Application secrets** | Your application's own keys (API keys, object storage, mail credentials…): one field per key in `project/fields.toml`. They are saved to `config/app.<env>.env`, which appears on the first Save, and written into the app's `.env` on every deploy. Validation names the keys your `.env.example` leaves blank that the deployed `.env` would not set. |

Password fields are never sent back to the page: a saved one shows a *saved* badge, and
Expand Down Expand Up @@ -303,8 +303,8 @@ each row with the button that fixes it. In order:
| Prove it | `deployctl ci deploy --env production` | Redeploys what runs, through GitHub, end to end. |
| Deploy on merge | `deployctl ci auto-deploy on --env production` | Every push to the deploy branch now builds and deploys. |

CI pulls the image with its own short-lived token. `ci.yml`'s `deploy:` job must grant
`packages: read` for that; `ci doctor` checks it. Why each step, and what to do when GitHub
CI pulls the image with its own short-lived token. The job that calls the deploy (`ci.yml`'s
`deploy:`, or `release.yml`'s) must grant `packages: read` for that; `ci doctor` checks it. Why each step, and what to do when GitHub
is the thing that is broken: [35-CONTINUOUS-DEPLOYMENT.md](35-CONTINUOUS-DEPLOYMENT.md).

Once CI/CD is on, the panel opens on **Operate**: what runs and who shipped it, deploying or
Expand Down
40 changes: 38 additions & 2 deletions docs/35-CONTINUOUS-DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ Two conditions, both checked by `ci doctor`:

- **The job that calls the deploy grants `packages: read`.** A called workflow gets no more
than its caller allows, and `ci.yml` is your file, so `ci init` never rewrites it — add
under its `deploy:` job:
under its `deploy:` job (or whichever job of yours calls `deploy.yml`):

```yaml
permissions:
Expand All @@ -112,13 +112,49 @@ GitHub's runners connect from a large, changing set of addresses. Three ways in:
connection goes through it (ssh `ProxyJump`); `ci setup-key` installs the CI key there too
and `ci pin-hosts` pins its host key. The usual shape for a fleet on a private network.

However it gets in, the deploy job makes **one connection per host** and runs every ssh and
rsync of the run through it (`DEPLOYCTL_SSH_MUX=1`, ssh's `ControlMaster`). From a runner each
new handshake costs seconds and an update makes dozens: herbally's deploy step went from
~350s to ~170s. The sockets sit in a directory of the run's own and are closed when the
engine exits; nothing is written to `~/.ssh/config`. A laptop can opt in the same way
(`DEPLOYCTL_SSH_MUX=1 deployctl deploy update …`).

**Checks and build run side by side.** The generated `ci.yml` builds the image while the
checks run, so they cost no time unless they are the slower; the deploy waits for both. A
commit that fails its checks still gets an image — never deployed on a merge, but listed by
`deployctl image tags`, so a hand-picked `ci deploy --tag` should be one whose checks passed.
`ci.yml` is yours: an existing one keeps its order (`needs: [checks]` on `publish` puts it back).

### A release branch

Some projects build on one branch and release from another: `main` collects merged work and
builds an image per commit; merging `main` into `prod` ships. Say so with a build branch:

```bash
deployctl ci init --env production --branch prod --build-branch main
```

```
push to main ci.yml: your checks + build-image.yml ──► <short-sha> (no deploy)
push to prod release.yml: the main commit prod's code matches
└─ waits for that commit's ci.yml run to pass
└─► deploy.yml with its tag (with AUTO_DEPLOY on)
```

`release.yml` is managed like the other two. A prod whose code matches no `main` commit — a
commit made on prod, a conflict resolved in the merge — is refused rather than built: prod
never has an image of its own, so what ships is always what the checks ran against. `ci deploy`
dispatches from `DEPLOY_BRANCH` (prod), and `deploy.yml` still refuses runs from any other
branch. The panel's *Configure → Build branch* is the same setting; leave it empty when one
branch does both.

---

## Day to day

| You want to | Command | Panel (Operate tab) |
|---|---|---|
| Ship code | Merge into the deploy branch. | — |
| Ship code | Merge into the deploy branch (with a build branch: merge it into the deploy branch). | — |
| Change config (a key, a worker count) | edit `config/` → `deployctl ci sync-config` → `deployctl ci deploy --allow-config-change` (redeploys the running tag with the new config) | Configure → Save → *Apply a config change*: Sync config → Deploy the change |
| Ship code that needs a new key | The config change first (row above), then merge. | the same |
| Deploy a specific tag | `deployctl ci deploy --tag T` | *Deploy a version* → Deploy |
Expand Down
19 changes: 14 additions & 5 deletions docs/60-COMMAND-REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ private repository) or `repository` (every plan). A private repository on GitHub
plan this account cannot see, gets `repository`, with a warning that repository secrets are
readable from any branch. `--branch` saves the deploy branch as `DEPLOY_BRANCH`.

### `deployctl ci init --env E [--branch B] [--force]`
### `deployctl ci init --env E [--branch B] [--build-branch B] [--force]`
**Touches: local.** Writes `.github/workflows/`: `build-image.yml` (builds and pushes the
image, and hands its tag on as the `tag` output) and `deploy.yml` (installs this version of
deployctl, pinned, and deploys), both **managed** — regenerated from this version's
Expand All @@ -220,6 +220,12 @@ replaced with `--force`. `ci.yml` is written only when there is none — it is t
own (its checks go there); an existing one is left alone, and if it never calls the two
workflows, the jobs to add are printed. `--branch` saves `DEPLOY_BRANCH` first.

With a **build branch** (`BUILD_BRANCH`, saved by `--build-branch`; `''` clears it), images
are built on that branch and deployed from `DEPLOY_BRANCH`: `build-image.yml` publishes from
the build branch, a new `ci.yml` checks and builds there without deploying, and a third
managed workflow, `release.yml`, runs on a push to `DEPLOY_BRANCH`, finds the build-branch
commit it matches, waits for that commit's CI to pass, and deploys its image.

> The build workflow bakes `IMAGE_REPO` in at generation time. Change `IMAGE_REPO` later and
> the two silently disagree — `validate` checks for this.

Expand Down Expand Up @@ -266,8 +272,9 @@ Existing files with different contents are only replaced with `--force`.

### `deployctl ci doctor --env E [--json] [--repo-level]`
**Touches: GitHub (read).** Checks everything continuous deployment needs — `gh`, the
secrets' scope, the workflows (present, current for this version, `ci.yml` calling
`deploy.yml`), `DEPLOY_SSH_KEY`, `DEPLOY_KNOWN_HOSTS` covering every host and the jump host,
secrets' scope, the workflows (present, current for this version, some workflow calling
`deploy.yml` — `ci.yml`, `release.yml`, or one of the project's — from a job that grants
`packages: read`), `DEPLOY_SSH_KEY`, `DEPLOY_KNOWN_HOSTS` covering every host and the jump host,
the config in GitHub matching this machine's, the install token while deployctl's repository
is private, and whether `AUTO_DEPLOY` is on — and prints the command that fills each gap.
`--json` is what the control panel reads. Exit `0` when nothing is missing, `1` otherwise.
Expand All @@ -283,12 +290,14 @@ migrates. Runs on `DEPLOY_BRANCH`.
### `deployctl ci runs --env E [--limit N] [--json]`
**Touches: GitHub (read).** Recent deploys through GitHub, newest first — manual runs and the
deploys merges triggered (skipped ones, with `AUTO_DEPLOY` off, are left out) — with action,
tag, outcome and a link.
tag, outcome and a link. Merges are read from whichever workflow calls `deploy.yml`, on
`DEPLOY_BRANCH`.

### `deployctl ci auto-deploy <on|off> --env E`
**Touches: GitHub.** Sets the repository variable `AUTO_DEPLOY`: on, every push to
`DEPLOY_BRANCH` that passes its checks deploys itself; off, deploys are `ci deploy`. Always
at repository level — `ci.yml` reads it before any environment is entered.
at repository level — the workflow calling the deploy (`ci.yml`, or `release.yml`) reads it
before any environment is entered.

---

Expand Down
Loading