ci: build from refs

[doc-han]

What this does

The CI workflow that's supposed to test any Lightning branch didn't actually
work. It had specific software versions typed into it by hand, and Lightning
has since moved on to newer ones. Instead of updating those by hand again
(which just breaks the next time Lightning upgrades), the workflow now reads
the required versions directly from whatever Lightning branch it's testing.
It also installs a system package Lightning needs that was missing before.

Separately, if someone runs this harness on a machine with the wrong software
versions installed, it now says so plainly and tells them how to fix it,
instead of failing with a cryptic error partway through.

How to use and test it

bun run typecheck
LIGHTNING=../lightning WORKER=../kit bun run test

Expect ✓ tests/happy-path.spec.ts (1 test), ~35s on a warm checkout.

Preflight:

# checkout's .tool-versions matches your installed toolchain: silent pass
# mismatch: throws, e.g.
#   erlang 28.5 pinned, but `erl` here is 27.3.3
#   Fix: `asdf install` inside <dir>

In CI:

# test on main branch
gh workflow run test-lightning-branch.yml -f lightning_ref=main -f worker=latest

# test on this branch
gh workflow run test-lightning-branch.yml --ref ci-build-from-refs -f lightning_ref=main -f worker=latest

Run #11
(ubuntu-latest): setup-beam resolved Erlang/OTP 28 + Elixir 1.18.4 from
Lightning's .tool-versions, libsodium-dev installed, suite passed —
1 passed (1), 211s cold (no caching yet).

Where to look, in order

1. .github/workflows/test-lightning-branch.yml
   — the spec for this PR. Lightning is checked out into .cache/lightning
   with actions/checkout (not cloned by the harness), and erlef/setup-beam
   / actions/setup-node read its .tool-versions directly
   (version-type: strict, required when reading from a file).
2. src/toolchain.ts — the preflight. Erlang and Elixir
   are hard failures; node is advisory only (a major-version skew — v22 vs a
   v24 pin — works fine in practice, so it only warns).
3. src/ci.ts — one function, stepSummary. A no-op outside
   Actions (checks $GITHUB_STEP_SUMMARY), so it's safe to call from anywhere
   without an environment check at each call site.
4. src/stack.ts + tests/matchers.ts
   — the two places that now call stepSummary: a boot failure (Lightning or
   worker dying/timing out) and a failed run assertion. Both already had a
   formatted message; this just mirrors it to the summary page too.
5. README.md — new "Running in CI" section, and drops the
   now-stale note about needing bootstrap-from-config (Kickstart is on
   Lightning main as of this PR).

[doc-han]

The flow, end to end

The workflow checks out this repo, then checks out Lightning (any ref/repo,
same grammar src/source.ts already used) into .cache/lightning. It reads
that checkout's .tool-versions to install Erlang/Elixir/node, installs
libsodium-dev, then runs bun run test with LIGHTNING pointed at the
checkout path — from there it's the exact same code path as a local
LIGHTNING=../lightning bun run test. Inside that path, up now calls
checkToolchain() before deps.get, and any boot failure or failed-run
assertion additionally writes to $GITHUB_STEP_SUMMARY.

Stops

src/toolchain.ts

readToolVersions parses .tool-versions (asdf's format: <tool> <version>
per line, # comments). checkToolchain then runs erl/elixir from
inside the checkout directory (cwd: dir in probe()) — that's the only
way asdf's shims resolve the checkout's own pins rather than whatever's
active in the harness's own directory. This matters: if this ran from the
harness root instead, it would silently check the wrong checkout's toolchain
against no pins at all in .tool-versions-less setups, or the harness's own
pins if it had any.

Elixir's version string embeds the OTP it was compiled for
(Elixir 1.18.4 (compiled with Erlang/OTP 28)), so the check parses that out
and compares it against the -otp-N suffix in the pin (1.18.4-otp-28) — an
Elixir binary compiled for the wrong OTP major is a real failure mode asdf
lets happen silently otherwise.

Diagnosis story: this file exists because asdf install erlang 28.5 exited
0 while the build had actually failed (a configure error deep in the
build log). The only tell was asdf list erlang not showing 28.5. That's a
fact about asdf/kerl, not about Lightning, but it's why the preflight
checks the result (erl/elixir actually running and reporting the pinned
version) rather than trusting any installer's exit code.

src/ci.ts

Nine lines. The only reason it's its own file rather than inlined at the two
call sites: both stack.ts and tests/matchers.ts need it, and neither
should import from the other.

.github/workflows/test-lightning-branch.yml

Diff-read this one, don't just read the final version — the previous version
looked plausible (postgres service, setup-beam, bun install, run test) but had
never actually completed a run; the three visible past runs in the Actions
tab failed in the "Run contract tests" step, and are from a prior
docker-compose-based version of this workflow (predating boot-from-source),
not this native-process one.

Two things worth checking closely:

• version-type: strict is required by erlef/setup-beam whenever
  version-file is set — omitting it makes the action error out rather than
  silently falling back to a default. Confirmed against the action's own
  README rather than assumed.
• KEEP_STACK=1 is kept from the old workflow so tmp/*.log survive to the
  upload-artifact step; the explicit bun run down at the end replaces
  vitest's own teardown. Same shape as before, just re-verified it's still
  necessary now that the checkout path changed.

[josephjclark]

There's deprecation warning on the action saying that node 20 is deprecated. Can we make sure the node actions are all on latest?

[josephjclark]

There is so much noise from lightning and setup logging in the contract tests bit. The only test stuff runs on line 1487. Is there anything we can do about this? Disable logging? but that might be a problem for debug

[josephjclark]

Is it time to update this workflow name? Or are we still treating this just as a test canary?

[doc-han]

Workflow name updated. Now looks something like below
OpenFn/lightning@main vs worker latest

[josephjclark]

This is very cool! Dynamic action names

We still need to update the static/generic name though because it looks like thisin the UI right now