feat(tests): add per-test runtime pinning to the integration matrix

[jsteinich]

Related issue

Implements the mechanism from #337. First step toward OpenTofu coverage.

Description

A test can now opt out of the default tested Terraform cross-product and declare the runtimes it needs, via pinnedRuntimes in tools/build-test-matrix.mjs:

const pinnedRuntimes = {
  "typescript/provider-features/test.ts": [
    { product: "terraform", version: "1.16.1" },
    { product: "opentofu", version: "1.12.6" },
  ],
};

Why this first. Until now the only way to run anything against OpenTofu was a full matrix leg across all 60+ targets, which needs #208, #301, #392 and #393 all resolved before a single tofu job can be green. A pin runs one target against one CLI, so each of those blockers can now be fixed and verified independently and tofu coverage grows a test at a time. It also gives an individual test a newer Terraform than tested without moving the floor for everything — which is what #312 needs.

To be clear about what this does and does not do: it does not fix any of the four blockers. It makes them separately fixable.

Mechanism

Matrix entries gain a binary field — the version-suffixed binary the CI image installs — and the workflow sets TERRAFORM_BINARY_NAME from it rather than string-concatenating "terraform" with a version. terraform stays the bare version, so TERRAFORM_VERSION and the cache-key semantics are unchanged for existing entries.

The plugin cache is now keyed by binary rather than version. OpenTofu resolves providers from registry.opentofu.org into the same TF_PLUGIN_CACHE_DIR, so a tofu run must not share a cache entry with a terraform run.

Pins are validated against .terraform.versions.json when the matrix is built, so a typo fails the prepare-integration-tests job with a clear message rather than a "binary not found" deep inside a test run.

First consumer

test/typescript/provider-features, pinned to Terraform 1.16.1 and OpenTofu 1.12.6 — the first OpenTofu job in this repo's CI. It covers the provider-defined functions binding end to end: provider.functions.rfc3339Parse(...) must synthesize to a provider::time::rfc3339_parse(...) expression rather than resolving at synth time. That needs a CLI that emits functions in provider schemas (Terraform/OpenTofu >= 1.8), so it could not have run at the old 1.6.5 ceiling at all.

It uses hashicorp/time, which is mirrored on registry.opentofu.org and installs cleanly under both CLIs — deliberately not kreuzwerker/docker, which hits the signing problem in #301.

Testing

The matrix builder cannot run natively on Windows (pre-existing: execFileSync("npx") needs .cmd, and the testDir prefix strip assumes forward slashes), so I verified the pin logic against a stubbed target list:

entries: 8
  typescript/provider-features/test.ts   tf=1.16.1  binary=terraform1.16.1  hcl=false
  typescript/provider-features/test.ts   tf=1.12.6  binary=tofu1.12.6       hcl=false
  typescript/synth-app/test.ts           tf=1.5.7   binary=terraform1.5.7   hcl=false
  typescript/synth-app/test.ts           tf=1.16.1  binary=terraform1.16.1  hcl=false
  typescript/iterators/test.ts           tf=1.5.7   binary=terraform1.5.7   hcl=false
  typescript/iterators/test.ts           tf=1.5.7   binary=terraform1.5.7   hcl=true
  typescript/iterators/test.ts           tf=1.16.1  binary=terraform1.16.1  hcl=false
  typescript/iterators/test.ts           tf=1.16.1  binary=terraform1.16.1  hcl=true

The pinned target gets exactly its two entries and no default cross-product; non-pinned targets are unchanged, including the HCL doubling. Pin validation confirmed to fail fast:

Error: pinnedRuntimes["typescript/provider-features/test.ts"] pins opentofu 9.9.9,
which is not in .terraform.versions.json opentofu.available, so the CI image has
no tofu9.9.9 binary.

I could not run the new integration test locally — it needs pnpm package and the integration harness, and pipenv/rsync are unavailable on this machine. This PR's own CI run is the verification for the test itself, and it is the interesting part: the tofu job is the first of its kind here.

Known gap

The disabled windows_integration job consumes the same matrix and would try to install Terraform for a tofu-pinned entry. Noted in place rather than building OpenTofu support into a job guarded by if: false.

Follow-ups this unblocks

• Integration test: ephemeral resources end-to-end, gated on Terraform >= 1.10 #312 — ephemeral end-to-end can pin rather than wait on the default matrix.
• test(tofu): terraform-cloud test feeds the matrix version into an HCP Terraform workspace, blocking an OpenTofu matrix leg #392, provider-generator: generated docs links use the fetching CLI's registry hostname, producing 404s under OpenTofu #393, dependency-manager: migrate to opentofu registry #208, test(tofu): Track OpenTofu provider signing support in matcher tests #301 — each now fixable and verifiable on its own pinned target.

🤖 Generated with Claude Code