Skip to content

chore: update runtime compatibility matrix #62

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
github-actions wants to merge 1 commit into
base: main
Choose a base branch
from
Open

chore: update runtime compatibility matrix #62

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 55 additions & 31 deletions packages/std/__tests__/_matrix/results.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@
"label": "Node.js 20",
"version": "20.20.2",
"status": "pass",
"passed": 699,
"passed": 928,
"failed": 0,
"total": 699,
"total": 928,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
Expand All @@ -33,9 +33,9 @@
"label": "Node.js 22",
"version": "22.23.1",
"status": "pass",
"passed": 699,
"passed": 928,
"failed": 0,
"total": 699,
"total": 928,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
Expand All @@ -58,9 +58,34 @@
"label": "Node.js 24",
"version": "24.18.0",
"status": "pass",
"passed": 699,
"passed": 928,
"failed": 0,
"total": 699,
"total": 928,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
"Intl.NumberFormat": true,
"crypto.getRandomValues": true,
"crypto.randomUUID": true,
"crypto.subtle": true,
"Blob": true,
"Blob.toStringTag": true,
"structuredClone": true,
"TextEncoder": true,
"BigInt": true,
"WeakRef": true,
"Proxy": true
},
"failures": []
},
{
"id": "node26",
"label": "Node.js 26",
"version": "26.5.0",
"status": "pass",
"passed": 928,
"failed": 0,
"total": 928,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
Expand All @@ -83,9 +108,9 @@
"label": "Bun",
"version": "1.3.14",
"status": "pass",
"passed": 699,
"passed": 928,
"failed": 0,
"total": 699,
"total": 928,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
Expand All @@ -108,9 +133,9 @@
"label": "Deno",
"version": "2.9.4",
"status": "pass",
"passed": 699,
"passed": 928,
"failed": 0,
"total": 699,
"total": 928,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
Expand All @@ -131,46 +156,37 @@
{
"id": "esrun",
"label": "ES-Runtime",
"version": "0.11.0",
"status": "partial",
"passed": 698,
"failed": 1,
"total": 699,
"percent": 99.9,
"version": "0.12.0",
"status": "pass",
"passed": 928,
"failed": 0,
"total": 928,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
"Intl.NumberFormat": true,
"crypto.getRandomValues": true,
"crypto.randomUUID": true,
"crypto.subtle": true,
"Blob": true,
"Blob.toStringTag": false,
"Blob.toStringTag": true,
"structuredClone": true,
"TextEncoder": true,
"BigInt": true,
"WeakRef": true,
"Proxy": true
},
"failures": [
{
"title": "Types > isBlob > valid cases",
"message": "expected false to be true // Object.is equality",
"category": "runtime-deviation",
"reason": "Runtime `Blob` is missing `Symbol.toStringTag`, so `Object.prototype.toString.call(blob)` returns `[object Object]`. This deviates from the File API spec; the fix belongs in the runtime.",
"requires": "Blob.toStringTag",
"affects": ["isBlob"]
}
]
"failures": []
},
{
"id": "llrt",
"label": "LLRT",
"version": "0.8.0-beta",
"status": "partial",
"passed": 683,
"failed": 16,
"total": 699,
"percent": 97.7,
"passed": 911,
"failed": 17,
"total": 928,
"percent": 98.2,
"capabilities": {
"Intl.Segmenter": false,
"Intl.NumberFormat": false,
Expand Down Expand Up @@ -210,6 +226,14 @@
"requires": "Intl.NumberFormat",
"affects": ["formatCompact", "formatCurrency"]
},
{
"title": "streamToText > splits every byte into its own chunk",
"message": "expected '\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\…' to be '🌍🔐é' // Object.is equality",
"category": "runtime-deviation",
"reason": "Runtime `TextDecoder` ignores the `stream: true` option, so the trailing bytes of a multi-byte character split across two chunks are decoded as replacement characters instead of being held back for the next call. Required by the Encoding spec; the fix belongs in the runtime. Text arriving in whole characters per chunk is unaffected.",
"requires": "TextDecoder stream: true",
"affects": ["streamToText", "streamToLines"]
},
{
"title": "String > stringReverse > basic reversal",
"message": "Intl.Segmenter is not supported in this environment. Use a polyfill or a modern runtime.",
Expand Down
31 changes: 15 additions & 16 deletions website/app/docs/env-support/page.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,25 +19,18 @@ Where a runtime falls short, the cause is listed so you can tell the difference

{/* MATRIX:START */}

_Measured on 2026-07-28 against `@opentf/std@0.14.1`, running the full suite of 699 tests on each runtime._
_Measured on 2026-07-28 against `@opentf/std@0.14.1`, running the full suite of 928 tests on each runtime._

| Runtime | Version | Tests passing | Status |
| --- | --- | --- | --- |
| Node.js 20 | `20.20.2` | 699 / 699 (100%) | ✅ Fully supported |
| Node.js 22 | `22.23.1` | 699 / 699 (100%) | ✅ Fully supported |
| Node.js 24 | `24.18.0` | 699 / 699 (100%) | ✅ Fully supported |
| Bun | `1.3.14` | 699 / 699 (100%) | ✅ Fully supported |
| Deno | `2.9.4` | 699 / 699 (100%) | ✅ Fully supported |
| ES-Runtime | `0.11.0` | 698 / 699 (99.9%) | ⚠️ Partial |
| LLRT | `0.8.0-beta` | 683 / 699 (97.7%) | ⚠️ Partial |

### ES-Runtime 0.11.0

**Runtime spec deviation — `Blob.toStringTag` unavailable** (1 test)

Runtime `Blob` is missing `Symbol.toStringTag`, so `Object.prototype.toString.call(blob)` returns `[object Object]`. This deviates from the File API spec; the fix belongs in the runtime.

Affected utilities: `isBlob`. Everything else works as expected.
| Node.js 20 | `20.20.2` | 928 / 928 (100%) | ✅ Fully supported |
| Node.js 22 | `22.23.1` | 928 / 928 (100%) | ✅ Fully supported |
| Node.js 24 | `24.18.0` | 928 / 928 (100%) | ✅ Fully supported |
| Node.js 26 | `26.5.0` | 928 / 928 (100%) | ✅ Fully supported |
| Bun | `1.3.14` | 928 / 928 (100%) | ✅ Fully supported |
| Deno | `2.9.4` | 928 / 928 (100%) | ✅ Fully supported |
| ES-Runtime | `0.12.0` | 928 / 928 (100%) | ✅ Fully supported |
| LLRT | `0.8.0-beta` | 911 / 928 (98.2%) | ⚠️ Partial |

### LLRT 0.8.0-beta

Expand All @@ -53,6 +46,12 @@ Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot f

Affected utilities: `formatCompact`, `formatCurrency`. Everything else works as expected.

**Runtime spec deviation — `TextDecoder stream: true` unavailable** (1 test)

Runtime `TextDecoder` ignores the `stream: true` option, so the trailing bytes of a multi-byte character split across two chunks are decoded as replacement characters instead of being held back for the next call. Required by the Encoding spec; the fix belongs in the runtime. Text arriving in whole characters per chunk is unaffected.

Affected utilities: `streamToText`, `streamToLines`. Everything else works as expected.

**Missing platform API — `Intl.Segmenter` unavailable** (13 tests)

Grapheme segmentation needs `Intl.Segmenter`. Runtimes built without full ICU cannot count or reverse multi-codepoint characters correctly.
Expand Down