Conformance triage batch 1: summaries, eshost fix, skip classifications

BRIEFING-cpython-core.md

@@ -0,0 +1,103 @@
+# Briefing: `cpython-core` compat work (CPython 3.12 test suite)
+
+_Self-contained handoff doc. Generated 2026-07-17 from the latest committed run._
+
+## Context
+
+This repo (`elide-dev` conformance testsuite) runs upstream conformance suites
+against pinned [Elide](https://elide.dev) builds and publishes versioned
+reports. `cpython-core` runs CPython 3.12's top-level `Lib/test/test_*`
+modules against Elide's Python runtime (GraalPy-based). Coverage is 438
+modules (see `PYTHON.md` and `manifests/cpython-core.toml`): language/object
+model, containers, numbers, text/codecs, data formats, import machinery, and
+an exploratory full-suite remainder.
+
+The suite is at **95.9%** with a *pure long tail* — 72 failing cases spread
+across 70 distinct root-cause signatures. There is no single big win here;
+this is classification and burn-down work, well suited to parallel sub-agents
+working one test module at a time.
+
+## Current state (Elide `1.4.1+20260716.b8b6531`, digest `3d3ea83ed640`)
+
+| pass | fail | error | skip | total | pass rate (excl. skips) |
+|---:|---:|---:|---:|---:|---:|
+| 1,690 | 57 | 15 | 103 | 1,865 | **95.9%** |
+
+Counts are unittest cases/classes as reported by the CPython runner. Latest
+reports:
+
+- `reports/1.4.1+20260716.b8b6531/3d3ea83ed640/cpython-core/impact.md`
+- `expectations/cpython-core.ratchet.toml` — known-failure baseline
+
+## How to run
+
+```bash
+bun run setup                       # once: deps + submodules
+# smoke:
+bun run testsuite --elide nightly --suite cpython-core --include 'test_json' --log
+# one module under investigation:
+bun run testsuite --elide nightly --suite cpython-core --include 'test_builtin' --log
+# full suite (~90s):
+bun run testsuite --elide nightly --suite cpython-core
+# after fixes land in an Elide nightly, re-baseline:
+bun run testsuite --elide nightly --suite cpython-core --ratchet
+```
+
+## Failure analysis (72 failing cases)
+
+Rollup by test module (from impact.json example ids):
+
+| failing | module | flavor of failures |
+|---:|---|---|
+| 17 | `test_builtin` | exec/eval builtins mapping, iterator misbehavior edge cases, shutdown cleanup, subprocess-based cases |
+| 6 | `test_descr` | descriptor protocol edges (e.g. Carlo Verre hackcheck multi-inherit) |
+| 6 | `test_dict` | dict internals/ordering edge cases |
+| 5 | `test_ast` | `ast.AST._fields` NULL-check / attribute deletion behavior |
+| 5 | `test_re` | regex engine divergences |
+| 4 | `test_exceptions` | exception context/`__context__` details (e.g. test_3114) |
+| 4 | `test_weakref` | finalize-at-shutdown, `test_atexit` (subprocess-based) |
+| 4 | `test_fstring` | f-string parse/format edges |
+| 4 | `test_list` | MemoryError on huge ranges, free-after-iterating |
+| 3 each | `test_tuple`, `test_class` | attribute-error message formats, free-after-iterating |
+| ~11 | singletons | `test_json.test_tool` (subprocess), `test_math`, `test_extcall`, `test_genexps`, `test_functools`, `test_patma`, … |
+
+Recurring cross-cutting themes worth knowing before diving in:
+
+1. **Subprocess-dependent cases** (`script_helper.assert_python_ok`,
+   `test_json.test_tool` piping, `test_weakref.test_atexit`): blocked on
+   Elide's `child_process`/subprocess support — the same gap tracked in the
+   `node-api` briefing. These are *platform limitations* today; consider
+   classifying them as skips with a comment instead of failures.
+2. **Attribute-error message format tests** (`test_class`, `test_descr`):
+   CPython 3.12 asserts on suggestion-rich error messages ("Did you mean:
+   ..."). Cheap wins if Elide's error messages can match; otherwise skip-
+   classify as implementation detail.
+3. **GC/refcount-semantics tests** (`free_after_iterating`, weakref
+   finalization order, MemoryError on `list(range(2**63))`): often
+   fundamentally different on a JVM runtime. Classify rather than chase.
+4. **Genuine runtime defects** (`test_re`, `test_fstring`, `test_ast`,
+   `test_builtin` non-subprocess cases): real fixes, one module at a time.
+
+## Recommended workflow (parallelizable)
+
+This suite is ideal for a fan-out: one agent per test module, each tasked to
+
+1. run `--include '<module>' --log` and reproduce,
+2. classify each failure: **(a)** Elide defect / **(b)** platform limitation
+   (subprocess, GC semantics, native details) / **(c)** harness gap,
+3. for (a): produce a minimal Python repro + pointer into the Elide runtime;
+   for (b): propose a `[skip]` entry with a one-line justification;
+   for (c): file against this repo.
+
+Start with `test_builtin` (17), `test_descr`/`test_dict` (12), then
+`test_ast`/`test_re` (10).
+
+## Notes for the implementer
+
+- Runtime fixes happen in the Elide repo (GraalPy-based Python engine); this
+  repo pins nightlies and measures.
+- 103 skips are already classified in `expectations/cpython-core.toml` and
+  excluded from the denominator; keep skip entries commented.
+- The pass rate is already 95.9% — the value here is *classification honesty*
+  (separating defects from platform limits) more than raw number movement.
+- After a fix lands: full run → check `changes.md` → `--ratchet` → commit.

BRIEFING-javac-jtreg.md

@@ -0,0 +1,93 @@
+# Briefing: `javac-jtreg` compat work (OpenJDK langtools javac suite)
+
+_Self-contained handoff doc. Generated 2026-07-17 from the latest committed run._
+
+## Context
+
+This repo (`elide-dev` conformance testsuite) runs upstream conformance suites
+against pinned [Elide](https://elide.dev) builds and publishes versioned
+reports. `javac-jtreg` runs the full OpenJDK langtools `tools/javac` tree
+(5,554 jtreg `.java` test files, 100% coverage — see `JAVAC.md` and
+`manifests/javac-langtools.toml`) through jtreg, compiling with **Elide's
+embedded javac** and running generated programs with a regular JDK.
+
+The suite is at **98.7%** with only 47 failures. This is precision work:
+most failures sit in one opaque bucket that needs classification before
+fixing, so step one is diagnosis, not patching.
+
+## Current state (Elide `1.4.1+20260716.b8b6531`, digest `3d3ea83ed640`)
+
+| pass | fail | error | skip | total | pass rate (excl. skips) |
+|---:|---:|---:|---:|---:|---:|
+| 3,655 | 45 | 2 | 232 | 3,934 | **98.7%** |
+
+Counts are jtreg test executions. Latest reports:
+
+- `reports/1.4.1+20260716.b8b6531/3d3ea83ed640/javac-jtreg/impact.md`
+- `expectations/javac-jtreg.ratchet.toml` — known-failure baseline
+- Note: `tools/javac/processing` failures are muted/skipped per earlier triage
+  (annotation processing), and 232 skips are classified in
+  `expectations/javac-jtreg.toml`.
+
+## How to run
+
+```bash
+bun run setup                       # once: deps + submodules + sparse checkout
+# smoke:
+bun run testsuite --elide nightly --suite javac-jtreg --include 'tools/javac/IllDefinedOrderOfInit.java' --log
+# one failing test with full output:
+bun run testsuite --elide nightly --suite javac-jtreg --include 'tools/javac/T6411379.java' --log --show-failure-output
+# full suite (~12 min):
+bun run testsuite --elide nightly --suite javac-jtreg
+# after fixes land in an Elide nightly, re-baseline:
+bun run testsuite --elide nightly --suite javac-jtreg --ratchet
+```
+
+## Failure analysis (47 failing executions, 11 signatures)
+
+| count | signature | examples |
+|---:|---|---|
+| 36 | `Compilation failed: Compilation failed` (opaque — needs classification) | `tools/javac/MethodParameters/ClassReaderTest/ClassReaderTest.java`, `T6411379`, `T6423583`, `T6458749`, `T6855236` |
+| 2 | test timed out | `tools/javac/file/zip/T6836682.java` |
+| 6 | output does not match reference `.out` file | `T8181464/LambdaInAnnotationsCausesNPETest{1,2,3}`, `T8171332/Processor`, `7129225/TestImportStar`, `annotations/neg/8171322/TypeVariableAsAnnotationTest`, `typeAnnotations/failures/TypeVariableMissingTA` |
+| 1 | wrong compiler output line count | `tools/javac/newlines/NewLineTest.java` |
+| 1 | toolbox javac task rc != 0 | `tools/javac/platform/PlatformProviderTest.java` |
+| 1 | AssertionError in test main | (see impact.md) |
+
+Reading of the buckets:
+
+1. **The 36-test "Compilation failed" bucket is heterogeneous** — the summary
+   line carries no diagnostic. First task: re-run each with
+   `--show-failure-output --log` and bucket the actual javac diagnostics.
+   Expect a mix of: missing `-parameters`/ClassReader metadata handling
+   (`MethodParameters/*`), old regression tests exercising unusual compiler
+   options, and genuine Elide-javac divergences from stock javac.
+2. **Reference-file diffs (6)** are usually *diagnostic formatting*
+   divergences (error message wording/position), clustered around annotation
+   processing and lambda-in-annotation error recovery. Compare Elide's output
+   with the `.out` golden files; these can be quick fixes if the embedded
+   javac is just a version step behind the pinned OpenJDK checkout.
+3. **Timeouts (2)** are zip-file-handling tests (`file/zip/T6836682`) — check
+   whether they hang in Elide's filesystem layer or just exceed the limit.
+
+## Recommended workflow
+
+1. **Classification pass first** (an agent can do this mechanically): for each
+   of the 36 opaque failures, run with `--show-failure-output`, capture the
+   real diagnostic, and produce a table: test → actual error → suspected
+   cause (Elide javac defect / test needs specific JDK feature / harness gap).
+2. Group the classified failures and hand coherent groups to the compiler
+   team: `MethodParameters/*` is likely one fix; annotation-related reference
+   diffs likely another.
+3. Skip-classify anything that depends on javac internals Elide intentionally
+   diverges from (document in `expectations/javac-jtreg.toml`).
+
+## Notes for the implementer
+
+- Compiler fixes happen in Elide's embedded javac (Elide repo); this repo
+  pins nightlies and measures.
+- At 98.7% with 47 failures, expectations-honesty matters more than raw rate:
+  every remaining failure should end up either fixed or annotated-skip with a
+  reason.
+- After a fix lands: full run → check `changes.md` (new passes, no
+  regressions) → `--ratchet` → commit reports + expectations together.

BRIEFING-node-api.md

@@ -0,0 +1,129 @@
+# Briefing: `node-api` compat work (Node.js core test suite)
+
+_Self-contained handoff doc. Generated 2026-07-17 from the latest committed run._
+
+## Context
+
+This repo (`elide-dev` conformance testsuite) runs upstream conformance suites
+against pinned [Elide](https://elide.dev) builds and publishes versioned
+reports. The `node-api` suite runs a sparse slice of Node.js core's
+`test/parallel/` JS tests (assert, async_hooks, buffer, console, dns, events,
+fs, module/require, path, process, stream, timers, vm, worker groups — see
+`manifests/node-api.toml`). Tests run against a checked-in `common/` overlay
+(`node-api-overlay`), so `require('../common')` works without full Node
+internals.
+
+This is the lowest-scoring suite (25.4%) and was, until today, hidden from the
+README summary table. It is now visible; the goal is to make the number honest
+and then move it.
+
+## Current state (Elide `1.4.1+20260716.b8b6531`, digest `3d3ea83ed640`)
+
+| pass | fail | error | skip | total | pass rate (excl. skips) |
+|---:|---:|---:|---:|---:|---:|
+| 309 | 875 | 34 | 345 | 1,563 | **25.4%** |
+
+Counts are whole test *files* (one Node test file = one result). Latest reports:
+
+- `reports/1.4.1+20260716.b8b6531/3d3ea83ed640/node-api/impact.md` — 481
+  root-cause signatures, largest first (4,800 lines; read the top 100)
+- `.../node-api/impact.json` — machine-readable (`bySignature`)
+- `expectations/node-api.ratchet.toml` — known-failure baseline
+
+## How to run
+
+```bash
+bun run setup                       # once: deps + submodules + sparse checkouts
+# smoke:
+bun run testsuite --elide nightly --suite node-api --include 'test/parallel/test-path.js' --log
+# a target group:
+bun run testsuite --elide nightly --suite node-api --include 'test/parallel/test-buffer*.js' --log
+# full suite (~6 min):
+bun run testsuite --elide nightly --suite node-api
+# after fixes land in an Elide nightly, re-baseline:
+bun run testsuite --elide nightly --suite node-api --ratchet
+```
+
+## Failure analysis (909 failing files)
+
+### Systemic root causes (fix once, unblock many)
+
+1. **`__filename` / `process.argv[1]` is the literal string `"unknown"`** —
+   **~49 files directly**, likely more indirectly. Symptoms across several
+   signatures:
+   - `The worker script or module filename ... Received "unknown"` (31+2+1+1)
+   - `ENOENT: no such file or directory, open 'unknown'` (6+3+1+1) — tests
+     that re-open `__filename`
+   Elide isn't propagating the entry-script path to `process.argv`/module
+   metadata when launched by the harness. Almost certainly ONE runtime fix.
+   **Best effort/reward ratio in the entire cross-suite triage.**
+2. **`node:child_process` unimplemented** — **~39 files direct**:
+   `spawnSync()` (14), `spawn()` (10+3), `exec()` (5), `fork()` (4),
+   `execFileSync()` (3). Also implicated in some of the 34 timeouts and in
+   cpython-core failures (subprocess-based tests) — cross-suite leverage.
+   High complexity; treat as a project, not a quick win.
+3. **`node:http` (+ `_http_common`) missing** — 64 files fail at
+   `Cannot load module: 'http'`. Very large surface; park unless http is
+   already on Elide's roadmap. Alternatively add these to `[skip]` with a
+   comment to make the pass rate reflect reachable tests.
+4. **Timeouts** — 34 files (`Node API test timed out`), e.g.
+   `test-fs-promises-watch.js`. Mostly fs.watch / event-loop-keepalive
+   related; diagnose a couple before assuming one cause.
+
+### Module-level clusters (each a coherent mini-project)
+
+- **buffer** (~80 files): assorted semantic gaps — `buffer.constants`
+  undefined, `Buffer.from` on SharedArrayBuffer/objects with `valueOf`
+  (`unsupported input type`, 3), `buf.copy` return value undefined (8),
+  inspect truncation format (20-bucket), negative-allocation / range checks
+  missing (`Missing expected exception`, part of 36+7+4).
+- **events / EventEmitter** (~25): symbol event names
+  (`Cannot convert a Symbol value to a string`, 4), `receiver is not an
+  EventEmitter` when using prototype-less receivers (4),
+  `listeners side-effects` (`Cannot convert undefined or null to object`, 3),
+  listener-count semantics.
+- **streams** (~90): `Readable.toWeb` missing (4), `Stream is not a
+  constructor` from `require('stream')` shape (3), many
+  `mustCall` count mismatches (13+10+9+6...) pointing at async scheduling /
+  destroy/finish event-ordering divergences.
+- **process** (~40): `process.chdir()` (5) and `process.getuid()` (3) throw
+  `UnsupportedOperationException`, `execve` missing (3+2),
+  `hasUncaughtExceptionCaptureCallback` missing (3),
+  `getactiveresources` tracking (2+2+...), env delete semantics.
+- **vm** (~15): internal `Symbol(elide.vm.context)` leaks into
+  `Object.getOwnPropertyNames`/`ownKeys` of contextified objects (3 —
+  trivial fix: filter the internal symbol), `defineProperty called on
+  non-object` on context args (2), property forwarding.
+- **timers** (~10): `timeout.refresh()` missing (3), timer cancellation
+  warnings (3), active-resource tracking.
+- **worker_threads** (~10 beyond the argv issue): structured-clone
+  `DOMException` on SharedArrayBuffer/MessagePort transfer (4).
+
+## Ranked work items
+
+1. **argv/`__filename` "unknown" fix** — ~49+ files, complexity **low**. Do
+   this first; re-run to see the true shape of worker/fs failures behind it.
+2. **vm internal-symbol filter** — 3 files, complexity **trivial**. Good
+   warm-up in the same session as (1).
+3. **buffer semantics sweep** — ~40–80 files, complexity **medium**, highly
+   parallelizable (each signature is independent). Run
+   `--include 'test/parallel/test-buffer*.js' --log` and burn down.
+4. **process misc (`chdir`, `getuid`, capture-callback, refresh)** — ~15
+   files, complexity **low-medium** each.
+5. **events/EventEmitter conformance** — ~25 files, complexity **medium**.
+6. **child_process `spawn`/`spawnSync`** — ~39 files + cross-suite
+   (cpython-core) leverage, complexity **high**. Schedule as its own project.
+7. **streams event-ordering** — large but diffuse; recommend AFTER (1) and
+   (3), since many stream tests also touch buffer/process gaps.
+8. **`node:http`** — 64 files; park or skip-classify for now.
+
+## Notes for the implementer
+
+- Fixes happen in the Elide runtime repo; this repo pins nightlies and
+  measures (`--elide nightly` picks up a fresh build).
+- One file = one result here, so "N files" = N report-visible wins.
+- The `common/` overlay lives in the harness work dir
+  (`.harness/work/node-api/node-api-overlay/test/common/index.js`); `mustCall`
+  mismatch stack traces point there, not at a runtime bug in itself.
+- After a fix lands: full run → check `changes.md` (new passes, no
+  regressions) → `--ratchet` → commit reports + expectations together.