Skip to content

feat(ts): ts_npm_module and strict, reproducible dependency resolution - #62

Merged
walterjgsp merged 4 commits into
tsfrom
feat/ts-npm-deno-cache
Oct 4, 2026
Merged

walterjgsp merged 4 commits into
tsfrom
feat/ts-npm-deno-cache

Conversation

@walterjgsp

@walterjgsp walterjgsp commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Work for #61 (design and spike results are in the issue comments). Two parts that share one resolver:

1. ts_npm_module: npm packages through Deno's own npm: resolution

CommonJS packages, subpath imports and packages with only an exports map work without a node_modules directory, without a lockfile, and without reimplementing resolution in Go.

  • The tarball is pinned by sha256 in the BUILD file. The build extracts it into a slice of a Deno npm cache (npm/registry.npmjs.org/<pkg>/<version>/ plus a minimal registry.json).
  • The import map sends name and name/ to npm: specifiers; deno check and deno test merge the slices into the per-run DENO_DIR and run with --cached-only, so a module missing from deps fails at once (npm package not found in cache) instead of being downloaded.
  • Dependencies resolve automatically (resolve_transitive, on by default, like ts_module): list only the package you import. Several versions of a package are kept side by side when dependents need different ones (Deno resolves per dependent). resolve_transitive = False is the fully pinned mode: every dependency its own ts_npm_module with its own hash, no network, and the build names any that is missing. Declared dependencies are never resolved again, so the modes mix.

2. A real resolver, and a fix for ts_module's automatic resolution

New semver package (ranges incl. ||, hyphen and x-ranges, prereleases) and registry client (strict resolution, integrity-verified downloads), used by both rules. Reading the old ts_module code showed it:

  • fell back to the latest version when no version satisfied a range;
  • verified no download against the registry's integrity data;
  • kept the first version of a package silently when a second dependent needed an incompatible one;
  • only warned when resolution failed, leaving an incomplete tree;
  • understood only ^, ~ and >=.

All of these are now errors (unresolvable peer dependencies stay warnings), dependencies are read from the verified tarball's package.json instead of registry metadata, and conflicts name both dependents and point at ts_npm_module. This can make builds fail that used to pass with an incomplete or inconsistent tree. It is in the changelog under Fixed.

Reproducible without a lockfile or a date

Only dependency versions published by the end of the day the root version was published are considered (for both rules), so the version you pin by hash fixes the result and nothing is configured. If the registry does not know the root's publish time (private registry, tarball from another URL) the build warns and does not pin. (An earlier version of this PR had a resolve_as_of parameter; it was removed because the default works without it.)

Test plan

  • plz test //...: 255 tests in 28 targets. New unit tests for semver (spec ordering, ~140 range cases), the registry client (resolution, dist-tags, deprecated, unsupported specifiers, integrity), slice building, the cache merge and import map, and both resolution paths against a mock registry that serves real tarballs (strictness, integrity mismatch, wrong-name tarball, conflicts, compatible sharing, peers, optional deps, staged-slice precedence, side-by-side versions, the automatic date pin)
  • Fixtures built from real tarballs and the real registry, with automatic resolution and with every dependency pinned: debug + ms (CommonJS), highlight.js (CommonJS, highlight.js/lib/core subpaths), @codemirror/legacy-modes (no main, only exports, 11 dependencies resolved automatically with none declared). No Download lines at test time
  • A missing slice fails fast (npm package not found in cache); checked by hand

Not in this PR (why it is a draft)

  • ts_bundle / ts_binary (esbuild cannot resolve npm:; options are deno bundle, which is experimental, or pointing esbuild at the slice directories), the Vitest and browser runners.
  • A config flag and migration notes for ts_module users beyond the changelog; removing the old npm import-map path.
  • The cache layout (registry.json with _deno.packumentFormat) is internal to Deno, so it is tied to the pinned Deno version. Only linux-x64 was exercised.
  • The rule must not force Please's sandbox: it failed on the CI runner (fopen /proc/self/setgroups: Permission denied) while passing locally. Automatic resolution needs the network, so those targets run unsandboxed; the pinned mode follows [sandbox].

Refs #61

🤖 Generated with Claude Code

walterjgsp and others added 3 commits October 4, 2026 08:41
…on (prototype)

Provide an npm package to Deno's npm: resolution, offline: a sandboxed build step extracts the sha256-pinned tarball into a slice of a Deno npm cache (registry.json + extracted package), bundling the slices of its dependencies and failing if one is not declared. Targets merge the slices into their DENO_DIR, the import map sends the package name to an npm: specifier, and deno check / deno test run with --cached-only so a module missing from deps fails instead of being downloaded. No node_modules and no lockfile. Works for CommonJS packages, subpath imports and exports-only ESM packages (fixtures: debug, highlight.js, @codemirror/legacy-modes). Not wired yet: ts_bundle, ts_binary, Vitest and browser runners, a helper that prints the declarations.

Refs #61

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The build step needs no network access, and forcing the sandbox fails on hosts without user namespaces (the CI runner: fopen /proc/self/setgroups: Permission denied). Follow Please's [sandbox] setting like the other rules and say so in the docs.

Refs #61

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ucibly

Add a semver package (ranges, prereleases, x- and hyphen ranges, ||) and an npm registry client (strict resolution, integrity-verified downloads), and rebuild dependency resolution on them for both ts_module and ts_npm_module. Resolution no longer falls back to latest, verifies every download against the registry's integrity data, reads dependencies from the verified tarball, and fails on conflicts and unresolvable dependencies instead of warning. ts_npm_module resolves dependencies by default (resolve_transitive, as ts_module) and keeps several versions side by side; resolve_transitive = False keeps the fully pinned mode. Resolution is reproducible without a lockfile or a date: only versions published by the end of the day the root version was published are considered.

Refs #61

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@walterjgsp walterjgsp changed the title feat(ts): ts_npm_module, npm packages through Deno's own resolution (prototype) feat(ts): ts_npm_module and strict, reproducible dependency resolution Oct 4, 2026
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@walterjgsp
walterjgsp marked this pull request as ready for review October 4, 2026 14:15
@walterjgsp
walterjgsp merged commit ab39064 into ts Oct 4, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant