Skip to content

ts: use CommonJS and entry-less npm packages (bundle an npm package into a ts_library) #61

Closed
Closed
ts: use CommonJS and entry-less npm packages (bundle an npm package into a ts_library)#61
Labels
enhancementNew feature or request

Description

@walterjgsp
walterjgsp
opened on Oct 4, 2026

Summary

Some npm packages cannot be used from a ts_library, ts_test or ts_bundle today: CommonJS packages, and pure-ESM packages that have no main / module entry. There is a pattern that makes them usable without changing the plugin (bundle the package with ts_bundle, wrap the bundle in a ts_library), shown below. This issue records the problem, what failed, the working pattern and its rough edges, and proposes making it automatic.

Why packages fail

ts_module unpacks the tarball and records a single entry (from module, main or an index.* file). The import map then:

  • maps pkg to that entry and pkg/ to the entry's directory;
  • ignores the exports field of package.json (it is parsed into PackageJSON.Exports and never used);
  • gives esbuild (ts_bundle) the same entries as --alias:pkg=<entry file>, and esbuild appends a subpath to the alias, so pkg/sub.js becomes <entry file>/sub.js.

What was tried (all with highlight.js 11.12.0 and similar)

Attempt Result
import hljs from "highlight.js/lib/core" resolves to .../highlightjs/lib/lib/core: subpaths are relative to the entry directory (lib/)
import "highlight.js/core.js" (with the extension) file found, but the package is CommonJS (module.exports = x): TS1192 / TS2497 "no default export" / "can only be referenced with ECMAScript imports"
the same with // @ts-ignore on the imports down to TS2307 on the extensionless require("./html_renderer") inside the package
plus flags = ["--unstable-sloppy-imports"] on the ts_library (this option already exists) extensionless requires resolve, then TS2303 Circular definition of import alias 'hljs': the package's own lib/core.d.ts does import hljs from "highlight.js", which the import map points back at the same package
@highlightjs/cdn-assets (ESM builds) no main / module, so no entry: Import "..." not a dependency and not in import map
@codemirror/legacy-modes (pure ESM, exports: {"./mode/*": ...}) same: no entry, and exports is ignored
ts_bundle directly on import "highlight.js/core.js" esbuild: Could not resolve "./third_party/ts/highlightjs/lib/index.js/core.js" (the alias-appends-subpath behaviour above)

What works today (no plugin change)

Two targets: bundle the package with esbuild (which understands CommonJS), then wrap the bundle in a ts_library.

third_party/ts/highlight/entry.ts (imports the unpacked module by relative path, which esbuild resolves because the module is laid out next to the entry in the bundle's working directory):

import core from "../highlightjs/lib/core.js";
import go from "../highlightjs/lib/languages/go.js";
import rust from "../highlightjs/lib/languages/rust.js";
// ...one import per language

const hljs: any = core;
hljs.registerLanguage("go", go);
hljs.registerLanguage("rust", rust);

export function highlightToHtml(source: string, language: string): string | null {
  if (!hljs.getLanguage(language)) return null;
  return hljs.highlight(source, { language, ignoreIllegals: true }).value;
}

third_party/ts/highlight/highlight.d.ts (hand-written):

export declare function highlightToHtml(source: string, language: string): string | null;

third_party/ts/highlight/BUILD:

ts_bundle(
    name = "bundle",
    main = "entry.ts",
    out = "highlight.js",
    minify = True,   # required, see below
    deps = ["//third_party/ts:highlightjs"],   # a ts_module
)

ts_library(
    name = "highlight",
    srcs = [":bundle", "highlight.d.ts"],   # a build label plus a .d.ts
    module_name = "@veritas/highlight",
)

Consumers import { highlightToHtml } from "@veritas/highlight" and list //third_party/ts/highlight in deps (also for ts_test and ts_bundle, like any other module). Verified: deno check passes, a ts_test with runner = "vitest" runs it, and the final ts_bundle includes it.

Result for 12 languages: 76 KB minified, 24 KB gzipped; the consumer's bundle went from 375 KB to 442 KB.

Rough edges

  1. Relative path into the unpacked module (../highlightjs/lib/core.js): it depends on how the bundle sandbox lays out dependencies.
  2. Hand-written .d.ts: nothing generates it, and the package's own types cannot be used (see the circular alias above).
  3. minify = True is required: the bundle keeps JSDoc such as @typedef { import("./html_renderer").Renderer }, which deno check of the wrapping ts_library then tries to resolve (TS2307). Minifying strips the comments.
  4. The entry file is maintained by hand (every language is an import and a registration).
  5. Every consumer target lists the library in its own deps.

Proposal: make it automatic

Any subset of these would help; they are independent.

  • A rule for the pattern, for example ts_npm_bundle(name, module, imports = [...], exports = [...], types = ..., minify = True): generates the entry from a list of specifiers and exported names, resolves them through the unpacked ts_module (no hand-written relative paths), bundles with esbuild, and wraps the result in a ts_library. Types: accept a .d.ts, or emit a permissive stub, or generate declarations when the package ships ESM types.
  • A js_library rule: plain JavaScript plus an optional .d.ts, published with the module metadata so ts_library / ts_test / ts_bundle can import it, without type-checking the JavaScript. This is what the wrapper above is, spelled directly.
  • Honor exports in the import map: record it in ts_module.json, expand subpath patterns (./mode/*) and conditions (import, default), and use exports["."] as the entry when there is no main / module. That makes pure-ESM packages such as @codemirror/legacy-modes and many modern libraries work directly, with no bundling.
  • Fix the esbuild alias for subpaths: emit an alias for the package directory (or per-subpath aliases) instead of only the entry file, so pkg/sub resolves in ts_bundle.
  • CommonJS (the module.exports case without bundling) is the hardest and probably not worth supporting directly; bundling covers it.

Acceptance criteria

  • A documented, tested way to depend on a CommonJS npm package from ts_library, ts_test and ts_bundle, without hand-written relative paths
  • A pure-ESM package with an exports map and no main can be imported (the legacy-modes case)
  • The pattern or rule is covered by a fixture and the CI smoke test (bundle a small CommonJS package, import it from a ts_library, run a ts_test on it)
  • The docs list the supported package shapes and the limits

Reference

  • Related: the ts_module unpack and import map code in tools/please_ts/unpack and tools/please_ts/importmap.

Activity

  1. walterjgsp
    added
    enhancementNew feature or request
    on Oct 4, 2026

  2. walterjgsp commented on Oct 4, 2026

    @walterjgsp
    walterjgsp
    on Oct 4, 2026
    ContributorAuthor

    Analysis and proposed design

    Short version: don't fix this with more resolution rules or a bundling wrapper. Let Deno resolve npm packages itself (npm: specifiers), and feed it a cache built offline from hash-pinned tarballs. That needs no node_modules, no lockfile, and no exports reimplementation, and it keeps the plugin's stated principles ("Zero node_modules", "Zero Foreign Lockfiles", offline --no-remote).

    Root causes in the code

    1. importmap.RegisterRootModules maps pkg/ to the entry's directory, not the package root, hence highlight.js/lib/core becoming .../lib/lib/core.
    2. The ts_bundle aliases (bundle.go) map pkg to the entry file, and esbuild appends a subpath to an alias, hence <entry file>/core.js.
    3. package.json exports is parsed into PackageJSON.Exports and never used, and a package with only exports has no entry at all.
    4. Fixing each of these means reimplementing Node resolution (patterns, conditions, CommonJS interop) in Go.

    Spike results (Deno 2.9.7, the plugin's pinned default; scratch folder, nothing in the repo changed)

    Deno resolves both failing shapes natively. With no node_modules directory and an import map "highlight.js/": "npm:/highlight.js@11.12.0/":

    Case deno check deno run deno bundle
    highlight.js/lib/core and language subpaths (CommonJS) passes works 60 KB
    @codemirror/legacy-modes/mode/go (no main/module, only exports: ./mode/*) passes works bundles

    The cache can be built offline from verified tarballs. Deno's npm cache is npm/registry.npmjs.org/<pkg>/<version>/ (the extracted package) plus a small registry.json. I built it from tarballs whose sha512 I checked, with a packument of name, dist-tags, a version entry (dist, and dependencies/peerDependencies taken from the verified tarball's own package.json) and _deno.packumentFormat: "full". With the proxy pointed at a dead port:

    Check Result
    debug@4.3.7 (CommonJS) with transitive ms deno check passes, deno run prints the right output
    deno bundle works; Deno downloads an esbuild binary (not an npm package) to $DENO_DIR/dl/esbuild-0.25.5-1/esbuild-linux-x64 (10 MB), so pre-placing that exact file makes bundling offline; the bundle then runs with no cache at all
    Without _deno.packumentFormat in registry.json Deno tries to download the packument

    Deno trusts the cache and does not verify it:

    Scenario Result
    Wrong integrity string in registry.json, package dir present works, ignored
    A file inside the extracted package dir edited the edited code runs
    Package dir, transitive slice or esbuild binary missing Deno tries the network and, with it unreachable, hangs until a timeout. run --cached-only fails instantly with a clear message; check --cached-only still hung on a missing transitive slice

    Today's hermeticity (for comparison)

    ts_module pins the root tarball by sha256 (remote_file), but with resolve_transitive = True (the default) the unpack step runs unsandboxed and picks transitive dependencies from the registry by semver range, unpinned. vitest_toolchain runs an unpinned deno cache too. So the proposed model is stricter than the current one, not looser.

    Proposed design

    1. One target per npm package, transitive ones included: a ts_module-style rule with remote_file + sha256, and its dependencies listed in deps, like Please's pip rules. The hashes live in BUILD files; no separate lockfile. A small helper can print the BUILD snippets for a package and its closure.
    2. The build step runs offline. It extracts the verified tarball into Deno's cache layout and writes the registry.json. It fails at build time if any dependencies entry has no matching declared dependency (the dependency closure check), instead of letting Deno fail at runtime.
    3. The import map maps name and name/ to the npm: forms for npm packages. First-party module_name aliases are unchanged.
    4. The runners merge the dependencies' cache slices into the per-run DENO_DIR they already create, and run with --no-remote (and --cached-only for run/test).
    5. Bundling: pre-place the pinned esbuild binary under dl/ and use deno bundle, or keep esbuild as the hash-pinned BundlerTool (the plugin already supports it). deno bundle prints "experimental and subject to changes", which is a risk to weigh.

    What this does to the proposals in this issue

    • Honor exports, fix the esbuild alias, and CommonJS: replaced by Deno's own resolution.
    • js_library and ts_npm_bundle: not needed for npm packages. The bundle-and-wrap pattern stays a documented fallback.

    Costs and risks

    • The _deno.packumentFormat key and the dl/esbuild-<version>-<n> path are Deno-internal. They tie the format to the pinned Deno version, so every Deno bump needs a CI test (the fixtures below).
    • Integrity is enforced by Please's sha256 on the tarball and nowhere else; the cache must be treated as trusted input.
    • Network isolation is not guaranteed by --no-remote/--cached-only alone; real isolation needs Please's sandbox, which ts_test does not enable by default.
    • Not verified yet: a cache built through Please rules, merging slices from several targets, Vitest and browser runners, platforms other than linux-x64 (the esbuild binary is per platform).
    • Behaviour change for existing users: ship behind a config flag for one release.

    Plan

    1. Prototype: a rule that builds a cache slice from a pinned tarball, the import map entries for it, and the runner merging slices (Deno runner and type check first).
    2. Fixtures first: a CommonJS package with a transitive dependency, plus highlight.js and legacy-modes for the two shapes in this issue, with a CI smoke test.
    3. Bundling, the helper that prints BUILD snippets, the flag, docs.
    4. Flip the default and remove the npm part of the import map, the semver resolution in unpack, and the esbuild alias code.
  3. walterjgsp
    added 2 commits that reference this issue on Oct 4, 2026

    fix(ts): do not force the sandbox on ts_npm_module

    9700013

    feat(ts): resolve npm dependencies automatically, strictly and reprod…

    6fa91f1

  4. walterjgsp commented on Oct 4, 2026

    @walterjgsp
    walterjgsp
    on Oct 4, 2026
    ContributorAuthor

    Done: the prototype (#62) and the wiring (#77) are merged and released in ts-v0.7.0.

    • ts_npm_module provides an npm package through Deno's own resolution (CommonJS, entry-less and exports-only packages, subpaths), pinned by sha256, with strict and reproducible automatic dependency resolution. No lockfile.
    • It works in type checks, ts_test (Deno and Vitest), ts_browser_test, ts_bundle and ts_binary. Bundles use deno bundle with a pinned esbuild (esbuild_toolchain); ts_binary embeds the packages.
    • Not done, as noted in docs/ts/usage.md: the helper that prints pinned declarations (automatic resolution makes it much less necessary), pinning deno compile's denort download, and the non-npm ts_bundle path still fetches esbuild at run time.
  5. walterjgsp
    closed this as completedon Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions