Skip to content

WASM: reduce import-time eager Numba compilation #930

Description

@mmcky

Part of #925 (Phase 1). Measure first: the Phase 0 deployment (#928) includes cold/warm import-time benchmarks — if warm import lands under a couple of seconds, deprioritise this issue.

Problem

import quantecon triggers eager Numba compilation of five callables:

Callable Where Why eager
_probvec_parallel random/utilities.py#L94 module-level guvectorize with explicit signature
_probvec_cpu random/utilities.py#L98 same
sample_without_replacement gufunc random/utilities.py#L162 same
_ints_arr_to_bits vertex_enumeration.py#L305 same
comb_jit util/numba.py#L80 eager @jit signature intp(intp, intp)

Natively this costs milliseconds and nobody notices. In the browser, each eager compile is a full LLVM optimisation + WASM object emission + in-process LLD link + side-module load, and the emscripten-forge patches force-disable cache=True for @guvectorize/@vectorize — so the four gufunc compiles are not amortised by the persistent cache and are paid every session, on the critical path of every notebook. (comb_jit is an ordinary dispatcher with cache=True, so it does benefit from the cache after the first session.)

Options, in increasing order of ambition

  1. Convert the eagerly-signed gufuncs to lazy compilation — drop the explicit signature lists (dynamic gufuncs), or construct the parallel/cpu variants on first use inside probvec.
  2. Replace the small gufuncs with @njit(cache=True) loop implementations exposed through @overload — converts them from never-cached to persistently cached on Emscripten.
  3. Defer heavy submodule imports in quantecon/__init__.py via module __getattr__ (PEP 562) — also improves native import time.

Acceptance criteria

  • Warm-cache import quantecon time in JupyterLite measured before and after; result recorded here
  • No public API change (qe.random.probvec(..., parallel=...) keeps its signature)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions