diff --git a/.github/AGENTS.md b/.github/AGENTS.md index 47576d02..6751dc71 100644 --- a/.github/AGENTS.md +++ b/.github/AGENTS.md @@ -3,13 +3,15 @@ CI/CD workflows and repo automation. ## Workflows (source of truth) -- `conda-package.yml` — Intel channel conda build/test pipeline -- `conda-package-cf.yml` — conda-forge-oriented build/test pipeline -- `build-with-clang.yml` — Intel clang compatibility checks -- `build-with-standard-clang.yml` — standard clang compatibility checks -- `build_pip.yml` — pip build pipeline with pre-release NumPy +- `conda-package.yml` — Intel-channel conda build and test +- `conda-package-cf.yml` — conda-forge conda build and test +- `build_pip.yml` — editable pip build with `icx`, including pre-release NumPy +- `build-with-clang.yml` — build with `icx` from the oneAPI apt repository +- `build-with-standard-clang.yml` — build with upstream clang - `pre-commit.yml` — lint/format checks -- `openssf-scorecard.yml` — security scanning +- `coverity.yml` — Coverity static analysis (see `coverity/README.md`) +- `openssf-scorecard.yml` — OpenSSF Scorecard +- `zizmor.yml` — GitHub Actions security lint ## Policy - Treat workflow YAML as canonical for platform/Python matrices. diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml new file mode 100644 index 00000000..e436b17b --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -0,0 +1,64 @@ +name: Bug report +description: Report incorrect results, a crash, or a build failure +labels: ["bug"] +body: + - type: markdown + attributes: + value: | + For security vulnerabilities, do not open an issue — follow + [SECURITY.md](https://github.com/IntelPython/mkl_umath/blob/main/SECURITY.md). + + - type: textarea + id: description + attributes: + label: Description + description: What happened, and what did you expect instead? + validations: + required: true + + - type: textarea + id: reproducer + attributes: + label: Reproducer + description: A minimal, self-contained snippet. Include the ufunc, input values, and dtype. + render: python + validations: + required: true + + - type: dropdown + id: install-source + attributes: + label: How was `mkl_umath` installed? + options: + - Intel conda channel (software.repos.intel.com) + - conda-forge + - pip, Intel index (software.repos.intel.com) + - pip / PyPI + - Built from source + validations: + required: true + + - type: textarea + id: versions + attributes: + label: Versions + description: | + Output of: + ``` + python -c "import mkl_umath, numpy; print(mkl_umath.__version__); print(numpy.__version__)" + ``` + Add your OS and Python version too. + render: shell + validations: + required: true + + - type: textarea + id: notes + attributes: + label: Anything else + description: | + Optional. Whether NumPy patching was active (`mkl_umath.is_patched()`), + whether the result differs from stock NumPy, or a non-default + `MKL_NUM_THREADS`. + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 00000000..f348f1ae --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,8 @@ +blank_issues_enabled: true +contact_links: + - name: Security vulnerability + url: https://www.intel.com/content/www/us/en/security-center/vulnerability-handling-guidelines.html + about: Report security vulnerabilities through Intel's process, not a public issue. + - name: Question about usage + url: https://github.com/IntelPython/mkl_umath/blob/main/README.md + about: Check the README first, including the patching section. diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml new file mode 100644 index 00000000..a1d08282 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -0,0 +1,37 @@ +name: Feature request +description: Propose a new ufunc loop, dtype, or capability +labels: ["enhancement"] +body: + - type: textarea + id: problem + attributes: + label: What problem does this solve? + description: The use case, not the implementation. + validations: + required: true + + - type: textarea + id: proposal + attributes: + label: Proposal + description: | + What you would like `mkl_umath` to do. If it covers a NumPy ufunc, name + it — NumPy's semantics are the contract for patched loops. + validations: + required: true + + - type: input + id: upstream + attributes: + label: Upstream equivalent + description: Link to the NumPy docs for the ufunc, if there is one. + validations: + required: false + + - type: textarea + id: alternatives + attributes: + label: Alternatives considered + description: Optional. Workarounds you are using today. + validations: + required: false diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index d1544886..dd9d0aac 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -19,7 +19,8 @@ Higher-precedence rules override lower-precedence context. ## Contribution expectations - Keep changes atomic and single-purpose. -- Preserve runtime patching API (`use_in_numpy()`, `restore()`, `is_patched()`) unless explicitly requested. +- Preserve the runtime patching API (`patch_numpy_umath()`, `restore_numpy_umath()`, + `is_patched()`, and the `mkl_umath()` context manager) unless explicitly requested. - For behavior changes, update tests in `mkl_umath/tests/` in the same step. - For bugs, include a regression test. - Do not modify generated artifacts directly when template/source files are the intended edit points. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 00000000..051112d9 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,25 @@ +# Description + + + +## Verification + + + +- Tests: +- Lint: + +## Not verified + + + +## Checklist + +- [ ] Results match stock NumPy, or the difference is intentional and called out above. +- [ ] Behavior changes have tests in `mkl_umath/tests/`; bug fixes have a regression test. +- [ ] `CHANGELOG.md` updated under `## [dev]` with a `[gh-NNN]` link, or the change isn't user-visible. + + diff --git a/.gitignore b/.gitignore index 3ab62969..92a6e677 100644 --- a/.gitignore +++ b/.gitignore @@ -54,6 +54,12 @@ var/ pip-log.txt pip-delete-this-directory.txt +# Virtual environments, test caches, local env files +.venv/ +venv/ +.pytest_cache/ +.env + # Unit test / coverage reports htmlcov/ .tox/ @@ -87,3 +93,9 @@ mkl_umath/src/__umath_generated.c mkl_umath/src/mkl_umath_loops.c mkl_umath/src/mkl_umath_loops.h mkl_umath/src/_patch.c + +# ASV benchmark artifacts +.asv/ + +# Developer-local coding agent settings +.claude/settings.local.json diff --git a/AGENTS.md b/AGENTS.md index 01525618..7119dd7c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,109 +1,60 @@ -# AGENTS.md +# AGENTS.md — mkl_umath Entry point for agent context in this repo. -## What this repository is -`mkl_umath` exposes Intel® OneMKL-powered universal function loops for NumPy, originally part of Intel® Distribution for Python* and factored out per NEP-36 (Fair Play). - -It provides: -- `mkl_umath._ufuncs` — OneMKL-backed NumPy ufunc loops -- `mkl_umath._patch_numpy` — runtime patching interface (`patch_numpy_umath` `restore_numpy_umath`, `is_patched()`) -- Performance-optimized math operations (sin, cos, exp, log, etc.) using Intel MKL VM +## What this project is +`mkl_umath` provides NumPy ufunc loops backed by Intel® oneMKL Vector Math, and +can patch them into NumPy at runtime. It was factored out of Intel® Distribution +for Python* per NEP-36 (Fair Play). ## Key components -- **Python interface:** `mkl_umath/__init__.py`, `_init_helper.py` -- **Core C implementation:** `mkl_umath/src/` (ufuncsmodule.c, mkl_umath_loops.c.src) -- **Cython patch layer:** `mkl_umath/src/_patch_numpy.pyx` -- **Code generation:** `generate_umath.py`, `generate_umath_doc.py` -- **Build system:** meson-python + Cython - -## Build dependencies -**Required:** -- Compiler toolchain: Intel `icx` or `clang` (with Intel-only flags gated when using clang) -- Intel® oneMKL (`mkl-devel`) -- meson-python, CMake, Ninja, Cython>=3.1.0, NumPy - -**Build against an existing `mkl` installation:** - -Install the build dependencies via Conda: -```bash -conda install -c https://software.repos.intel.com/python/conda \ - mkl-devel dpcpp_linux-64 "cython>=3.1.0" meson-python cmake ninja numpy -``` -or via pip: -```bash -pip install mkl-devel "cython>=3.1.0" meson-python cmake ninja numpy -``` -then build: -```bash -CC=icx pip install --no-deps --no-build-isolation . # clang is also supported in CI -``` - -## CI/CD -- **Platforms:** Linux, Windows -- **Python versions:** 3.10, 3.11, 3.12, 3.13, 3.14 -- **Workflows:** `.github/workflows/` - - `conda-package.yml` — main conda build/test pipeline - - `conda-package-cf.yml` — conda-forge-oriented build/test pipeline - - `build_pip.yml` — validates pip build with pre-release NumPy - - `build-with-clang.yml` — Intel clang compatibility check - - `build-with-standard-clang.yml` — standard clang compatibility check - - `openssf-scorecard.yml` — security scorecard - -## Distribution -- **Conda:** `https://software.repos.intel.com/python/conda` -- **PyPI:** `https://software.repos.intel.com/python/pypi` -- Requires Intel-optimized NumPy from Intel channels - -## Usage -```python -import mkl_umath -mkl_umath.patch_numpy_umath() # Patch NumPy to use MKL loops -# ... perform NumPy operations (now accelerated) ... -mkl_umath.restore_numpy_umath() # Restore original NumPy loops -``` - -## How to work in this repo -- **Performance:** Changes should maintain or improve MKL VM utilization -- **Compatibility:** Must work with upstream NumPy APIs (NEP-36 compliance) -- **Testing:** Add tests to `mkl_umath/tests/test_basic.py` -- **Build hygiene:** `meson.build` is the source of truth for build config — verify Linux + Windows -- **Docs:** Update docstrings via `ufunc_docstrings_numpy{1,2}.py` - -## Code structure -- **Generated code:** `*.src` files are templates (conv_template.py processes them) -- **Precision flags:** fp:precise, fimf-precision=high, fprotect-parens (non-negotiable) -- **Security:** Stack protection, FORTIFY_SOURCE, NX/DEP enforced in `meson.build` -- **Build options:** `opt_report` and `mkl_threading` are exposed via `meson.options` - - -## Common pitfalls -- **NumPy source:** Requires Intel-optimized NumPy from Intel channel (`software.repos.intel.com/python/conda`). PyPI NumPy may cause runtime failures or incorrect results. -- **Precision flags:** `fp:precise`, `fimf-precision=high` enforce IEEE 754 compliance. Removing them breaks numerical correctness in scientific computing. -- **Patching order:** If using multiple Intel patches (e.g., `mkl_random` + `mkl_umath`), apply `mkl_umath` last. Verify with `is_patched()` after each. -- **Compiler/toolchain:** `icx` and `clang` are both supported; when using clang, keep Intel-only flags behind compiler guards. -- **Build validation:** - - After setup: `which ${CC:-icx}` → should resolve to the intended compiler toolchain - - Check: `python -c "import numpy; print(numpy.__version__)"` → confirm NumPy is available - -## Notes -- `_vendored/` contains vendored NumPy code generation utilities -- Version in `mkl_umath/_version.py` (read dynamically by `meson.build`) -- Patching is runtime-only; no NumPy source modification +- **Package and public API:** `mkl_umath/`, `mkl_umath/__init__.py` +- **Ufunc extension:** `mkl_umath/src/ufuncsmodule.c`, plus `__umath_generated.c` + from `mkl_umath/generate_umath.py` +- **Loop templates:** `mkl_umath/src/mkl_umath_loops.{c,h}.src` +- **Patching:** `mkl_umath/src/_patch_numpy.pyx`; persistent and one-shot + patching in `patch.py`, `with_patch.py`, `_patch_startup.py`, and the + `__main__.py` CLI +- **Tests:** `mkl_umath/tests/` +- **Vendored helpers:** `_vendored/` +- **Packaging:** `conda-recipe/`, `conda-recipe-cf/` +- **Benchmarks:** `benchmarks/` + +## Build/runtime basics +- Build system: `pyproject.toml` + `meson.build`, with options in `meson.options` +- Build deps: `mkl-devel`, `numpy`, `meson-python`, `cmake`, `ninja`, `cython`, + and a C compiler (CI uses `icx` and `clang`) +- Runtime deps: `numpy`; the conda recipes add the MKL and compiler runtimes +- Setup, checks, and style: `CONTRIBUTING.md` +- Single test: `pytest mkl_umath/tests/::` +- Single-file lint: `pre-commit run --files ` + +## Development guardrails +- Preserve NumPy ufunc behavior; patched loops stand in for NumPy's own. +- Edit the `*.src` templates and `generate_umath.py`, not generated C. +- Keep the floating-point precision flags in `meson.build`; Intel-only flags stay + behind its compiler checks. +- Keep patching reversible, with `is_patched()` reporting the truth. +- Keep both extensions free-threading compatible. +- Pair behavior changes with tests and keep diffs minimal. +- Avoid hardcoding mutable versions/matrices/channels in docs. + +## Where truth lives +- Build/config: `pyproject.toml`, `meson.build`, `meson.options` +- Dependencies: `pyproject.toml`, `conda-recipe*/meta.yaml` +- CI/workflows: `.github/workflows/*.yml` +- Public API: `mkl_umath/__init__.py`, `mkl_umath/src/_patch_numpy.pyx` +- Tests: `mkl_umath/tests/` + +For behavior policy, see `.github/copilot-instructions.md`. ## Directory map -Below directories have local `AGENTS.md` for deeper context: -- `.github/AGENTS.md` — CI/CD workflows and automation -- `mkl_umath/AGENTS.md` — Python API and code generation -- `mkl_umath/src/AGENTS.md` — C/Cython implementation layer -- `mkl_umath/tests/AGENTS.md` — unit tests and validation -- `conda-recipe/AGENTS.md` — Intel channel conda packaging -- `conda-recipe-cf/AGENTS.md` — conda-forge compatible recipe -- `_vendored/AGENTS.md` — vendored NumPy utilities - ---- - -For broader IntelPython ecosystem context, see: -- `dpnp` (Data Parallel NumPy) -- `mkl_random` (MKL-based random number generation) -- `numba-dpex` (Numba + SYCL) +Use nearest local `AGENTS.md` when present: +- `.github/AGENTS.md` — CI workflows and automation policy +- `mkl_umath/AGENTS.md` — package modules, API, and code generation +- `mkl_umath/src/AGENTS.md` — loop templates and the two extensions +- `mkl_umath/tests/AGENTS.md` — test scope and conventions +- `conda-recipe/AGENTS.md` — Intel-channel conda packaging +- `conda-recipe-cf/AGENTS.md` — conda-forge recipe +- `_vendored/AGENTS.md` — vendored NumPy template tooling +- `benchmarks/AGENTS.md` — ASV performance suite diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..aa803756 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,44 @@ +# Contributing to `mkl_umath` + +See [README.md](README.md) for usage, [AGENTS.md](AGENTS.md) for a map of the +source tree, and [SECURITY.md](SECURITY.md) to report a vulnerability. + +## Setup + +On Linux: + +```sh +conda create -n mkl_umath-dev -c conda-forge python pip mkl-devel \ + dpcpp_linux-64 numpy meson-python ninja cmake "cython>=3.1.0" pytest +conda activate mkl_umath-dev +CC=icx pip install -e . --no-build-isolation --no-deps \ + -Csetup-args=-Dmkl_threading=gnu_thread +``` + +`gnu_thread` is required with conda-forge's MKL; the default threading layer +fails at import. + +## Checks + +```sh +pytest mkl_umath/tests +pre-commit run --all-files +``` + +The pre-commit hooks enforce formatting; otherwise, match the surrounding code. + +## Guidelines + +- Keep changes small and focused. +- Match stock NumPy results, and call out any intentional difference. +- Add tests with behavior changes, and a regression test with bug fixes. +- Edit the `*.src` templates and `generate_umath.py`, not generated C. +- Keep the floating-point precision flags in `meson.build`. +- Keep patching reversible. + +## Pull requests + +Work on a branch and fill in the PR template. For user-visible changes, add a +`CHANGELOG.md` entry under `[dev]` with a `gh-NNN` link. + +Contributions are licensed under the terms in [LICENSE.txt](LICENSE.txt). diff --git a/README.md b/README.md index 9e6cafad..432da96a 100644 --- a/README.md +++ b/README.md @@ -121,3 +121,10 @@ then build against the existing installation with: ```sh CC=icx pip install --no-build-isolation --no-deps . ``` + +--- +# Contributing + +See [CONTRIBUTING.md](CONTRIBUTING.md) for the development workflow: setting up a +build environment, running the tests and lint hooks, code style, and what to +include in a pull request. diff --git a/_vendored/AGENTS.md b/_vendored/AGENTS.md index 0ba2553a..475188d3 100644 --- a/_vendored/AGENTS.md +++ b/_vendored/AGENTS.md @@ -1,21 +1,17 @@ # AGENTS.md — _vendored/ -Vendored dependencies from upstream projects (NumPy). +Build-time template tooling copied from NumPy's `numpy/_build_utils`. ## Files -- **conv_template.py** — NumPy's template processor (from `numpy.distutils`) -- **__init__.py** — Python package marker +- `conv_template.py` — expands `/**begin repeat ... end repeat**/` blocks in + `.src` files +- `process_src_template.py` — command-line wrapper that `meson.build` runs on + the `.src` files; loads `conv_template.py` +- `README.md` — provenance -## Why vendored? -- `numpy.distutils` removed in NumPy 2.0+ / Python 3.12+ -- Needed for `.src` template processing at build time -- Vendored to maintain build compatibility across NumPy versions - -## Maintenance -- Source: NumPy's `numpy/distutils/conv_template.py` -- Update if template syntax changes upstream (rare) -- Do not modify vendored code (keep attribution intact) - -## Usage -- Imported by `generate_umath.py` for `.src` → `.c` conversion -- Processes `/**begin repeat ... end repeat**/` blocks +## Guardrails +- Prefer updating upstream source when feasible; keep local vendored diffs + minimal. +- Do not refactor vendored code opportunistically in unrelated PRs. +- `black` and `isort` are configured to skip the vendored files + (`pyproject.toml`). diff --git a/benchmarks/AGENTS.md b/benchmarks/AGENTS.md new file mode 100644 index 00000000..4d2a6a13 --- /dev/null +++ b/benchmarks/AGENTS.md @@ -0,0 +1,24 @@ +# AGENTS.md — benchmarks/ + +ASV performance suite for `mkl_umath`. + +## Scope +- `asv.conf.json` — ASV configuration, channels, and regression thresholds +- `benchmarks/micro/` — per-ufunc micro-benchmarks +- `benchmarks/npbench/` — end-to-end kernels adapted from npbench +- `benchmarks/_patch_setup.py` — patches NumPy at import +- `README.md` — coverage table, threading default, and run commands + +## Guardrails +- Treat `asv.conf.json` as canonical for ASV settings; treat `README.md` as + canonical for what each module covers. +- Benchmarks run on patched NumPy only: `_patch_setup.py` raises if patching + fails, so results never silently come from stock NumPy. +- Comparability across machines depends on the thread default in + `benchmarks/__init__.py` and the warmup call in each benchmark's `setup`. + Changing either invalidates comparison against existing results — call it + out explicitly. +- Keep inputs deterministic; benchmarks seed their own RNG. +- Report performance numbers with reproducible context: hardware, thread count, + versions, and the command used. +- Results under `benchmarks/.asv/` are local artifacts. diff --git a/conda-recipe-cf/AGENTS.md b/conda-recipe-cf/AGENTS.md index cea38d84..cce41b51 100644 --- a/conda-recipe-cf/AGENTS.md +++ b/conda-recipe-cf/AGENTS.md @@ -1,22 +1,18 @@ # AGENTS.md — conda-recipe-cf/ -Conda-forge compatible build recipe (alternative to Intel channel recipe). +conda-forge variant of the conda recipe. -## Difference from conda-recipe/ -- No `conda_build_config.yaml` (uses conda-forge defaults) -- May use different compiler toolchain -- For conda-forge feedstock integration (if upstreamed) +## Differences from `conda-recipe/` +- Resolves dependencies from conda-forge only +- Installs with `pip install` directly; no wheel build or retag +- Links MKL's GNU threading layer (`-Dmkl_threading=gnu_thread` in `build.sh`) + and uses `llvm-openmp` instead of `intel-openmp` +- Sets its version by hand in `meta.yaml` instead of reading git tags -## Files -- **meta.yaml** — conda-forge compatible metadata -- **build.sh** / **bld.bat** — platform build scripts -- **run_tests.{sh,bat}** — test invocation +Both recipes build with `icx` and use the same `conda_build_config.yaml`. -## Status -- Not currently used in main CI workflows -- Maintained for potential conda-forge submission -- Use `conda-recipe/` for Intel channel builds - -## Notes -- If upstreaming to conda-forge, this recipe should be preferred -- Compiler requirements may differ (Clang/GCC vs Intel icx) +## Guardrails +- Keep conda-forge recipe semantics separate from the Intel-channel recipe. +- Keep the `meta.yaml` version equal to `mkl_umath/_version.py`. +- Keep changes in step with `.github/workflows/conda-package-cf.yml`, which + builds this recipe and runs the full test suite against the result. diff --git a/conda-recipe/AGENTS.md b/conda-recipe/AGENTS.md index 08182026..776941ff 100644 --- a/conda-recipe/AGENTS.md +++ b/conda-recipe/AGENTS.md @@ -1,31 +1,17 @@ # AGENTS.md — conda-recipe/ -Conda package build recipe for Intel channel distribution. +Intel-channel conda packaging. ## Files -- **meta.yaml** — package metadata, dependencies, build requirements -- **build.sh** — Linux build script -- **bld.bat** — Windows build script -- **conda_build_config.yaml** — build matrix (Python versions, numpy pins) -- **run_tests.{sh,bat}** — post-build test invocation +- `meta.yaml` — package metadata, dependencies, and the package test +- `build.sh` / `bld.bat` — build a wheel with `python -m build` using `icx`, + then install it; `build.sh` also retags the wheel's platform +- `conda_build_config.yaml` — NumPy and compiler pins +- `run_tests.sh` / `run_tests.bat` — not used; conda-build runs the test + commands in `meta.yaml` -## Build configuration -- **Channels:** `https://software.repos.intel.com/python/conda`, `conda-forge` -- **Python versions:** 3.10, 3.11, 3.12, 3.13, 3.14 -- **Build system:** meson-python (via `python -m build`) -- **Compilers:** Intel C compiler (icx) -- **Dependencies:** mkl-devel, intel-openmp, dpcpp_{linux,win}-64, numpy - -## Build outputs -- Conda package: `mkl_umath--.conda` -- Platform-specific: `linux-64/`, `win-64/` - -## CI usage -- Built in `.github/workflows/conda-package.yml` -- Artifacts uploaded per Python version -- Test stage uses built artifacts from channel - -## Maintenance -- Keep `conda_build_config.yaml` in sync with CI matrix -- NumPy pin: must match Intel channel NumPy versions -- MKL version: track oneAPI releases +## Guardrails +- Treat recipe files as canonical for packaging intent and dependency pins. +- Keep recipe changes in step with `.github/workflows/conda-package.yml`, which + builds this recipe and runs the full test suite against the result. +- The package test in `meta.yaml` runs only `test_basic.py`. diff --git a/mkl_umath/AGENTS.md b/mkl_umath/AGENTS.md index d5bc8b1c..52ad710e 100644 --- a/mkl_umath/AGENTS.md +++ b/mkl_umath/AGENTS.md @@ -1,36 +1,23 @@ # AGENTS.md — mkl_umath/ -Core MKL-backed ufunc implementation: Python interface, Cython patching, and C/MKL integration. +Package sources: public API, patching entry points, and ufunc code generation. -## Structure -- `__init__.py` — public API surface (`_ufuncs`, `_patch_numpy`, version) -- `_init_helper.py` — module initialization helpers -- `_version.py` — version string (read dynamically by `meson.build`) -- `src/` — C implementation and Cython patch layer -- `tests/` — basic functionality and patching tests -- `generate_umath.py` — code generation for ufunc loops -- `generate_umath_doc.py` — docstring generation -- `ufunc_docstrings_numpy{1,2}.py` — NumPy version-specific docstrings +## Key files +- `__init__.py` — public API: the ufuncs from `_ufuncs` and the patching + functions from `_patch_numpy` +- `generate_umath.py` — generates `__umath_generated.c` in the build directory +- `patch.py`, `_patch_startup.py`, `with_patch.py`, `__main__.py` — persistent + (`.pth`) and one-shot patching behind `python -m mkl_umath` +- `_version.py` — the version; `meson.build` reads it +- `generate_umath_doc.py`, `ufunc_docstrings_numpy{1,2}.py` — docstring sources + adapted from NumPy; the build does not run them +- `src/` — loop templates and the two extensions +- `tests/` — test suite -## Patching API -```python -mkl_umath.patch_numpy_umath() # Replace NumPy loops with MKL -mkl_umath.restore_numpy_umath() # Restore original NumPy loops -mkl_umath.is_patched() # Check patch status -``` - -## Development guardrails -- **API stability:** Patching must be runtime-only, no NumPy source modification -- **Precision:** fp:precise, fimf-precision=high, fprotect-parens are non-negotiable -- **Compatibility:** Must work with upstream NumPy (NEP-36 compliance) -- **Testing:** Add tests to `tests/test_basic.py` for new ufuncs or patch behavior - -## Code generation -- `*.src` files are templates processed by `_vendored/conv_template.py` -- Generated files: `src/__umath_generated.c`, loop implementations -- Docstrings: dual NumPy 1.x/2.x support via separate docstring modules - -## Notes -- `_patch_numpy.pyx` is Cython; changes require Cython rebuild -- MKL VM loops in `src/mkl_umath_loops.c.src` -- `src/ufuncsmodule.c` — NumPy ufunc registration and dispatch +## Guardrails +- Use `patch_numpy_umath()` / `restore_numpy_umath()` in new code and docs; + `use_in_numpy()` and `restore()` are deprecated aliases. +- Keep patching reversible: anything installed can be uninstalled, and + `is_patched()` reports the truth. +- New modules must be listed in `py.install_sources` in `meson.build`, or they + are not installed. diff --git a/mkl_umath/src/AGENTS.md b/mkl_umath/src/AGENTS.md index c2b81013..50f0d975 100644 --- a/mkl_umath/src/AGENTS.md +++ b/mkl_umath/src/AGENTS.md @@ -1,40 +1,24 @@ # AGENTS.md — mkl_umath/src/ -C/Cython implementation layer: MKL VM integration, ufunc loops, and NumPy patching. +C and Cython sources for the ufunc loops and the patching extension. -## Core files -- **ufuncsmodule.c** — NumPy ufunc registration and module init -- **ufuncsmodule.h** — ufunc module public headers -- **mkl_umath_loops.c.src** — MKL VM loop implementations (template, ~60k LOC) -- **mkl_umath_loops.h.src** — loop function declarations (template) -- **_patch_numpy.pyx** — Cython patching layer (runtime NumPy loop replacement) -- **fast_loop_macros.h** — loop generation macros -- **blocking_utils.h** — blocking/chunking utilities for large arrays +## Key files +- `mkl_umath_loops.c.src`, `mkl_umath_loops.h.src` — loop templates, expanded + at build time by `_vendored/process_src_template.py` and compiled into the + `libmkl_umath_loops` shared library +- `ufuncsmodule.c` — the `_ufuncs` extension, built with the generated + `__umath_generated.c` +- `_patch_numpy.pyx` — the `_patch_numpy` extension; swaps loops into NumPy's + ufuncs with `PyUFunc_ReplaceLoopBySignature` and keeps the originals for + restore +- `fast_loop_macros.h`, `blocking_utils.h` — loop helpers -## Template system -- `.src` files are processed by `_vendored/conv_template.py` at build time -- Generates type-specialized loops for float32, float64, complex64, complex128 -- Pattern: `/**begin repeat ... end repeat**/` blocks - -## MKL VM integration -- Calls `vdSin`, `vsExp`, etc. from Intel MKL Vector Math (VM) -- Blocking strategy: chunk large arrays for cache efficiency -- Error handling: MKL VM status → NumPy error state - -## Patching mechanism (_patch_numpy.pyx) -- Cython extension exposing `patch_numpy_umath()`, `restore_numpy_umath()`, - `is_patched()` -- Replaces function pointers in NumPy's ufunc loop tables -- Thread-safe: guards against concurrent patching -- Reversible: stores original pointers for restoration - -## Build output -- `mkl_umath_loops.c` → shared library (libmkl_umath_loops.so/.dll) -- `_patch_numpy.pyx` → Python extension (_patch.*.so) -- `ufuncsmodule.c` + `__umath_generated.c` → `_ufuncs` extension - -## Development notes -- **Precision flags:** fp:precise, fimf-precision=high enforced in `meson.build` -- **Security:** Stack protections, FORTIFY_SOURCE enabled -- **Vectorization:** `-fveclib=SVML -fvectorize` for SIMD (Intel compiler only) -- **Optimization reports:** `-Dopt_report=true` meson option for `-qopt-report=3` +## Guardrails +- Edit the `.src` templates, not the generated `.c`/`.h`. +- Keep results consistent with NumPy for every dtype a loop handles, including + NaN and signed-zero handling. +- Keep the patch lock and the saved original loops so patching stays + thread-safe and reversible. +- Keep both extensions free-threading compatible: `freethreading_compatible=True` + in `_patch_numpy.pyx` and `Py_MOD_GIL_NOT_USED` in `ufuncsmodule.c`. +- Build flags, including precision and hardening flags, live in `meson.build`. diff --git a/mkl_umath/tests/AGENTS.md b/mkl_umath/tests/AGENTS.md index abc3d058..7ebe5716 100644 --- a/mkl_umath/tests/AGENTS.md +++ b/mkl_umath/tests/AGENTS.md @@ -1,29 +1,21 @@ # AGENTS.md — mkl_umath/tests/ -Unit tests for MKL-backed ufuncs and NumPy patching. +Test suite for the loops and patching. `meson.build` installs it with the +package. -## Test files -- **test_basic.py** — core functionality, numerical correctness -- **test_patching.py** — patching API and state transitions -- **test_cli.py** — `python -m mkl_umath` CLI patch install/uninstall/status +## Files +- `test_basic.py` — loop results compared against NumPy +- `test_patching.py` — patch and restore state, and the `mkl_umath()` context + manager +- `test_cli.py` — persistent patch install, uninstall, and status +- `test_freethreading.py` — concurrent ufunc use and patching; the GIL check + runs only on a free-threaded build -## Test coverage -- Ufunc correctness: compare MKL loops vs NumPy reference -- Patching: `patch_numpy_umath()`, `restore_numpy_umath()`, `is_patched()` state transitions -- Edge cases: NaN, Inf, empty arrays, large arrays -- Dtype coverage: float32, float64, complex64, complex128 +## Expectations +- Behavior changes include test updates in the same PR; bug fixes include a + regression test. +- Keep tests deterministic and free of timing assertions. -## Running tests -```bash -pytest mkl_umath/tests/ -``` - -## CI integration -- Tests run in conda-package.yml workflow -- Separate test jobs per Python version (3.10-3.14) -- Linux + Windows platforms - -## Adding tests -- New ufuncs → add to `test_basic.py` with NumPy reference comparison -- Patching behavior → test state transitions and thread safety -- Use `numpy.testing.assert_allclose` for floating-point comparisons +## Entry points +- `pytest mkl_umath/tests` from a checkout +- `pytest --pyargs mkl_umath` against an installed package