Skip to content

add an interface for dask.fft - #214

Open
vtavana wants to merge 2 commits into
masterfrom
dask-interface
Open

vtavana wants to merge 2 commits into
masterfrom
dask-interface

Conversation

@vtavana

@vtavana vtavana commented Aug 13, 2025 •

Copy link
Copy Markdown
Contributor

address #5

@vtavana vtavana self-assigned this Aug 13, 2025
@vtavana
vtavana force-pushed the dask-interface branch 2 times, most recently from 90578c9 to 647d164 Compare August 13, 2025 13:21
@vtavana
vtavana marked this pull request as ready for review August 28, 2025 12:08
Copilot AI review requested due to automatic review settings August 28, 2025 12:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a new Dask interface for the mkl_fft package, providing a drop-in replacement for the dask.fft module that leverages Intel MKL's optimized FFT implementations.

  • Implements mkl_fft.interfaces.dask_fft module with all FFT functions wrapped using Dask's fft_wrap
  • Adds comprehensive test suite for the new Dask interface functionality
  • Updates build configuration and documentation to support the new interface

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pyproject.toml Adds dask_interface optional dependency and includes dask in test dependencies
mkl_fft/interfaces/dask_fft.py New module implementing Dask FFT interface by wrapping numpy_fft functions
mkl_fft/tests/third_party/dask/test_dask_fft.py Comprehensive test suite for Dask FFT functionality
mkl_fft/tests/test_interfaces.py Updates interface tests to include Dask FFT
mkl_fft/interfaces/init.py Conditionally imports dask_fft module when Dask is available
mkl_fft/interfaces/README.md Adds documentation for the new Dask interface
conda-recipe/meta.yaml Updates conda recipe to include dask test dependency and import test
conda-recipe-cf/meta.yaml Updates conda-forge recipe similarly
CHANGELOG.md Documents the new Dask interface feature
.github/workflows/*.yml Updates CI workflows to include dask as test dependency

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@IntelPython IntelPython deleted a comment from Copilot AI Aug 28, 2025
@IntelPython IntelPython deleted a comment from Copilot AI Aug 28, 2025
@IntelPython IntelPython deleted a comment from Copilot AI Aug 28, 2025

@jharlow-intel jharlow-intel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some conflicts need to be resolved.

It's okay that the 3.14 internal CI is broken for now, though

@ndgrigorian

Copy link
Copy Markdown
Collaborator

@antonwolfy @jharlow-intel
addressed conflicts, we can get this in now

@ndgrigorian

Copy link
Copy Markdown
Collaborator

changes in the workflows made it more convenient to put this on top of the conda-forge recipe updates, this can go in after that

@ndgrigorian

Copy link
Copy Markdown
Collaborator

@jharlow-intel @antonwolfy ready for review

@jharlow-intel jharlow-intel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for keeping this going, and sorry it sat so long! The interface itself is close: wrapping mkl_fft's NumPy functions with dask's own fft_wrap keeps the chunking checks and signatures identical to dask.array.fft, and the tests adapted from dask's suite cover chunked-axis errors plus n/s/axes/norm. The main thing is that master has moved a lot since April, so it needs another rebase; a few smaller things inline.

  • This needs another rebase before it can merge: master has moved about 330 commits since April, and 7 files now conflict (pyproject.toml, mkl_fft/interfaces/__init__.py, both conda recipes, and the three workflows). When you redo interfaces/__init__.py, it'd be great to follow master's newer pattern there (appending to __all__ and deleting the probe import).

import contextlib
from itertools import combinations_with_replacement

import dask

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this use pytest.importorskip("dask.array") instead of a bare import? Right now running the installed suite (pytest --pyargs mkl_fft) without dask fails at collection instead of skipping, and test_dask_fftwrap in test_interfaces.py touches mfi.dask_fft unconditionally, so it fails the same way. It might also be worth avoiding the private dask helpers (da._array_expr_enabled, dask.array.numpy_compat), since those can move between dask releases.

import mkl_fft.interfaces.dask_fft as dask_fft

requires_dask_2024_8_2 = pytest.mark.skipif(
dask.__version__ < "2024.8.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This compares version strings, so "2024.10.0" < "2024.8.2" is True and the norm tests get skipped on dask 2024.10 through 2024.12. packaging.version.Version(dask.__version__) would fix it. And since norm needs dask >= 2024.8.2, should the dask_interface extra pin dask>=2024.8.2 too?

except ImportError:
pass
else:
from . import dask_fft

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since import mkl_fft imports mkl_fft.interfaces eagerly, anyone with dask installed now pulls in dask.array on every import mkl_fft, and a broken dask install would make import mkl_fft itself fail. Would a lazy import (a module-level __getattr__ for dask_fft) work here? I know scipy_fft is eager today, but dask.array is a heavier import.

Comment thread CHANGELOG.md

### Added
* Enabled support of Python 3.13 [gh-164](https://github.com/IntelPython/mkl_fft/pull/164)
* Added a new interface for FFT module of Dask accessible through `mkl_fft.interfaces.dask_fft` [gh-214](https://github.com/IntelPython/mkl_fft/pull/214)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads-up: this entry ended up in an old released section (next to the Python 3.13 entry, gh-164). Could you move it to the current dev section when you rebase?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants