On Wed, Jul 22, 2020 at 1:11 AM mreineck ***@***.***> wrote: I noticed that the dask FFT wrapper internally uses numpy's FFT. Unless this is done to avoid a scipy dependency, I'd suggest switching to scipy.fft (not scipy.fftpack, which is now legacy) instead. The advantage of scipy.fft is that it is much faster than numpy.fft when transforming multi-D arrays (even if only one axis is transformed), because it uses vector instructions where available. In addition to standard FFTs it also provides DCTs, DSTs and Hartley transforms. Optional shared-memory parallelization is available, and there are dedicated functions for single-precision transforms. You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < #6440 >, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AACKZTB6UAP6WCOLAVISPLTR42NMDANCNFSM4PEOJF4A >

It looks like there is some attempt to use scipy.fftpack if scipy is
installed, but not otherwise.

I think this is actually unused code. If scipy is available, then fft_wrap checks if it has been invoked with specific scipy.fftpack functions, but later on in the file it is only ever called with numpy functions.

I will have a look through the file and see if I can provide a patch by myself. If I manage to run unit tests locally, that should be feasible.

OK, that was easier than I thought ...

If you have any FFT benchmarks (especially ones with multidimensional transforms), it might be interesting to do a comparison.

I'm not sure how to address the discrepancy in the data type of the returned arrays when the input is float32 or complex64 ; this is something you will have to decide.

One issue which may come up is that calling into scipy.fft.fft triggers an asarray call:
https://github.com/scipy/scipy/blob/0765482e9a7cd7a7975f6161d2ab91994762ac8e/scipy/fft/_pocketfft/helper.py#L89-L92

Doing this would prevent other array libraries like CuPy from using the dask FFT wrapper

Interesting. But on the other hand, what does numpy do when promoting the input array to double precision? Won't this be something equivalent to asarray too?
I must admit that I don't really understand the implications of this, but if this is a fundamental problem, it should probably be addressed in scipy .
BTW, CuPy has its own scipy.fft backend ... no idea if this makes this problem better or worse.

It's important to note that SciPy does not implement NEP-18, nor intends to do so. The change in #6443 will prevent __array_function__ dispatching of FFTs when SciPy is available, meaning np.fft*(dask_cupy_array) would not work anymore. IMO, the solution would be to provide a way for the user to choose explicitly to use SciPy through some Dask configuration while maintaining NumPy as default even if SciPy is available.

EDIT: I had mistakenly written dask.array.fft*(cupy_array) but the correct is np.fft*(dask_cupy_array) (which I changed now), where dask_cupy_array is a Dask array backed by CuPy.