By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Describe the issue:

I have an array of float numbers. Numpy fails to calculate the standard deviation while the manual calculation is okay. There is no nan or inf in the array.

If default parameters ( ddof=0 ) are used, no warning is shown, but the warning appears if ddof=1 is used.

The array can be downloaded from here .

Reproduce the code example:

np.std(arr, ddof=1)

Error message:

Actually, it is a warning:
C:\Users\***\miniconda3\envs\py38\lib\site-packages\numpy\core\_methods.py:212: RuntimeWarning: overflow encountered in reduce
  arrmean = umr_sum(arr, axis, dtype, keepdims=True, where=where)

This is happening because the array has the dtype float16. Switching to float32 stops the overflow.

I converted my array to float16 to save memory, but why that happens?
That's weird.

This is expected behaviour, and the limits of float16 are why others have proposed using bfloat16 which has a larger range than float16 at the expense of accuracy. Unfortunately numpy does not currently support bfloat16 natively, we await a proof of concept outside numpy and then will consider merging it.

Mean does upcast float16, there is an old issue about whether std could do that also. The numerical limits are thus expected in general, but we could work around for it specifically for std (I admit, I wonder a bit where else we would need to). That issue is gh-8237 .

Closing as duplicate of gh-8237 .