/home/rkohli1/.cache/bazel/_bazel_rkohli1/235fe154e0a4c7e0c0527cd185fe6b6b/server/
.nfs00000000820050bd00000e9e (Device or resource busy).
At this point, I just tried deleting the entire .cache folder (I had to first kill a process which was preventing me from deleting it). I tried running configure with the --expunge_async flag as well but it doesn't help. It takes me back to the first error message.
Not sure if it's relevant, but I'm trying to install tensorflow with GPU support and use cuda 8.0 and cudNN 5
I raised this issue on stackoverflow (http://stackoverflow.com/questions/40144776/tensorflow-installation-error-directory-not-empty), and someone pointed out that it's due to a bug in bazel. Please advise me if I'm wrong.
No sure if this is correct, but after I make the following change in the configure file:
function bazel_clean_and_fetch() {
bazel clean --expunge currently doesn't work on Windows
TODO(pcloudy): Re-enable it after bazel clean --expunge is fixed.
if ! is_windows; then
#bazel clean --expunge
bazel clean --expunge_async
bazel fetch //tensorflow/...
I can install tensorflow 0.11 from source, with
bazel 0.3.1
cuDNN 5
Cuda 8.0
Wait there is multiple various issue collated to this one:
NFS mount point are known to be problematic. Use --output_base to direct the cache dir of bazel out of the NFS mount point.
This is a known issue on Windows and @meteorcloudy is working on fixing it IIUC.
Anyway rm the bazel cache should fix the issue all the time.
Closing this issue please reopen a specific one for your use cases if you are not on those case.
hi @damienmg, I encountered the same issue as yours. As you suggested, I used ./configure --output_base=/temp/cache_bazel, however, I still found below warnings during configuration:
WARNING: Output base '/home/AIJ/.cache/bazel/_bazel_AIJ/aa61f742fcd63eed03445cc6cf85534c' is on NFS. This may lead to surprising failures and undetermined behavior.
Does this mean the output base has not changed to my specified folder, i.e. /temp/cache_bazel? And what should I do to make the cache dir of bazel out to be local?
Thanks!
I was trying to build TF0.12 from source with bazel using NFS.
Neither of the suggestions from above worked for me:
editing .config file and adding --output_base did not work for fetch
everything from that thread resulted in the same NFS warning and issue with bazel's cache
This solution seems to be helping:
http://stackoverflow.com/questions/40144776/tensorflow-installation-error-directory-not-empty
Solution:
edit .config file and replace
bazel clean --expunge
bazel clean --expunge_async
I attempted both solutions suggest by @sfincke and @yselivonchyk, but without luck. Finally, I managed to change the cache location by running TEST_TMPDIR=/tmp/bazel/ ./configure, which solved the issue.
This global variable sets the overall cache directory as described here: https://bazel.build/versions/master/docs/output_directories.html.
tom-jin, fvisin, shatu, sdhdez, walkers-mv, snownus, hoavt-54, bvisness, sxhmilyoyo, rtp4jc, and 4 more reacted with thumbs up emoji
fvisin, shatu, walkers-mv, hoavt-54, karthikvadla, and vsahil reacted with hooray emoji
hoavt-54, rtp4jc, karthikvadla, and vsahil reacted with heart emoji
All reactions
Update,
I tried the same thing again: compile latest version of TF with bazel while using NFS file system after adjusting .config "bazel clean --expunge_async".
It did not work. After sometime at a random step server just hangs. The NFS consumes full network capacity while process is doing nothing. Most astonishingly, kill -9 does not help the process.
So, I would not recommend doint that on NFS unless you are free to restart your servers.
I tried some bazel commands to use custom cache location, but it did not work either.
