import bz2
gives error:
Traceback (most recent call last):
import bz2
File "/usr/local/lib/python3.8/bz2.py", line 19, in <module>
    from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'
Explanation#
Python installation was built from source but didn’t have all necessary system packages installed for full Python build.
Work around#
Install bzip2-devel and sqlite-devel and rebuild Python, or download a pre-built Python binary.
ModuleNotFoundError: No module named <…>#
Users may experience a ModuleNotFoundError for some Python packages that are available on their local machine.
Explanation#
The Custom Worker Container Workflow provides seamless support for importing user-specific dependency packages in the Python environments into the Cerebras appliances. In case of failure, a fallback policy has also been enabled to mount the site packages from the user node Python virtual environment to the worker environment via a predefined NFS-based cluster volume.
Work around#
If you observe a ModuleNotFoundError, you can disable the Custom Worker Container Workflow by following the instructions here, and follow the R1.8 approach:
pip install the packages in virtual environment
Copy the custom package directory from venv/lib/python3.8/site-packages/<package_name> to a NFS-mountable location for the Cerebras cluster. Only copy the custom packages, not the entire virtual environment.
Make sure to add this location to the --mount_dirs command line argument and the corresponding parent location to the --python_paths command line argument when calling run.py.