Nvidia-dali-tf-plugin

During optimization level 3 using Adaround, I’m getting the following warning:

Warning: DALI is not installed, using TensorFlow dataset for layer-by-layer training. Using DALI will significantly improve training time. To install it, use:
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist nvidia-dali-cuda110 nvidia-dali-tf-plugin-cuda110

I attempted to install DALI, but it failed because I first tried using the link in the warning, but it had changed. Then, I found this one on the Nvidia page:

pip install nvidia-dali-tf-plugin-cuda110

When I try to install it, I’m getting the following error:

Build from source failed with error: Command ‘[‘g++’, ‘-Wl,-R,$ORIGIN/…’, ‘-std=c++14’, ‘-DNDEBUG’, ‘-shared’, ‘/tmp/dali_stub_2pz0lpb6/dali_stub.cc’, ‘-o’, ‘/tmp/dali_stub_2pz0lpb6/libdali.so’, ‘-fPIC’, ‘-I/tmp/pip-install-5ntjmzsa/nvidia-dali-tf-plugin-cuda110_845d47d3b4434259a213986eeecfeba2/include’, ‘-I/home/admin/anaconda3/envs/conda_env/nvvm/include’, ‘-O2’]’ returned non-zero exit status 1.
Traceback (most recent call last):
File “”, line 2, in
File “”, line 34, in
File “/tmp/pip-install-5ntjmzsa/nvidia-dali-tf-plugin-cuda110_845d47d3b4434259a213986eeecfeba2/setup.py”, line 37, in
setup(name=‘nvidia-dali-tf-plugin-cuda110’,
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/init.py”, line 107, in setup
return distutils.core.setup(**attrs)
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/_distutils/core.py”, line 185, in setup
return run_commands(dist)
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/_distutils/core.py”, line 201, in run_commands
dist.run_commands()
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/_distutils/dist.py”, line 969, in run_commands
self.run_command(cmd)
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/dist.py”, line 1234, in run_command
super().run_command(command)
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/_distutils/dist.py”, line 988, in run_command
cmd_obj.run()
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/wheel/_bdist_wheel.py”, line 387, in run
self.run_command(“build”)
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/_distutils/cmd.py”, line 318, in run_command
self.distribution.run_command(command)
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/dist.py”, line 1234, in run_command
super().run_command(command)
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/_distutils/dist.py”, line 988, in run_command
cmd_obj.run()
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/_distutils/command/build.py”, line 131, in run
self.run_command(cmd_name)
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/_distutils/cmd.py”, line 318, in run_command
self.distribution.run_command(command)
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/dist.py”, line 1234, in run_command
super().run_command(command)
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/setuptools/_distutils/dist.py”, line 988, in run_command
cmd_obj.run()
File “/tmp/pip-install-5ntjmzsa/nvidia-dali-tf-plugin-cuda110_845d47d3b4434259a213986eeecfeba2/setup.py”, line 25, in run
helper.install()
File “/tmp/pip-install-5ntjmzsa/nvidia-dali-tf-plugin-cuda110_845d47d3b4434259a213986eeecfeba2/dali_tf_plugin_install_tool.py”, line 436, in install
raise e
File “/tmp/pip-install-5ntjmzsa/nvidia-dali-tf-plugin-cuda110_845d47d3b4434259a213986eeecfeba2/dali_tf_plugin_install_tool.py”, line 428, in install
self.build()
File “/tmp/pip-install-5ntjmzsa/nvidia-dali-tf-plugin-cuda110_845d47d3b4434259a213986eeecfeba2/dali_tf_plugin_install_tool.py”, line 356, in build
subprocess.check_call(cmd, cwd=self.src_path, shell=False) # nosec B603
File “/home/admin/anaconda3/envs/conda_env/lib/python3.10/subprocess.py”, line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command ‘[‘g++’, ‘-Wl,-R,$ORIGIN/…’, ‘-std=c++14’, ‘-DNDEBUG’, ‘-shared’, ‘/tmp/dali_stub_2pz0lpb6/dali_stub.cc’, ‘-o’, ‘/tmp/dali_stub_2pz0lpb6/libdali.so’, ‘-fPIC’, ‘-I/tmp/pip-install-5ntjmzsa/nvidia-dali-tf-plugin-cuda110_845d47d3b4434259a213986eeecfeba2/include’, ‘-I/home/admin/anaconda3/envs/conda_env/nvvm/include’, ‘-O2’]’ returned non-zero exit status 1.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for nvidia-dali-tf-plugin-cuda110
Running setup.py clean for nvidia-dali-tf-plugin-cuda110
Failed to build nvidia-dali-tf-plugin-cuda110
ERROR: Failed to build installable wheels for some pyproject.toml based projects (nvidia-dali-tf-plugin-cuda110)

Has anyone encountered this issue or have any suggestions on how to resolve it?

I was able to resolve the problem using the following command:

CPATH=/home/admin/anaconda3/envs/conda_env/lib/python3.10/site-packages/tensorflow/include/third_party/gpus/cuda/include pip install --extra-index-url https://pypi.nvidia.com nvidia-dali-tf-plugin-cuda110

However, this installs the latest version of nvidia-dali-tf, which is not compatible with TensorFlow 2.12.

The URL https://developer.download.nvidia.com/compute/redist should contain legacy versions, but the files are no longer there.