NVIDIA TRX 5070 Ti support

General Discussion
1

Hello Folks,

I have Ubuntu 24.04 installed on the desktop with NVIDIA RTX 5070 Ti, installed Hailo Docker suite. While within the docker I perfectly see the GPU, Hailo compiler fails to recognize it and drops optimization level to Zero. What can be done about it if anyone can help?

2

Hi,

The following solution worked for me, however with regular installation (not Docker).
IMHO it should work also in your case.

The issue is that the compiler can’t find the CUDA libraries installed via pip - they’re not on LD_LIBRARY_PATH by default, so it falls back to optimization level 0.

Fix: Before running the compiler, add the pip-installed NVIDIA libs to your library path:

export LD_LIBRARY_PATH=$(python3 -c "
import os
paths = []
for pkg in ['cudnn','cublas','cuda_runtime','cufft','cusolver','cusparse','curand','nvjitlink','cuda_cupti']:
    try:
        mod = __import__(f'nvidia.{pkg}', fromlist=[pkg])
        p = os.path.join(os.path.dirname(mod.__file__), 'lib')
        if os.path.isdir(p): paths.append(p)
    except: pass
print(':'.join(paths))
"):${LD_LIBRARY_PATH:-}

export TF_FORCE_GPU_ALLOW_GROWTH=true
export CUDA_VISIBLE_DEVICES=0

Then run your hailo optimize command as usual. The GPU should be detected and full optimization will run.

Do not set CUDA_VISIBLE_DEVICES="" - an empty string disables the GPU entirely. Use 0 (or whichever GPU index you want).

Please let me know if this worked for you.

Thanks,

3

Thanks Much for your answer. Will try and let you know.

1 Like