I was able to convert my pytorch model to the hailo format and run it on the Hailo8, but it is stuck at optimization level 0, because I cannot get the software to recognize my GPU. What my I be doing wrong?
nvcc –version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Tue_May_27_02:21:03_PDT_2025
Cuda compilation tools, release 12.9, V12.9.86
Build cuda_12.9.r12.9/compiler.36037853_0
I have the same issue running Hailo AI Software Suite Docker on Linux(hailo_virtualenv):
hailo@0a64a7a07cf2:~$ hailo --version
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1754321717.964957 6586 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1754321717.970170 6586 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
[info] Current Time: 18:35:20, 08/04/25
[info] CPU: Architecture: x86_64, Model: Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz, Number Of Cores: 64, Utilization: 0.0%
[info] Memory: Total: 125GB, Available: 121GB
[info] System info: OS: Linux, Kernel: 5.4.0-214-generic
[info] Hailo DFC Version: 3.32.0
[info] HailoRT Version: 4.22.0
[info] PCIe: No Hailo PCIe device was found
[info] Running hailo --version
HailoRT v4.22.0
Hailo Dataflow Compiler v3.32.0
While GPU is available, it looks like some issues with tensorflow initialization:
(hailo_virtualenv) hailo@0a64a7a07cf2:~$ python
Python 3.10.12 (main, May 27 2025, 17:12:29) [GCC 11.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.
import torch
torch.cuda.is_available()
True
torch.cuda.get_device_name(0)
‘Quadro RTX 4000’
import tensorflow as tf
2025-08-04 18:41:46.669334: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1754322106.687698 6805 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1754322106.693292 6805 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-08-04 18:41:46.711965: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
import tensorflow as tf
print(tf.config.list_physical_devices(‘GPU’))
[PhysicalDevice(name=‘/physical_device:GPU:0’, device_type=‘GPU’)]
I am still having trouble. As suggested in another thread, I tried running:
python -c “import tensorflow as tf; print(tf.config.list_physical_devices(‘GPU’))”
2025-08-13 14:56:54.589817: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0.
2025-08-13 14:56:54.719817: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1755089814.770142 204098 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1755089814.784753 204098 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-08-13 14:56:54.907385: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
[PhysicalDevice(name=‘/physical_device:GPU:0’, device_type=‘GPU’)]
So TensorFlow finds the GPU. Are the other warnings normal though?