[info] No GPU chosen and no suitable GPU found, falling back to CPU

Hello Hailo Community,

I was able to convert my pytorch model to the hailo format and run it on the Hailo8, but it is stuck at optimization level 0, because I cannot get the software to recognize my GPU. What my I be doing wrong?

nvcc –version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Tue_May_27_02:21:03_PDT_2025
Cuda compilation tools, release 12.9, V12.9.86
Build cuda_12.9.r12.9/compiler.36037853_0

nvidia-smi
Fri Aug 1 08:11:31 2025
±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 575.64.03 Driver Version: 575.64.03 CUDA Version: 12.9 |
|-----------------------------------------±-----------------------±---------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 3080 Off | 00000000:01:00.0 On | N/A |
| 30% 35C P3 65W / 320W | 4100MiB / 10240MiB | 8% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+

echo $LD_LIBRARY_PATH
/usr/local/cuda-12.9/lib64:

lspci | grep -i nvidia
0000:01:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3080 Lite Hash Rate] (rev a1)
0000:01:00.1 Audio device: NVIDIA Corporation GA102 High Definition Audio Controller (rev a1)

Thanks,

John

What did you use: Full Software Kit or the Docker version? Just DFC?

Hey @user226 ,

Welcome to the Hailo Community!

Are you running on windows or linux ? docker or DFC or ai suite or Model zoo ?

I have the same issue running Hailo AI Software Suite Docker on Linux(hailo_virtualenv):

hailo@0a64a7a07cf2:~$ hailo --version
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1754321717.964957 6586 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1754321717.970170 6586 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
[info] Current Time: 18:35:20, 08/04/25
[info] CPU: Architecture: x86_64, Model: Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz, Number Of Cores: 64, Utilization: 0.0%
[info] Memory: Total: 125GB, Available: 121GB
[info] System info: OS: Linux, Kernel: 5.4.0-214-generic
[info] Hailo DFC Version: 3.32.0
[info] HailoRT Version: 4.22.0
[info] PCIe: No Hailo PCIe device was found
[info] Running hailo --version
HailoRT v4.22.0
Hailo Dataflow Compiler v3.32.0

While GPU is available, it looks like some issues with tensorflow initialization:

(hailo_virtualenv) hailo@0a64a7a07cf2:~$ python
Python 3.10.12 (main, May 27 2025, 17:12:29) [GCC 11.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import torch
torch.cuda.is_available()
True
torch.cuda.get_device_name(0)
‘Quadro RTX 4000’
import tensorflow as tf
2025-08-04 18:41:46.669334: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1754322106.687698 6805 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1754322106.693292 6805 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-08-04 18:41:46.711965: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
import tensorflow as tf
print(tf.config.list_physical_devices(‘GPU’))
[PhysicalDevice(name=‘/physical_device:GPU:0’, device_type=‘GPU’)]