I have an issue with using a GPU, which is required for optimizing tasks by the Hailo SW Suite. In short, Hailo SW Suite is not picking up the CuDNN (which is loaded and works fine) and generates this message:
[info] No GPU chosen and no suitable GPU found, falling back to CPU
(see detailed logs below)
I can reproducible this issue on two different hosts with different GPUs: RTX 5080 and RTX 2060.
-
It doesn’t work for Hailo AI Software Suite docker container, neither for Hailo AI Software Suite SDK environment
-
Furthermore, the latest Docker container hailo8_ai_sw_suite_2025-07_docker includes, on one hand, CUDA 11.8 and, on the other hand, DFC wants CUDA 12.5 (see the logs below)
I already reported this issue in 2 similar threads, but those threads became inactive, so bringing it up here as a stand-alone topic.
Log from the hailo8_ai_sw_suite_2025-07_docker → (note the highlighted items)
Welcome to Hailo AI Software Suite Container
To list available commands, please type:
HailoRT: hailortcli -h
Dataflow Compiler: hailo -h
Hailo Model Zoo: hailomz -h
Run TAPPAS Detection Application: tappas/detection/detection.sh
*(hailo_virtualenv) hailo@ge46fox-Lenovo-Legion-5-15IMH05H:/local/workspace$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
±----------------------------------------------------------------------------------------+
(hailo_virtualenv) hailo@ge46fox-Lenovo-Legion-5-15IMH05H:/local/workspace$ hailo -h
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
[info] First time Hailo Dataflow Compiler is being used. Checking system requirements… (this might take a few seconds)
[Warning] It is recommended to have 32 GB of RAM, while this system has only 17 GB.
[Warning] CUDA version should be 12.5 or higher, found 11.8 .
[Warning] CUDNN version should be 9 or higher, found ..
Component Requirement Found
========== ========== ========== ==========
OS Ubuntu Ubuntu Required
Release 20.04 22.04 Required
Package python3-tk V Required
Package graphviz V Required
Package libgraphviz-dev V Required
Package python3.10-dev V Required
RAM(GB) 16 17 Required
RAM(GB) 32 17 Recommended
CPU-Arch x86_64 x86_64 Required
CPU-flag avx V Required
GPU-Driver 560 575 Recommended
CUDA 12.5 11.8 Recommended
CUDNN 9 . Recommended
Var:CC unset unset Required
Var:CXX unset unset Required
Var:LD unset unset Required
Var:AS unset unset Required
Var:AR unset unset Required
Var:LN unset unset Required
Var:DUMP unset unset Required
Var:CPY unset unset Required*
(hailo_venv) ge46fox@ge46fox-Lenovo-Legion-5-15IMH05H:~/hailo_ai_sw_suite$python3 -c “import tensorflow as tf;
print(‘TensorFlow version:’, tf.version);
print(‘Built with cuDNN:’, tf.test.is_built_with_cuda());
print(‘GPU available:’, tf.config.list_physical_devices(‘GPU’));
a = tf.random.normal([8, 224, 224, 3]);
b = tf.keras.layers.Conv2D(64, 3)(a);
print(‘Computation successful, output shape:’, b.shape)”
TensorFlow version: 2.18.0**
Built with cuDNN: True**
GPU available: [PhysicalDevice(name=‘/physical_device:GPU:0’, device_type=‘GPU’)]
I0000 00:00:1755105444.790273 16970 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 4156 MB memory: → device: 0, name: NVIDIA GeForce RTX 2060, pci bus id: 0000:01:00.0, compute capability: 7.5
I0000 00:00:1755105445.295004 16970 cuda_dnn.cc:529] Loaded cuDNN version 91200
Computation successful, output shape: (8, 222, 222, 64)
Log from the locally installed Hailo AI Software Suite SDK→ (note the highlighted items)
(hailo_venv) ge46fox@ge46fox-Lenovo-Legion-5-15IMH05H:~/hailo_ai_sw_suite$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Jun__6_02:18:23_PDT_2024
Cuda compilation tools, release 12.5, V12.5.82
Build cuda_12.5.r12.5/compiler.34385749_0
(hailo_venv) ge46fox@ge46fox-Lenovo-Legion-5-15IMH05H:~/hailo_ai_sw_suite$ python -c “import tensorflow as tf;
print(‘TensorFlow version:’, tf.version);
print(‘Built with cuDNN:’, tf.test.is_built_with_cuda());
print(‘GPU available:’, tf.config.list_physical_devices(‘GPU’));
a = tf.random.normal([8, 224, 224, 3]);
b = tf.keras.layers.Conv2D(64, 3)(a);
print(‘Computation successful, output shape:’, b.shape)”
2025-08-13 19:25:08.618908: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
TensorFlow version: 2.18.0
Built with cuDNN: True
GPU available: [PhysicalDevice(name=‘/physical_device:GPU:0’, device_type=‘GPU’)]
I0000 00:00:1755105910.239850 17334 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 4108 MB memory: → device: 0, name: NVIDIA GeForce RTX 2060, pci bus id: 0000:01:00.0, compute capability: 7.5
I0000 00:00:1755105910.735380 17334 cuda_dnn.cc:529] Loaded cuDNN version 91200
Computation successful, output shape: (8, 222, 222, 64)
(hailo_venv) ge46fox@ge46fox-Lenovo-Legion-5-15IMH05H:~/hailo_ai_sw_suite$ rm hailo_venv/etc/hailo/check_system_requirements_was_called
(hailo_venv) ge46fox@ge46fox-Lenovo-Legion-5-15IMH05H:~/hailo_ai_sw_suite$ hailo -h
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
[info] First time Hailo Dataflow Compiler is being used. Checking system requirements… (this might take a few seconds)
[Warning] It is recommended to have 32 GB of RAM, while this system has only 17 GB.
[Warning] CUDNN version should be 9 or higher, found ..
Component Requirement Found
========== ========== ========== ==========
OS Ubuntu Ubuntu Required
Release 20.04 22.04 Required
Package python3-tk V Required
Package graphviz V Required
Package libgraphviz-dev V Required
Package python3.10-dev V Required
RAM(GB) 16 17 Required
RAM(GB) 32 17 Recommended
CPU-Arch x86_64 x86_64 Required
CPU-flag avx V Required
GPU-Driver 560 575 Recommended
CUDA 12.5 12.5 Recommended
CUDNN 9 . Recommended
Var:CC unset unset Required
Var:CXX unset unset Required
Var:LD unset unset Required
Var:AS unset unset Required
Var:AR unset unset Required
Var:LN unset unset Required
Var:DUMP unset unset Required
Var:CPY unset unset Required