No GPU found. Dataflow-Compiler v3.32.0

Max_Weichselgartner · August 7, 2025, 5:17pm

I recently installed the DFC 3.32.0.

Nvidia-Version: 570.169
Cuda compilation tools, release 12.5, V12.5.82

libcudnn9-cuda-12/unbekannt,now 9.10.2.21-1 amd64 [Installiert,automatisch]
libcudnn9-dev-cuda-12/unbekannt,now 9.10.2.21-1 amd64 [installiert]
libcudnn9-headers-cuda-12/unbekannt,now 9.10.2.21-1 amd64 [Installiert,automatisch]
libcudnn9-samples/unbekannt,now 9.10.2.21-1 all [Installiert,automatisch]
libcudnn9-static-cuda-12/unbekannt,now 9.10.2.21-1 amd64 [Installiert,automatisch]

I always get the error:
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1754570628.655024 50375 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1754570628.657445 50375 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered

How can i fix it?

omria · August 12, 2025, 9:47am

Hey @Max_Weichselgartner ,

This isn’t actually an error that needs fixing. The Hailo DFC is designed to work primarily on CPU, with GPU only used for optional optimization during quantization and calibration stages.

Why you’re seeing these messages: The GPU warnings come from TensorFlow/CUDA in your host Python environment, not from DFC itself. DFC runs perfectly fine on CPU-only setups.

Option 1: Use Hailo’s official environment (recommended) Run DFC inside the official Hailo AI Software Suite container or SDK environment. This avoids CUDA/TensorFlow conflicts entirely.

Option 2: Isolate your host environment If you prefer your current setup:

Set CUDA_VISIBLE_DEVICES= (empty) to hide GPU from TensorFlow
Use a clean virtual environment with only Hailo packages (no tensorflow packages)

Option 3: Fix TensorFlow GPU conflicts If you need TensorFlow GPU for other tasks:

Ensure only one CUDA/cuDNN version is installed
Use a fresh environment with exactly one TensorFlow package
Test with: python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

kd178 · August 13, 2025, 5:29pm

I have the same issue which is reproducible on two different hosts with different GPUs: RTX 5080 and RTX 2060. It looks like it’s not picking up the CuDNN which is loaded and works fine (see the logs below). Please and advise how to debug this issue:
[info] No GPU chosen and no suitable GPU found, falling back to CPU

The recommended Option 1 doesn’t work neither for Hailo AI Software Suite docker container nor for Hailo AI Software Suite SDK environment (see the logs below)
Furthermore, the latest Docker container hailo8_ai_sw_suite_2025-07_docker includes on one hand CUDA 11.8 and on the other hand DFC which wants CUDA 12.5

Log from the hailo8_ai_sw_suite_2025-07_docker → (note the highlighted items)

Welcome to Hailo AI Software Suite Container
To list available commands, please type:

HailoRT: hailortcli -h
Dataflow Compiler: hailo -h
Hailo Model Zoo: hailomz -h
Run TAPPAS Detection Application: tappas/detection/detection.sh

*(hailo_virtualenv) hailo@ge46fox-Lenovo-Legion-5-15IMH05H:/local/workspace$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

±----------------------------------------------------------------------------------------+
(hailo_virtualenv) hailo@ge46fox-Lenovo-Legion-5-15IMH05H:/local/workspace$ hailo -h
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
[info] First time Hailo Dataflow Compiler is being used. Checking system requirements… (this might take a few seconds)
[Warning] It is recommended to have 32 GB of RAM, while this system has only 17 GB.
[Warning] CUDA version should be 12.5 or higher, found 11.8 .
[Warning] CUDNN version should be 9 or higher, found ..
Component Requirement Found
========== ========== ========== ==========
OS Ubuntu Ubuntu Required
Release 20.04 22.04 Required
Package python3-tk V Required
Package graphviz V Required
Package libgraphviz-dev V Required
Package python3.10-dev V Required
RAM(GB) 16 17 Required
RAM(GB) 32 17 Recommended
CPU-Arch x86_64 x86_64 Required
CPU-flag avx V Required
GPU-Driver 560 575 Recommended
CUDA 12.5 11.8 Recommended
CUDNN 9 . Recommended
Var:CC unset unset Required
Var:CXX unset unset Required
Var:LD unset unset Required
Var:AS unset unset Required
Var:AR unset unset Required
Var:LN unset unset Required
Var:DUMP unset unset Required
Var:CPY unset unset Required*

(hailo_venv) ge46fox@ge46fox-Lenovo-Legion-5-15IMH05H:~/hailo_ai_sw_suite$python3 -c “import tensorflow as tf;
print(‘TensorFlow version:’, tf.version);
print(‘Built with cuDNN:’, tf.test.is_built_with_cuda());
print(‘GPU available:’, tf.config.list_physical_devices(‘GPU’));
a = tf.random.normal([8, 224, 224, 3]);
b = tf.keras.layers.Conv2D(64, 3)(a);
print(‘Computation successful, output shape:’, b.shape)”
TensorFlow version: 2.18.0**
Built with cuDNN: True**
GPU available: [PhysicalDevice(name=‘/physical_device:GPU:0’, device_type=‘GPU’)]
I0000 00:00:1755105444.790273 16970 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 4156 MB memory: → device: 0, name: NVIDIA GeForce RTX 2060, pci bus id: 0000:01:00.0, compute capability: 7.5
I0000 00:00:1755105445.295004 16970 cuda_dnn.cc:529] Loaded cuDNN version 91200
Computation successful, output shape: (8, 222, 222, 64)

Log from the locally installed Hailo AI Software Suite SDK→ (note the highlighted items)

(hailo_venv) ge46fox@ge46fox-Lenovo-Legion-5-15IMH05H:~/hailo_ai_sw_suite$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Jun__6_02:18:23_PDT_2024
Cuda compilation tools, release 12.5, V12.5.82
Build cuda_12.5.r12.5/compiler.34385749_0

(hailo_venv) ge46fox@ge46fox-Lenovo-Legion-5-15IMH05H:~/hailo_ai_sw_suite$ python -c “import tensorflow as tf;
print(‘TensorFlow version:’, tf.version);
print(‘Built with cuDNN:’, tf.test.is_built_with_cuda());
print(‘GPU available:’, tf.config.list_physical_devices(‘GPU’));
a = tf.random.normal([8, 224, 224, 3]);
b = tf.keras.layers.Conv2D(64, 3)(a);
print(‘Computation successful, output shape:’, b.shape)”
2025-08-13 19:25:08.618908: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
TensorFlow version: 2.18.0
Built with cuDNN: True
GPU available: [PhysicalDevice(name=‘/physical_device:GPU:0’, device_type=‘GPU’)]
I0000 00:00:1755105910.239850 17334 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 4108 MB memory: → device: 0, name: NVIDIA GeForce RTX 2060, pci bus id: 0000:01:00.0, compute capability: 7.5
I0000 00:00:1755105910.735380 17334 cuda_dnn.cc:529] Loaded cuDNN version 91200
Computation successful, output shape: (8, 222, 222, 64)

(hailo_venv) ge46fox@ge46fox-Lenovo-Legion-5-15IMH05H:~/hailo_ai_sw_suite$ rm hailo_venv/etc/hailo/check_system_requirements_was_called
(hailo_venv) ge46fox@ge46fox-Lenovo-Legion-5-15IMH05H:~/hailo_ai_sw_suite$ hailo -h
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
[info] First time Hailo Dataflow Compiler is being used. Checking system requirements… (this might take a few seconds)
[Warning] It is recommended to have 32 GB of RAM, while this system has only 17 GB.
[Warning] CUDNN version should be 9 or higher, found ..
Component Requirement Found
========== ========== ========== ==========
OS Ubuntu Ubuntu Required
Release 20.04 22.04 Required
Package python3-tk V Required
Package graphviz V Required
Package libgraphviz-dev V Required
Package python3.10-dev V Required
RAM(GB) 16 17 Required
RAM(GB) 32 17 Recommended
CPU-Arch x86_64 x86_64 Required
CPU-flag avx V Required
GPU-Driver 560 575 Recommended
CUDA 12.5 12.5 Recommended
CUDNN 9 . Recommended
Var:CC unset unset Required
Var:CXX unset unset Required
Var:LD unset unset Required
Var:AS unset unset Required
Var:AR unset unset Required
Var:LN unset unset Required
Var:DUMP unset unset Required
Var:CPY unset unset Required