GPU used in the container environment

General
1

Hi Hailo Team:

This is my first to use Hailo8. My current task is to convert my own CNN model(.pb) to a model , which can be run on Halio8.

I installed the latest Hailo AI Software Suite(hailo8_ai_sw_suite_2025-07_container) from the container provided by your official site.
However, I noticed the container environment is unable to find the GPU as the following message.

=====================

(hailo_virtualenv) hailo@W790WS:/local/workspace$ hailo --version
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1758591797.626935 362 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1758591797.630903 362 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
[info] Current Time: 10:43:19, 09/23/25
[info] CPU: Architecture: x86_64, Model: Intel(R) Xeon(R) w3-2423, Number Of Cores: 12, Utilization: 0.6%
[info] Memory: Total: 62GB, Available: 58GB
[info] System info: OS: Linux, Kernel: 6.14.0-29-generic
[info] Hailo DFC Version: 3.32.0
[info] HailoRT Version: 4.22.0
[info] PCIe: No Hailo PCIe device was found
[info] Running hailo --version

==========================

and here are some information of my GPU

±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.82.07 Driver Version: 580.82.07 CUDA Version: 13.0 |
±----------------------------------------±-----------------------±---------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA RTX 2000 Ada Gene… On | 00000000:17:00.0 Off | Off |
| 30% 39C P2 17W / 70W | 522MiB / 16380MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
| 1 NVIDIA RTX 4000 Ada Gene… On | 00000000:85:00.0 Off | Off |
| 30% 32C P8 7W / 130W | 104MiB / 20475MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------

Since my GPU card is a little different from the officail recommended type, I guess that is why the container can not find the GPU.

Is there any way I can get my GPU found the container?Shall I reinstall the GPU dirver in the container environment?

It will be great helpful if anyone can provide some advice and thank you in adavance.

2

Welcome to the Hailo Community!

Have a look at the script that is used to start the container. There are tests that will decide whether the container is created with GPU support or not. You can run the command manually and check what messages you will get.

From what I remember this test runs only when a new container is created. You may need to delete the current container and create it again. You can keep the image installed so the process is quick.

3

Hi, KlausK

Is the following the script you mentioned ?

NvidiaScirpt
NvidiaScirpt1095×377 89.6 KB

4

I tried the script above, but it looks not work.

After running

hailo@W790WS:~/hailo8$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

I get the following the warning Message:
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
gpg: 有効なOpenPGPデータが見つかりません。

What shall I do next? or anything else I can do?
My final goal is to transfer my own CNN model to the model can be run by Hailo.

5

Here is some update.

After running running hailo -h, I got the following message from the system.

=================================

(hailo_virtualenv) hailo@khi-ml:/local/workspace$ hailo -h
[info] No GPU chosen, Selected GPU 0
[info] First time Hailo Dataflow Compiler is being used. Checking system requirements… (this might take a few seconds)
[Warning] GPU driver version should be or higher, found 535.
[Warning] CUDA version should be 12.5 or higher, found 11.8 .
[Warning] CUDNN version should be 9 or higher, found ..
Component Requirement Found
========== ========== ========== ==========
OS Ubuntu Ubuntu Required
Release 20.04 22.04 Required
Package python3-tk V Required
Package graphviz V Required
Package libgraphviz-dev V Required
Package python3.10-dev V Required
RAM(GB) 16 32 Required
RAM(GB) 32 32 Recommended
CPU-Arch x86_64 x86_64 Required
CPU-flag avx V Required
GPU-Driver 560 535 Recommended
CUDA 12.5 11.8 Recommended
CUDNN 9 . Recommended
Var:CC unset unset Required
Var:CXX unset unset Required
Var:LD unset unset Required
Var:AS unset unset Required
Var:AR unset unset Required
Var:LN unset unset Required
Var:DUMP unset unset Required
Var:CPY unset unset Required
[info] Current Time: 15:13:50, 10/31/25
[info] CPU: Architecture: x86_64, Model: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz, Number Of Cores: 32, Utilization: 0.1%
[info] Memory: Total: 30GB, Available: 24GB
[info] System info: OS: Linux, Kernel: 5.15.0-160-generic
[info] Hailo DFC Version: 3.33.0
[info] HailoRT Version: 4.23.0
[info] PCIe: No Hailo PCIe device was found
[info] Running hailo -h

=================================

My questions are

1). It looks that the container has detected my GPU, am I right?

2). According to the warning message, shall I update the driver of GPU? And now, I just installed driver, which is closest to the driver recommended on the docker installation instruction.