Here are a few lines of sanity checks from outside the Docker container, confirming that both the NVIDIA driver and nvidia-docker2 are correctly installed and visible:
~$ nvidia-smi -q | grep “Driver Version” | awk ‘{print $4}’
580.95.05
~$ lspci | grep “VGA compatible controller: NVIDIA”
01:00.0 VGA compatible controller: NVIDIA Corporation Device 2d59 (rev a1)
~$ dpkg -l | grep ‘nvidia-docker|nvidia-container-toolkit’
ii nvidia-container-toolkit 1.13.5-1 amd64 NVIDIA Container toolkit
ii nvidia-container-toolkit-base 1.13.5-1 amd64 NVIDIA Container Toolkit Base
ii nvidia-docker2 2.13.0-1 all nvidia-docker CLI wrapper
Inside the HAILO AI Suite container, nvidia-smi does see the GPU:
local/workspace$ nvidia-smi
Fri Oct 24 11:02:46 2025
±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05 Driver Version: 580.95.05 CUDA Version: 13.0 |
±----------------------------------------±-----------------------±---------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 5060 … Off | 00000000:01:00.0 On | N/A |
| N/A 36C P8 7W / 50W | 655MiB / 8151MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
±----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
±----------------------------------------------------------------------------------------+
But hailomz compile doesn’t detect a usable CUDA device
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
I found that to resolve the issue I may need to downgrading my GPU driver version to 535.183: No available GPU in Hailo Software Suite Docker
However, since I have RTX 5060, I can install only 570 version.
Looking forward for advice.