No available GPU in Hailo Software Suite Docker

General
1

Hello,

I am trying to compile a retrained object detection model using hailomz compile, and I am unable to utilize my NVIDIA RTX 3090 GPU in the process. I am getting the following warnings:

[warning] Reducing optimization level to 0 (the accuracy won't be optimized and compression won't be used) because there's no available GPU
[warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results

I installed the Software Suite Docker according to the documentation. Below are some commands I’ve tried from other threads that might aid diagnosing the problem:

$ nvidia-smi -q | grep "Driver Version" | awk '{print $4}'
550.142

$ lspci | grep "VGA compatible controller: NVIDIA"
01:00.0 VGA compatible controller: NVIDIA Corporation Device 2204 (rev a1)

$ dpkg -l | grep 'nvidia-docker\|nvidia-container-toolkit'
ii  nvidia-container-toolkit                   1.17.3-1                              amd64        NVIDIA Container toolkit
ii  nvidia-container-toolkit-base              1.17.3-1                              amd64        NVIDIA Container Toolkit Base
ii  nvidia-docker2                             2.14.0-1                              all          NVIDIA Container Toolkit meta-package

Please let me know if you have any ideas as to how it might be solved. Thanks!

2

Hey @eren,

Welcome to the Hailo Community!

It seems there might be a mismatch between the versions you’re using and the supported ones. I recommend checking out this topic for more details:
GPU Not Detected

Let us know if you need further assistance!

3

Hey @omria ,

Thanks for the reply! I’ve seen that topic earlier and I’m not sure if it applies to my case. I am using the docker environment provided at the Hailo Software Downloads page. When I do nvidia-smi inside the docker environment, I successfully get 550.142 driver version as well as CUDA 11.8 version. Moreover, I tried installing CuDNN 8.9 inside the docker, but it did not change the outcome either. I also tried using the docker image version 2024-10 and 2024-7, but I keep getting the same error.

Thanks for the help in advance!

4

Update: I was able to resolve the issue through downgrading my GPU driver version to 535.183. Interestingly, the 2024-10 docker image currently displays a CUDA version of 12.2 instead of 11.8, but regardless, it works!