GPU is not used

General Discussion
1

The problem is while compiling the gpus are being detected but optimization is set to 0

[info] No GPU chosen, Selected GPU 1
Start run for network yolov8n_seg …
Initializing the hailo8 runner…
[info] Translation started on ONNX model yolov8n_seg
[info] Restored ONNX model yolov8n_seg (completion time: 00:00:00.12)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.04)
[info] Start nodes mapped from original model: ‘images’: ‘yolov8n_seg/input_layer1’.
[info] End nodes mapped from original model: ‘/model.22/cv2.2/cv2.2.2/Conv’, ‘/model.22/cv3.2/cv3.2.2/Conv’, ‘/model.22/cv4.2/cv4.2.2/Conv’, ‘/model.22/cv2.1/cv2.1.2/Conv’, ‘/model.22/cv3.1/cv3.1.2/Conv’, ‘/model.22/cv4.1/cv4.1.2/Conv’, ‘/model.22/cv2.0/cv2.0.2/Conv’, ‘/model.22/cv3.0/cv3.0.2/Conv’, ‘/model.22/cv4.0/cv4.0.2/Conv’, ‘/model.22/proto/cv3/act/Mul’.
[info] Translation completed on ONNX model yolov8n_seg (completion time: 00:00:02.26)
[info] Saved HAR to: /home/aftondil/hailo/yolov8n_seg.har
Using generic alls script found in /home/aftondil/miniconda3/envs/yolo/lib/python3.11/site-packages/hailo_model_zoo/cfg/alls/generic/yolov8n_seg.alls because there is no specific hardware alls
Preparing calibration data…
[info] Loading model script commands to yolov8n_seg from /home/aftondil/miniconda3/envs/yolo/lib/python3.11/site-packages/hailo_model_zoo/cfg/alls/generic/yolov8n_seg.alls
Ignoring classes parameter since the model has no NMS post-process.
[info] Appending model script commands to yolov8n_seg from string
[info] Found model with 3 input channels, using real RGB images for calibration instead of sampling random data.
[info] Starting Model Optimization
[warning] Reducing optimization level to 0 (the accuracy won’t be optimized and compression won’t be used) because there’s no available GPU
[warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results

2

Hey @Aftondil_Idiyev,

Welcome to the Hailo Community!

So you’re seeing that warning about optimization levels and GPU availability. The thing is, even though your system is detecting a GPU, it’s not actually able to use it for the optimization process—which is a bit different from just having it show up in your system.

Here’s what’s probably happening: the GPU might already be running something else, it could be misconfigured, or it might not be compatible with how the optimization pipeline works. That’s why the optimization level drops to 0.

To help you figure this out, could you share a few details?

  1. What version of DFC are you running?
  2. Can you run nvcc --version and let me know what CUDA and NVIDIA versions come back?
  3. Also, could you run nvidia-smi and share the output?
  4. What GPU are you using ?

That should help me pinpoint what’s going on!

3

  1. v3.29.0

  2. nvcc: NVIDIA (R) Cuda compiler driver
    Copyright (c) 2005-2023 NVIDIA Corporation
    Built on Fri_Jan__6_16:45:21_PST_2023
    Cuda compilation tools, release 12.0, V12.0.140
    Build cuda_12.0.r12.0/compiler.32267302_0

  3. ±----------------------------------------------------------------------------------------+
    | NVIDIA-SMI 570.195.03 Driver Version: 570.195.03 CUDA Version: 12.8 |
    |-----------------------------------------±-----------------------±---------------------+
    | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
    | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
    | | | MIG M. |
    |=========================================+========================+======================|
    | 0 NVIDIA GeForce RTX 3060 Ti Off | 00000000:01:00.0 Off | N/A |
    | 0% 22C P8 18W / 200W | 1MiB / 8192MiB | 0% Default |
    | | | N/A |
    ±----------------------------------------±-----------------------±---------------------+
    | 1 NVIDIA GeForce RTX 3060 Ti Off | 00000000:02:00.0 Off | N/A |
    | 0% 22C P8 18W / 200W | 1MiB / 8192MiB | 0% Default |
    | | | N/A |
    ±----------------------------------------±-----------------------±---------------------+
    | 2 NVIDIA GeForce RTX 3060 Ti Off | 00000000:06:00.0 Off | N/A |
    | 0% 23C P8 13W / 200W | 1MiB / 8192MiB | 0% Default |
    | | | N/A |
    ±----------------------------------------±-----------------------±---------------------+
    | 3 NVIDIA GeForce RTX 3060 Ti Off | 00000000:08:00.0 Off | N/A |
    | 0% 25C P8 16W / 200W | 1MiB / 8192MiB | 0% Default |
    | | | N/A |
    ±----------------------------------------±-----------------------±---------------------+

    ±----------------------------------------------------------------------------------------+
    | Processes: |
    | GPU GI CI PID Type Process name GPU Memory |
    | ID ID Usage |
    |=========================================================================================|
    | No running processes found |
    ±----------------------------------------------------------------------------------------+

  4. rtx3060 ti