hailomz not using full calibration set (64/877) or NVIDIA GPU for optimization

General
,
1

hello, hailo.

I am trying to compile a custom-trained yolov8s model for the Hailo-8L target, but I’m facing issues with calibration and optimization.

Hardware Setup:

  1. Hardware: Ubuntu 22.04 Desktop with RTX 4080
  2. SDK/Compiler: DFC (Dataflow Compiler) v3.33.0
  3. Model Zoo: HMZ (Hailo Model Zoo) v2.17 (Checked out from Git)
  4. Target: Hailo-8L, Rpi5 and AI HAT+ 13TOPS

Goal:

Compiling a custom-trained yolov8s model (3 classes, fire/normal/smoke) using my own .onnx file.

The Problems:

  • I have 877 images in my --calib-path directory, but the compiler only used 64 images. (How can I force it to use all 877?)

[info] No GPU chosen and no suitable GPU found, falling back to CPU.

  • The optimization process used the CPU (“No GPU chosen…”), even though I have an RTX 4080 with CUDA/CUDNN installed.

    [warning] Reducing optimization level to 0 (the accuracy won’t be optimized and compression won’t be used) because there’s less data than the recommended amount (1024), and there’s no available GPU
    [warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results
    [info] Model received quantization params from the hn
    [info] MatmulDecompose skipped
    [info] Starting Mixed Precision
    [info] Model Optimization Algorithm Mixed Precision is done (completion time is 00:00:00.23)
    [info] LayerNorm Decomposition skipped
    [info] Starting Statistics Collector
    [info] Using dataset with 64 entries for calibration
    Calibration: 100%|█████████████████████████████████████████████████████████████████████████████| 64/64 [00:19<00:00, 3.34entries/s]

  • Because of these issues, the compiler reduced the optimization_level to 0, and I am worried about the final accuracy.

i’m from S.Korea. i’m using translator, I ask for your understanding to my topics. thax for answer.

2

Welcome to the Hailo Community!

Your issue is the GPU installation. We will need to fix this first.

Can you please confirm what GPU driver, CUDA and CUDNN version you installed?

Please also check the documentation:

  • Hailo Dataflow Compiler User Guide - Dataflow Compiler Installation - System Requirements when you install the Hailo Dataflow Compiler
  • Hailo AI Software Suite User Guide - Docker installation - System requirements when you use the Hailo AI Software Suite Docker (my recommendation, as this makes the installation and updates easier)
3

thanks for answer.

nvidia-smi

Failed to initialize NVML: Driver/library version mismatch
NVML library version: 580.105

nvcc –version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Wed_Aug_14_10:10:22_PDT_2024
Cuda compilation tools, release 12.6, V12.6.68
Build cuda_12.6.r12.6/compiler.34714021_0

ls -l /usr/local | grep cuda

lrwxrwxrwx 1 root root 22 Oct 7 2024 cuda → /etc/alternatives/cuda
lrwxrwxrwx 1 root root 25 Oct 7 2024 cuda-11 → /etc/alternatives/cuda-11
drwxr-xr-x 14 root root 4096 Oct 7 2024 cuda-11.4
lrwxrwxrwx 1 root root 25 Dec 23 2024 cuda-12 → /etc/alternatives/cuda-12
drwxr-xr-x 15 root root 4096 Dec 23 2024 cuda-12.6

ls /usr/local/cuda/include | grep cudnn (no result)

ls /usr/local/cuda/lib64 | grep cudnn (no result)

It appears that cudnn is not installed on the system. what can i do?

4

You’re seeing this error because the NVIDIA kernel driver loaded in Ubuntu doesn’t match the NVML library version installed (580.105). The fix is usually to reinstall the correct driver so the kernel module and user-space libraries match. I recommend opening ChatGPT and stepping through the troubleshooting interactively. It can guide you based on your exact driver versions, error messages, and system state. It will help you verify the loaded driver, check for secure-boot issues, and reinstall the right version if needed.