Compatibility of Hailo AI Suite and NVIDIA 580-open driver

General
1

Hi everyone,

I’m setting up the Hailo AI Software Suite on Ubuntu 22.04 LTS and have a NVIDIA GeForce RTX 5060 Laptop GPU.
The documentation recommends driver 525, CUDA 11.8, and cuDNN 8.9, but on my system neither 525 nor 535 (nor the proprietary 580 package) load properly — nvidia-smi consistently returns “No devices were found”. After extensive testing, the only driver that correctly detects and initializes the GPU is 580-open (nvidia-driver-580-open), which uses NVIDIA’s open-kernel modules.
I’m trying to confirm whether newer driver versions — particularly the 580-open driver series — are fully compatible with the Suite.
Thank you.

2

Hi everyone,

Following up. No this is my turn to say “No GPU chosen and no suitable GPU found, falling back to CPU.“

Has anyone successfully run the HAILO AI Suite Docker on Ubuntu 22.04 with the 580-open driver?
Could the open-kernel NVIDIA drivers be incompatible with CUDA inside the container, even when --gpus all is specified?
Any suggestions on how to make HAILO detect the GPU would be hugely appreciated.

Thanks in advance!

I have a fresh Ubuntu 22.04 LTS installation on a laptop with an NVIDIA GeForce RTX 5060 GPU and 32 GB RAM.

Here’s what I’ve done so far:

  1. Installed nvidia-driver-580-open (the only driver that correctly detects the GPU).

  2. Installed the latest Docker CE.

  3. Set up nvidia-docker2 following the official instructions (https://hailo.ai/developer-zone/documentation/hailo-sw-suite-2025-10-for-hailo-8-8l/?sp_referrer=suite/suite_install.html).

  4. Installed and ran the HAILO AI Software Suite Docker.

When launching, I get this message:
Starting new container
Running Hailo AI SW Suite Docker image with the following command:
docker run

I have 633 images from the training dataset. Then I converted my trained yolov11l.pt to .onnx and ran

hailomz compile \
  --ckpt /local/shared_with_docker/best.onnx \
  --calib-path /local/shared_with_docker/calib_output2/calib \
  --yaml hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov11l.yaml \
  --classes 1 \
  --hw-arch hailo8

Unfortunately, the compile process starts but immediately prints:
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
[info] First time Hailo Dataflow Compiler is being used. Checking system requirements… (this might take a few seconds)
[Warning] CUDA version should be 12.5 or higher, found 11.8 .
[Warning] CUDNN version should be 9 or higher, found ..
Component Requirement Found
========== ========== ========== ==========
OS Ubuntu Ubuntu Required
Release 20.04 22.04 Required
Package python3-tk V Required
Package graphviz V Required
Package libgraphviz-dev V Required
Package python3.10-dev V Required
RAM(GB) 16 32 Required
RAM(GB) 32 32 Recommended
CPU-Arch x86_64 x86_64 Required
CPU-flag avx V Required
GPU-Driver 560 580 Recommended
CUDA 12.5 11.8 Recommended
CUDNN 9 . Recommended
Var:CC unset unset Required
Var:CXX unset unset Required
Var:LD unset unset Required
Var:AS unset unset Required
Var:AR unset unset Required
Var:LN unset unset Required
Var:DUMP unset unset Required
Var:CPY unset unset Required
In file included from /local/workspace/hailo_virtualenv/lib/python3.10/site-packages/numpy/core/include/numpy/ndarraytypes.h:1929,
from /local/workspace/hailo_virtualenv/lib/python3.10/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /local/workspace/hailo_virtualenv/lib/python3.10/site-packages/numpy/core/include/numpy/arrayobject.h:5,
from /home/hailo/.pyxbld/temp.linux-x86_64-cpython-310/local/workspace/hailo_model_zoo/hailo_model_zoo/core/postprocessing/cython_utils/cython_nms.c:1145:
/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " “#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION” [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with "
| ^~~~~~~
Start run for network yolov11l …
Initializing the hailo8 runner…
[info] Translation started on ONNX model yolov11l
[info] Restored ONNX model yolov11l (completion time: 00:00:00.27)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.10)
[info] NMS structure of yolov8 (or equivalent architecture) was detected.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.23/cv2.0/cv2.0.2/Conv /model.23/cv3.0/cv3.0.2/Conv /model.23/cv2.1/cv2.1.2/Conv /model.23/cv3.1/cv3.1.2/Conv /model.23/cv2.2/cv2.2.2/Conv /model.23/cv3.2/cv3.2.2/Conv.
[info] Start nodes mapped from original model: ‘images’: ‘yolov11l/input_layer1’.
[info] End nodes mapped from original model: ‘/model.23/cv2.0/cv2.0.2/Conv’, ‘/model.23/cv3.0/cv3.0.2/Conv’, ‘/model.23/cv2.1/cv2.1.2/Conv’, ‘/model.23/cv3.1/cv3.1.2/Conv’, ‘/model.23/cv2.2/cv2.2.2/Conv’, ‘/model.23/cv3.2/cv3.2.2/Conv’.
[info] Translation completed on ONNX model yolov11l (completion time: 00:00:02.42)
[info] Appending model script commands to yolov11l from string
[info] Added nms postprocess command to model script.
[info] Saved HAR to: /local/workspace/yolov11l.har
Preparing calibration data…
[info] Loading model script commands to yolov11l from /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov11l.alls
[info] Loading model script commands to yolov11l from string
[info] Found model with 3 input channels, using real RGB images for calibration instead of sampling random data.
[info] Starting Model Optimization
[warning] Reducing optimization level to 0 (the accuracy won’t be optimized and compression won’t be used) because there’s less data than the recommended amount (1024), and there’s no available GPU
[warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results
[info] Model received quantization params from the hn
[info] MatmulDecompose skipped
[info] Starting Mixed Precision
[info] Model Optimization Algorithm Mixed Precision is done (completion time is 00:00:00.85)
[info] LayerNorm Decomposition skipped
[info] Starting Statistics Collector
[info] Using dataset with 64 entries for calibration
Calibration: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [00:54<00:00, 1.18entries/s]
[info] Model Optimization Algorithm Statistics Collector is done (completion time is 00:00:55.36)
[info] Starting Fix zp_comp Encoding
[info] Model Optimization Algorithm Fix zp_comp Encoding is done (completion time is 00:00:00.00)
[info] Starting Matmul Equalization
[info] Model Optimization Algorithm Matmul Equalization is done (completion time is 00:00:00.07)
[info] Starting MatmulDecomposeFix
[info] Model Optimization Algorithm MatmulDecomposeFix is done (completion time is 00:00:00.00)
[info] No shifts available for layer yolov11l/conv42/conv_op, using max shift instead. delta=0.3988
[info] No shifts available for layer yolov11l/conv42/conv_op, using max shift instead. delta=0.1994
[info] No shifts available for layer yolov11l/conv43/conv_op, using max shift instead. delta=0.0822
[info] No shifts available for layer yolov11l/conv43/conv_op, using max shift instead. delta=0.0411
[info] No shifts available for layer yolov11l/conv42/conv_op, using max shift instead. delta=0.1994
[info] No shifts available for layer yolov11l/conv53/conv_op, using max shift instead. delta=0.0415
[info] No shifts available for layer yolov11l/conv53/conv_op, using max shift instead. delta=0.0207
[info] No shifts available for layer yolov11l/conv58/conv_op, using max shift instead. delta=0.1869
[info] No shifts available for layer yolov11l/conv58/conv_op, using max shift instead. delta=0.0934
[info] No shifts available for layer yolov11l/conv59/conv_op, using max shift instead. delta=0.3181
[info] No shifts available for layer yolov11l/conv59/conv_op, using max shift instead. delta=0.1590
[info] No shifts available for layer yolov11l/conv65/conv_op, using max shift instead. delta=0.0579
[info] No shifts available for layer yolov11l/conv65/conv_op, using max shift instead. delta=0.0289
[info] No shifts available for layer yolov11l/conv66/conv_op, using max shift instead. delta=0.7598
[info] No shifts available for layer yolov11l/conv66/conv_op, using max shift instead. delta=0.3799
[info] No shifts available for layer yolov11l/conv67/conv_op, using max shift instead. delta=0.3872
[info] No shifts available for layer yolov11l/conv67/conv_op, using max shift instead. delta=0.1936
[info] No shifts available for layer yolov11l/conv66/conv_op, using max shift instead. delta=0.3799
[info] activation fitting started for yolov11l/reduce_sum_softmax1/act_op
[info] No shifts available for layer yolov11l/conv74/conv_op, using max shift instead. delta=0.8335
[info] No shifts available for layer yolov11l/conv74/conv_op, using max shift instead. delta=0.4167
[info] No shifts available for layer yolov11l/conv76/conv_op, using max shift instead. delta=1.2689
[info] No shifts available for layer yolov11l/conv76/conv_op, using max shift instead. delta=0.6345
[info] No shifts available for layer yolov11l/conv74/conv_op, using max shift instead. delta=0.4167
[info] activation fitting started for yolov11l/reduce_sum_softmax2/act_op
[info] No shifts available for layer yolov11l/conv80/conv_op, using max shift instead. delta=1.9550
[info] No shifts available for layer yolov11l/conv80/conv_op, using max shift instead. delta=0.9775
[info] No shifts available for layer yolov11l/conv138/conv_op, using max shift instead. delta=1.0096
[info] No shifts available for layer yolov11l/conv138/conv_op, using max shift instead. delta=0.5048
Traceback (most recent call last):
File “/local/workspace/hailo_virtualenv/bin/hailomz”, line 33, in
sys.exit(load_entry_point(‘hailo-model-zoo’, ‘console_scripts’, ‘hailomz’)())
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main.py”, line 122, in main
run(args)
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main.py”, line 111, in run
return handlersargs.command
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main_driver.py”, line 248, in compile
_ensure_optimized(runner, logger, args, network_info)
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main_driver.py”, line 91, in _ensure_optimized
optimize_model(
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/core/main_utils.py”, line 353, in optimize_model
runner.optimize(calib_feed_callback)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2206, in optimize
result = self._optimize(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2025, in _optimize
checkpoint_info = self._sdk_backend.full_quantization(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1138, in full_quantization
new_checkpoint_info = self._full_acceleras_run(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1376, in _full_acceleras_run
new_checkpoint_info = self._optimization_flow_runner(optimization_flow, checkpoint_info)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 2029, in _optimization_flow_runner
optimization_flow.run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 239, in wrapper
return func(self, *args, **kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 357, in run
step_func()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py”, line 153, in parent_wrapper
func(self, *args, **kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 376, in step1
self.core_quantization()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 443, in core_quantization
self._create_hw_params()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 491, in _create_hw_params
algo.run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/optimization_algorithm.py”, line 55, in run
return super().run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/algorithm_base.py”, line 159, in run
self._run_int()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/create_hw_params/create_hw_params.py”, line 353, in _run_int
comp_to_retry = self._create_hw_params_component(matching_component_group)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/create_hw_params/create_hw_params.py”, line 211, in _create_hw_params_component
retry_negative_exp_list = self._hanlde_negative_exponent(layer, matching_component_group)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/create_hw_params/create_hw_params.py”, line 227, in _hanlde_negative_exponent
algo.run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/optimization_algorithm.py”, line 55, in run
return super().run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/algorithm_base.py”, line 159, in run
self._run_int()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/neg_exponent_fixer/neg_exp_fixer.py”, line 77, in _run_int
l_fix = self.fix_output(l_fix)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/neg_exponent_fixer/neg_exp_fixer.py”, line 123, in fix_output
self._log_negative_exponent_shift(l_fix)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/neg_exponent_fixer/neg_exp_fixer.py”, line 227, in _log_negative_exponent_shift
raise NegativeSlopeExponentNonFixable(
hailo_model_optimization.acceleras.utils.acceleras_exceptions.NegativeSlopeExponentNonFixable: Quantization failed in layer yolov11l/conv143 due to unsupported required slope. Desired shift is 9.0, but op has only 8 data bits. This error raises when the data or weight range are not balanced. Mostly happens when using random calibration-set/weights, the calibration-set is not normalized properly or batch-normalization was not used during training.
(hailo_virtualenv) hailo@sage-304375:/local/workspace$ hailomz compile --ckpt /local/shared_with_docker/best.onnx --calib-path /local/shared_with_docker/calib_output2/calib --yaml hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov11l.yaml --classes 1 --hw-arch hailo8
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
Start run for network yolov11l …
Initializing the hailo8 runner…
[info] Translation started on ONNX model yolov11l
[info] Restored ONNX model yolov11l (completion time: 00:00:00.27)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.11)
[info] NMS structure of yolov8 (or equivalent architecture) was detected.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.23/cv2.0/cv2.0.2/Conv /model.23/cv3.0/cv3.0.2/Conv /model.23/cv2.1/cv2.1.2/Conv /model.23/cv3.1/cv3.1.2/Conv /model.23/cv2.2/cv2.2.2/Conv /model.23/cv3.2/cv3.2.2/Conv.
[info] Start nodes mapped from original model: ‘images’: ‘yolov11l/input_layer1’.
[info] End nodes mapped from original model: ‘/model.23/cv2.0/cv2.0.2/Conv’, ‘/model.23/cv3.0/cv3.0.2/Conv’, ‘/model.23/cv2.1/cv2.1.2/Conv’, ‘/model.23/cv3.1/cv3.1.2/Conv’, ‘/model.23/cv2.2/cv2.2.2/Conv’, ‘/model.23/cv3.2/cv3.2.2/Conv’.
[info] Translation completed on ONNX model yolov11l (completion time: 00:00:02.36)
[info] Appending model script commands to yolov11l from string
[info] Added nms postprocess command to model script.
[info] Saved HAR to: /local/workspace/yolov11l.har
Preparing calibration data…
[info] Loading model script commands to yolov11l from /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov11l.alls
[info] Loading model script commands to yolov11l from string
[info] Found model with 3 input channels, using real RGB images for calibration instead of sampling random data.
[info] Starting Model Optimization
[warning] Reducing optimization level to 0 (the accuracy won’t be optimized and compression won’t be used) because there’s less data than the recommended amount (1024), and there’s no available GPU
[warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results
[info] Model received quantization params from the hn
[info] MatmulDecompose skipped
[info] Starting Mixed Precision
[info] Model Optimization Algorithm Mixed Precision is done (completion time is 00:00:00.89)
[info] LayerNorm Decomposition skipped
[info] Starting Statistics Collector
[info] Using dataset with 64 entries for calibration
Calibration: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [00:55<00:00, 1.16entries/s]
[info] Model Optimization Algorithm Statistics Collector is done (completion time is 00:00:56.59)
[info] Starting Fix zp_comp Encoding
[info] Model Optimization Algorithm Fix zp_comp Encoding is done (completion time is 00:00:00.00)
[info] Starting Matmul Equalization
[info] Model Optimization Algorithm Matmul Equalization is done (completion time is 00:00:00.07)
[info] Starting MatmulDecomposeFix
[info] Model Optimization Algorithm MatmulDecomposeFix is done (completion time is 00:00:00.00)
[info] No shifts available for layer yolov11l/conv42/conv_op, using max shift instead. delta=0.3988
[info] No shifts available for layer yolov11l/conv42/conv_op, using max shift instead. delta=0.1994
[info] No shifts available for layer yolov11l/conv43/conv_op, using max shift instead. delta=0.0822
[info] No shifts available for layer yolov11l/conv43/conv_op, using max shift instead. delta=0.0411
[info] No shifts available for layer yolov11l/conv42/conv_op, using max shift instead. delta=0.1994
[info] No shifts available for layer yolov11l/conv53/conv_op, using max shift instead. delta=0.0415
[info] No shifts available for layer yolov11l/conv53/conv_op, using max shift instead. delta=0.0207
[info] No shifts available for layer yolov11l/conv58/conv_op, using max shift instead. delta=0.1869
[info] No shifts available for layer yolov11l/conv58/conv_op, using max shift instead. delta=0.0934
[info] No shifts available for layer yolov11l/conv59/conv_op, using max shift instead. delta=0.3181
[info] No shifts available for layer yolov11l/conv59/conv_op, using max shift instead. delta=0.1590
[info] No shifts available for layer yolov11l/conv65/conv_op, using max shift instead. delta=0.0579
[info] No shifts available for layer yolov11l/conv65/conv_op, using max shift instead. delta=0.0289
[info] No shifts available for layer yolov11l/conv66/conv_op, using max shift instead. delta=0.7598
[info] No shifts available for layer yolov11l/conv66/conv_op, using max shift instead. delta=0.3799
[info] No shifts available for layer yolov11l/conv67/conv_op, using max shift instead. delta=0.3872
[info] No shifts available for layer yolov11l/conv67/conv_op, using max shift instead. delta=0.1936
[info] No shifts available for layer yolov11l/conv66/conv_op, using max shift instead. delta=0.3799
[info] activation fitting started for yolov11l/reduce_sum_softmax1/act_op
[info] No shifts available for layer yolov11l/conv74/conv_op, using max shift instead. delta=0.8335
[info] No shifts available for layer yolov11l/conv74/conv_op, using max shift instead. delta=0.4167
[info] No shifts available for layer yolov11l/conv76/conv_op, using max shift instead. delta=1.2689
[info] No shifts available for layer yolov11l/conv76/conv_op, using max shift instead. delta=0.6345
[info] No shifts available for layer yolov11l/conv74/conv_op, using max shift instead. delta=0.4167
[info] activation fitting started for yolov11l/reduce_sum_softmax2/act_op
[info] No shifts available for layer yolov11l/conv80/conv_op, using max shift instead. delta=1.9550
[info] No shifts available for layer yolov11l/conv80/conv_op, using max shift instead. delta=0.9775
[info] No shifts available for layer yolov11l/conv138/conv_op, using max shift instead. delta=1.0096
[info] No shifts available for layer yolov11l/conv138/conv_op, using max shift instead. delta=0.5048
Traceback (most recent call last):
File “/local/workspace/hailo_virtualenv/bin/hailomz”, line 33, in
sys.exit(load_entry_point(‘hailo-model-zoo’, ‘console_scripts’, ‘hailomz’)())
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main.py”, line 122, in main
run(args)
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main.py”, line 111, in run
return handlersargs.command
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main_driver.py”, line 248, in compile
_ensure_optimized(runner, logger, args, network_info)
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main_driver.py”, line 91, in _ensure_optimized
optimize_model(
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/core/main_utils.py”, line 353, in optimize_model
runner.optimize(calib_feed_callback)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2206, in optimize
result = self._optimize(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2025, in _optimize
checkpoint_info = self._sdk_backend.full_quantization(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1138, in full_quantization
new_checkpoint_info = self._full_acceleras_run(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1376, in _full_acceleras_run
new_checkpoint_info = self._optimization_flow_runner(optimization_flow, checkpoint_info)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 2029, in _optimization_flow_runner
optimization_flow.run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 239, in wrapper
return func(self, *args, **kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 357, in run
step_func()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py”, line 153, in parent_wrapper
func(self, *args, **kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 376, in step1
self.core_quantization()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 443, in core_quantization
self._create_hw_params()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 491, in _create_hw_params
algo.run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/optimization_algorithm.py”, line 55, in run
return super().run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/algorithm_base.py”, line 159, in run
self._run_int()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/create_hw_params/create_hw_params.py”, line 353, in _run_int
comp_to_retry = self._create_hw_params_component(matching_component_group)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/create_hw_params/create_hw_params.py”, line 211, in _create_hw_params_component
retry_negative_exp_list = self._hanlde_negative_exponent(layer, matching_component_group)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/create_hw_params/create_hw_params.py”, line 227, in _hanlde_negative_exponent
algo.run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/optimization_algorithm.py”, line 55, in run
return super().run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/algorithm_base.py”, line 159, in run
self._run_int()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/neg_exponent_fixer/neg_exp_fixer.py”, line 77, in _run_int
l_fix = self.fix_output(l_fix)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/neg_exponent_fixer/neg_exp_fixer.py”, line 123, in fix_output
self._log_negative_exponent_shift(l_fix)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/neg_exponent_fixer/neg_exp_fixer.py”, line 227, in _log_negative_exponent_shift
raise NegativeSlopeExponentNonFixable(
hailo_model_optimization.acceleras.utils.acceleras_exceptions.NegativeSlopeExponentNonFixable: Quantization failed in layer yolov11l/conv143 due to unsupported required slope. Desired shift is 9.0, but op has only 8 data bits. This error raises when the data or weight range are not balanced. Mostly happens when using random calibration-set/weights, the calibration-set is not normalized properly or batch-normalization was not used during training.

3

Here are a few lines of sanity checks from outside the Docker container, confirming that both the NVIDIA driver and nvidia-docker2 are correctly installed and visible:

~$ nvidia-smi -q | grep “Driver Version” | awk ‘{print $4}’
580.95.05
~$ lspci | grep “VGA compatible controller: NVIDIA”
01:00.0 VGA compatible controller: NVIDIA Corporation Device 2d59 (rev a1)
~$ dpkg -l | grep ‘nvidia-docker|nvidia-container-toolkit’
ii nvidia-container-toolkit 1.13.5-1 amd64 NVIDIA Container toolkit
ii nvidia-container-toolkit-base 1.13.5-1 amd64 NVIDIA Container Toolkit Base
ii nvidia-docker2 2.13.0-1 all nvidia-docker CLI wrapper

Inside the HAILO AI Suite container, nvidia-smi does see the GPU:

local/workspace$ nvidia-smi
Fri Oct 24 11:02:46 2025
±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 13.0     |
±----------------------------------------±-----------------------±---------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 5060 …    Off |   00000000:01:00.0  On |                  N/A |
| N/A   36C    P8              7W /   50W |     655MiB /   8151MiB |      0%      Default |
|                                         |                        |                  N/A |
±----------------------------------------±-----------------------±---------------------+

±----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
±----------------------------------------------------------------------------------------+

But hailomz compile doesn’t detect a usable CUDA device
[info] No GPU chosen and no suitable GPU found, falling back to CPU.

I found that to resolve the issue I may need to downgrading my GPU driver version to 535.183: No available GPU in Hailo Software Suite Docker

However, since I have RTX 5060, I can install only 570 version.

Looking forward for advice.

4

UPDATES:
The problem with GPU was due to the conflict within the docker container and the Dataflow Compiler, which anticipate a newer version instead of 11.8.

So looks like the latest version of the Hailo AI Suite Docker 10-2025 has the same CUDA version conflicts as well. I remembered the post about the 07-2025 in the community, so after that I decided to downgrade to the 01-2025 version, which solved my initial issue.

Later, I encountered other problems:
NegativeSlopeExponentNonFixable: Quantization failed in layer yolov11l/conv143 due to unsupported required slope. Desired shift is 9.0, but op has only 8 data bits. This error raises when the data or weight range are not balanced. Mostly happens when using random calibration-set/weights, the calibration-set is not normalized properly or batch-normalization was not used during training.

It is probably because of my model has only one class. To fix this, I modified the .alls file as follows:

quantization_param([conv120, conv143, conv165], force_range_out=[0.0, 1.0])

I also removed the ce loss types, keeping only l2rel for each layer. This finally allowed me to generate the .hef file.

5

However, I still encounter the following issue during compilation, where shifts are not available and are replaced by the maximum shift instead:
No shifts available for layer yolov11l/conv42/conv_op, using max shift instead. delta=0.3988
[info] No shifts available for layer yolov11l/conv42/conv_op, using max shift instead. delta=0.1994
[info] No shifts available for layer yolov11l/conv43/conv_op, using max shift instead. delta=0.0822
…
[info] No shifts available for layer yolov11l/conv138/conv_op, using max shift instead. delta=0.5048
[info] Finetune encoding skipped

Please let me know if you know how to fix it. I’ll post these updates to my original post as well.

6

Hey @Aleksei_Podturkin,

Great to hear the GPU is working! Just wanted to confirm that we’re aware of the Docker version mismatch issue, and we’re working on getting it fixed in the next release.

Regarding your issue:

You’re on the right track with your approach. Using the command in your .alls file is the correct workaround for handling the NegativeSlopeExponentNonFixable error, especially for YOLO models trained on datasets with a limited number of classes:

quantization_param([conv120, conv143, conv165], force_range_out=[0.0, 1.0])

As for the “No shifts available for layer … using max shift instead” messages:

These are informational logs, not errors. They indicate that the quantization process couldn’t find an optimal shift value for certain layers and defaulted to the maximum allowed shift. While not ideal, this doesn’t always cause problems. However, it can suggest that the data or weight ranges in those layers aren’t well balanced, which could potentially affect quantization accuracy.

You typically see this when:

  • The calibration set isn’t representative or properly normalized
  • The model doesn’t have batch normalization
  • The dataset is very small or has limited class diversity

The key question is: has your accuracy dropped after seeing these messages? If not, you’re good to proceed with the model. If you are seeing accuracy degradation, then we’ll need to address the quantization warnings mentioned above.

Let me know how it goes!

7

Hi @omria,

Thank you very much for getting back to me.

I’ve tested my .hef model and it seems to run well! I used batch normalisation during training; however, my training set included only 633 images.

Could you please advise on the correct approach for calibrating the dataset before compilation? I used the code below for my YOLO model, but I’m not fully sure if this is the recommended method.

Thank you again for your help.

def parse_arguments():
parser = argparse.ArgumentParser(description=“Create calibration images for Hailo PTQ (Ultralytics-style letterbox).”)
parser.add_argument(“–data_dir”, type=str, required=True, help=“Path with training images”)
parser.add_argument(“–target_dir”, type=str, default=“calib”, help=“Output dir”)
parser.add_argument(“–image_size”, type=int, default=640, help=“Square target size (e.g., 640)”)
parser.add_argument(“–num_images”, type=int, default=1024, help=“Number of calibration images”)
parser.add_argument(“–image_extensions”, nargs=“+”, type=str, default=[“.jpg”, “.jpeg”, “.png”],
help=“Image file extensions”)
parser.add_argument(“–pad_color”, nargs=3, type=int, default=[114, 114, 114],
help=“Letterbox pad color R G B”)
parser.add_argument(“–stride”, type=int, default=32,
help=“Model stride for optional auto padding (kept for compatibility)”)
return parser.parse_args()

def letterbox_pil(img: Image.Image, new_size: int, color=(114,114,114)):
“”"
Ultralytics-like letterbox for a square ‘new_size’ target.
- Resize to fit within new_size while keeping aspect.
- Pad equally to reach exact new_size x new_size with constant color.
“”"
w0, h0 = img.width, img.height
# scale (fit inside new_size)
r = min(new_size / w0, new_size / h0)
new_w, new_h = int(round(w0 * r)), int(round(h0 * r))
img = img.resize((new_w, new_h), Image.Resampling.LANCZOS)

# compute padding
dw = new_size - new_w
dh = new_size - new_h
# split padding equally (left/right, top/bottom)
left = dw // 2
right = dw - left
top = dh // 2
bottom = dh - top

# pad
if any(x < 0 for x in (left, right, top, bottom)):
    raise ValueError("Negative padding computed. Check sizes.")
img = ImageOps.expand(img, border=(left, top, right, bottom), fill=tuple(color))
return img

def find_image_files(directory, extensions):
image_files = 

for root, _, files in os.walk(directory):
for file in files:
if any(file.lower().endswith(ext.lower()) for ext in extensions):
image_files.append(os.path.join(root, file))
return image_files

def main():
args = parse_arguments()
target_dir_path = os.path.join(args.target_dir, “calib”)
os.makedirs(target_dir_path, exist_ok=True)

print(f"Searching for images in {args.data_dir}...")
all_image_paths = find_image_files(args.data_dir, args.image_extensions)
if not all_image_paths:
    print(f"No images found in {args.data_dir} with extensions {args.image_extensions}")
    return

random.shuffle(all_image_paths)
images_list = all_image_paths[:args.num_images]
print(f"Found {len(all_image_paths)} images; using {len(images_list)}")

if len(images_list) < args.num_images:
    print(f"Warning: Only found {len(images_list)} images, less than requested {args.num_images}")

print("Creating calibration data (Ultralytics-style letterbox, RGB, no normalization)...")
for idx, filepath in tqdm(list(enumerate(images_list))):
    try:
        with Image.open(filepath) as img:
            img = img.convert("RGB")
            processed = letterbox_pil(img, args.image_size, color=tuple(args.pad_color))
            # Save as JPEG (0–255 RGB). Hailo .alls will divide by 255.
            out_fp = os.path.join(target_dir_path, f"processed_{idx:05d}.jpg")
            processed.save(out_fp, format="JPEG", quality=95, subsampling=1, optimize=True)
    except Exception as e:
        print(f"Error processing {filepath}: {e}")

if name == “main”:
main()