Hello everyone. i trying to optimize my .har file so that i can convert it to .hef file. I am facing some errors. while my initial calib_dtype is float64, when i run the code, the hailo sdk automatically converts the dtype from float64 to either float32 or int32. this results in element size expected 8 bytes and getting only 4 bytes instead. please help me.
Python executable: /usr/bin/python3
Hailo SDK Client path: /home/osboxes/.local/lib/python3.10/site-packages/hailo_sdk_client/init.py
Hailo SDK Client: 3.30.0
NumPy: 1.23.3, SciPy: 1.13.1
dummy_calib shape: (1024, 640, 640, 3), dtype: float64
Loading HAR file: /home/osboxes/Desktop/Himanshu/yolov11s_haygriv.har
Optimization commands: {}
dummy_calib dtype before optimize: float64
Starting optimization…
[info] Starting Model Optimization
[warning] Reducing optimization level to 0 (the accuracy won’t be optimized and compression won’t be used) because there’s no available GPU
[warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results
[info] Model received quantization params from the hn
[info] MatmulDecompose skipped
[info] Starting Mixed Precision
[info] Model Optimization Algorithm Mixed Precision is done (completion time is 00:00:01.86)
[info] LayerNorm Decomposition skipped
[info] Starting Statistics Collector
[info] Using dataset with 64 entries for calibration
Calibration: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [02:34<00:00, 2.42s/entries]
[info] Model Optimization Algorithm Statistics Collector is done (completion time is 00:02:40.07)
[info] Starting Fix zp_comp Encoding
[info] Model Optimization Algorithm Fix zp_comp Encoding is done (completion time is 00:00:00.00)
[info] Starting Matmul Equalization
Traceback (most recent call last):
File “/home/osboxes/Desktop/Himanshu/prayagraj/har_to_optimised_har2.py”, line 24, in
runner.optimize(dummy_calib, data_type=“np_array”)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2128, in optimize
self._optimize(calib_data, data_type=data_type, work_dir=work_dir)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 1970, in _optimize
self._sdk_backend.full_quantization(calib_data, data_type=data_type, work_dir=work_dir)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1125, in full_quantization
self._full_acceleras_run(self.calibration_data, data_type)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1319, in _full_acceleras_run
optimization_flow.run()
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 306, in wrapper
return func(self, *args, **kwargs)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 335, in run
step_func()
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 250, in wrapped
result = method(*args, **kwargs)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py”, line 124, in parent_wrapper
func(self, *args, **kwargs)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 354, in step1
self.core_quantization()
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 250, in wrapped
result = method(*args, **kwargs)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 414, in core_quantization
self._matmul_equalization()
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 250, in wrapped
result = method(*args, **kwargs)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 599, in _matmul_equalization
algo.run()
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/algorithms/optimization_algorithm.py”, line 54, in run
return super().run()
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/algorithms/algorithm_base.py”, line 150, in run
self._run_int()
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/matmul_equalization.py”, line 123, in _run_int
self.equalize_input_paths(normal_subflow, transpose_subflow, layer)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/matmul_equalization.py”, line 162, in equalize_input_paths
set_scales_forward(self._model, normal_flow, layer.groups, factors)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/matmul_equalization.py”, line 340, in set_scales_forward
scales, zp_points = get_scales_output(in_layer, groups, factors, transpose)
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/matmul_equalization.py”, line 367, in get_scales_output
scales, zp = find_the_best_zp(
File “/home/osboxes/.local/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/optimal_zp_finder.py”, line 237, in find_the_best_zp
result = minimize(objective, x0, method=“SLSQP”, constraints=cons)
File “/home/osboxes/.local/lib/python3.10/site-packages/scipy/optimize/_minimize.py”, line 722, in minimize
res = _minimize_slsqp(fun, x0, args, jac, bounds,
File “/home/osboxes/.local/lib/python3.10/site-packages/scipy/optimize/_slsqp_py.py”, line 431, in _minimize_slsqp
slsqp(m, meq, x, xl, xu, fx, c, g, a, acc, majiter, mode, w, jw,
ValueError: failed to initialize intent(inout) array – expected elsize=8 but got 4
Hey @Himanshu_Panjwani ,
Welcome to the Hailo Community!
Hailo SDK automatically converts float64 calibration data to float32/int32, causing a byte-size mismatch (8 bytes expected, 4 bytes received).
Quick Fix:
- Convert Data Explicitly:
import numpy as np
# Convert to float32 before optimization
dummy_calib = dummy_calib.astype(np.float32)
# Run optimization
runner.optimize(dummy_calib, data_type="np_array")
- Alternative: Adjust Optimization Level:
runner.optimize(dummy_calib, data_type="np_array", optimization_level=2)
Note: Hailo SDK only supports float32/int32 for quantization calibration. Pre-converting your data helps avoid automatic type conversion issues.
Thanks for replying. I have tried quite a few similar approaches. However the error stays persistent.
Below is the screenshot of the code i have been trying…
Hi @Himanshu_Panjwani
We believe that this error is in fact related to some scipy dependency issue: Try to complie YOLOv11 with DFC 3.30 got ValueError: failed to initialize intent(inout) array – expected elsize=8 but got 4 - General - Hailo Community. We also encountered this issue and could resolve it by following the suggestion in above thread.
Thank you. What i did was try a different approach and it is working so far.
The har file was created and now i had to optimize it for further conversion. there were some changes that needed to be made in .nms.json file and .alls file and after doing that, it was like taking candy from a baby.
(hailo-venv) osboxes@osboxes:~/Desktop/Hailo/hailo_model_zoo/Spectrum_Models$ hailo optimize yolo11s-4Classes_24022025.har --use-random-calib-set
[info] Current Time: 00:18:59, 02/25/25
[info] CPU: Architecture: x86_64, Model: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz, Number Of Cores: 2, Utilization: 12.1%
[info] Memory: Total: 101GB, Available: 98GB
[info] System info: OS: Linux, Kernel: 6.8.0-52-generic
[info] Hailo DFC Version: 3.30.0
[info] HailoRT Version: Not Installed
[info] PCIe: No Hailo PCIe device was found
[info] Running `hailo optimize yolo11s-4Classes_24022025.har --use-random-calib-set`
[info] For NMS architecture yolov8 the default engine is cpu. For other engine please use the 'engine' flag in the nms_postprocess model script command. If the NMS has been added during parsing, please parse the model again without confirming the addition of the NMS, and add the command manually with the desired engine.
[info] The layer yolo11s_4_Classes/conv51 was detected as reg_layer.
[info] The layer yolo11s_4_Classes/conv54 was detected as cls_layer.
[info] The layer yolo11s_4_Classes/conv62 was detected as reg_layer.
[info] The layer yolo11s_4_Classes/conv65 was detected as cls_layer.
[info] The layer yolo11s_4_Classes/conv80 was detected as cls_layer.
[info] The layer yolo11s_4_Classes/conv77 was detected as reg_layer.
[info] Using the default score threshold of 0.001 (range is [0-1], where 1 performs maximum suppression) and IoU threshold of 0.7 (range is [0-1], where 0 performs maximum suppression).
Changing the values is possible using the nms_postprocess model script command.
[info] The activation function of layer yolo11s_4_Classes/conv54 was replaced by a Sigmoid
[info] The activation function of layer yolo11s_4_Classes/conv65 was replaced by a Sigmoid
[info] The activation function of layer yolo11s_4_Classes/conv80 was replaced by a Sigmoid
[info] Found model with 3 input channels, using real RGB images for calibration instead of sampling random data.
[info] Starting Model Optimization
[warning] Reducing optimization level to 0 (the accuracy won't be optimized and compression won't be used) because there's no available GPU
[warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results
[info] Model received quantization params from the hn
[info] MatmulDecompose skipped
[info] Starting Mixed Precision
[info] Model Optimization Algorithm Mixed Precision is done (completion time is 00:00:01.44)
[info] LayerNorm Decomposition skipped
[info] Starting Statistics Collector
[info] Using dataset with 64 entries for calibration
Calibration: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [02:34<00:00, 2.42s/entries]
[info] Model Optimization Algorithm Statistics Collector is done (completion time is 00:02:39.91)
[info] Starting Fix zp_comp Encoding
[info] Model Optimization Algorithm Fix zp_comp Encoding is done (completion time is 00:00:00.00)
[info] Starting Matmul Equalization
[info] Model Optimization Algorithm Matmul Equalization is done (completion time is 00:00:00.05)
[info] activation fitting started for yolo11s_4_Classes/reduce_sum_softmax1/act_op
[info] No shifts available for layer yolo11s_4_Classes/conv38/conv_op, using max shift instead. delta=0.7868
[info] No shifts available for layer yolo11s_4_Classes/conv38/conv_op, using max shift instead. delta=0.3934
[info] Finetune encoding skipped
[info] Bias Correction skipped
[info] Adaround skipped
[info] Quantization-Aware Fine-Tuning skipped
[info] Layer Noise Analysis skipped
[info] The calibration set seems to not be normalized, because the values range is [(0.0, 1.0), (0.0, 1.0), (0.0, 1.0)].
Since the neural core works in 8-bit (between 0 to 255), a quantization will occur on the CPU of the runtime platform.
Add a normalization layer to the model to offload the normalization to the neural core.
Refer to the user guide Hailo Dataflow Compiler user guide / Model Optimization / Optimization Related Model Script Commands / model_modification_commands / normalization for details.
[info] Model Optimization is done
[info] Saved HAR to: /home/osboxes/Desktop/Hailo/hailo_model_zoo/Spectrum_Models/yolo11s_4_Classes_optimized.har
Hi @Himanshu_Panjwani
Your approach may work but it is likely that it will result in a sub-optimal model as you are using random calibration data.
I understand your concern. But when i was converting my onnx into har format, at that time I also used my own yaml file and made required changes to add path of my own dataset in yaml configuration. So that said, I have not been using any random dataset for calibration.
If you think i should be made aware of any more such useful information, do reach out to me. And thank you for your assistance.
Hi @Himanshu_Panjwani
I do not know your full flow but from the logs you shared, I can see that you have some custom yolo11s models with 4 classes. This trained model would have some map metric. After compiling the checkpoint/onnx model to .hef you can evaluate the map of the model and check if accuracy is as expected. Since your optimize command specified --use-random-calib-set, I thought you used random data. In any case, you will know accuracy degradation once you compile successfully and evaluate the model.
Thanks @shashi for your assistance. I now get what you are trying to say… We thought the same initially, nut then we tested out the model in real world as well as in 25+ video files. The results were satisfying.
However, what you said certainly cannot be ignored. If i face any issue in the future, I will consult you for sure.
1 Like