Hello Hailo !
I successfully parsed my custom YoloV7 onnx model into a .har file.
From what I understood, the next step is doing the optimization step but it doesn’t work. Here is the command I ran and the output :
root@1581453f9888:/# hailo optimize /hailo/model.har --hw-arch hailo8 --calib-set-path /hailo/images.npy [info] No GPU chosen, Selected GPU 0 [info] First time Hailo Dataflow Compiler is being used. Checking system requirements… (this might take a few seconds) [Error] Unsupported OS - . Only Ubuntu is supported. [Error] See full log: /.install_logs/hailo_installation_log_2025.10.22T13:45:46.log [Error] Unsupported release - . Only 20.04 22.04 24.04 are supported. [Error] See full log: /.install_logs/hailo_installation_log_2025.10.22T13:45:46.log [Warning] CUDA version should be 12.5 or higher, found 12.8 . [Warning] CUDNN version should be 9 or higher, found 9.8. Component Requirement Found ========== ========== ========== ========== OS Ubuntu Required Release 20.04 Required Package python3-tk V Required Package graphviz V Required Package libgraphviz-dev V Required Package python3.10-dev V Required RAM(GB) 16 69 Required RAM(GB) 32 69 Recommended CPU-Arch x86_64 x86_64 Required CPU-flag avx V Required GPU-Driver 560 580 Recommended CUDA 12.5 12.8 Recommended CUDNN 9 9.8 Recommended Var:CC unset unset Required Var:CXX unset unset Required Var:LD unset unset Required Var:AS unset unset Required Var:AR unset unset Required Var:LN unset unset Required Var:DUMP unset unset Required Var:CPY unset unset Required System requirements check failed (see table above for details). Continue? [Y/n]y [info] Current Time: 13:46:24, 10/22/25 [info] CPU: Architecture: x86_64, Model: AMD Ryzen 9 5900X 12-Core Processor, Number Of Cores: 24, Utilization: 0.0% [info] Memory: Total: 62GB, Available: 59GB [info] System info: OS: Linux, Kernel: 6.12.0-55.38.1.el10_0.x86_64 [info] Hailo DFC Version: 3.33.0 [info] HailoRT Version: Not Installed [info] PCIe: No Hailo PCIe device was found [info] Running hailo optimize /hailo/model.har --hw-arch hailo8 --calib-set-path /hailo/images.npy [info] Starting Model Optimization [info] Using default optimization level of 2 [info] Using default compression level of 1 [info] Model received quantization params from the hn [info] MatmulDecompose skipped [info] Starting Mixed Precision [info] Assigning 4bit weights to layer yolo_v7_parly2_velizy2_mix_200ep_bs_1/conv91 with 4718.59k parameters [info] Assigning 4bit weights to layer yolo_v7_parly2_velizy2_mix_200ep_bs_1/conv35 with 2359.30k parameters [info] Assigning 4bit weights to layer yolo_v7_parly2_velizy2_mix_200ep_bs_1/conv46 with 2359.30k parameters [info] Ratio of weights in 4bit is 0.26 [info] Model Optimization Algorithm Mixed Precision is done (completion time is 00:00:00.33) [info] LayerNorm Decomposition skipped [info] Starting Statistics Collector [info] Using dataset with 64 entries for calibration Calibration: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [00:18<00:00, 3.46entries/s] [info] Model Optimization Algorithm Statistics Collector is done (completion time is 00:00:19.63) [info] Output layer yolo_v7_parly2_velizy2_mix_200ep_bs_1/conv69 with sigmoid activation was detected. Forcing its output range to be [0, 1] (original range was [2.1809076969176307e-19, 0.9999809265136719]). [info] Output layer yolo_v7_parly2_velizy2_mix_200ep_bs_1/conv81 with sigmoid activation was detected. Forcing its output range to be [0, 1] (original range was [5.1336217313799684e-14, 0.9999685287475586]). [info] Output layer yolo_v7_parly2_velizy2_mix_200ep_bs_1/conv92 with sigmoid activation was detected. Forcing its output range to be [0, 1] (original range was [7.526918324729515e-12, 0.9999983310699463]). [info] Starting Fix zp_comp Encoding [info] Model Optimization Algorithm Fix zp_comp Encoding is done (completion time is 00:00:00.00) [info] Matmul Equalization skipped [info] Starting MatmulDecomposeFix [info] Model Optimization Algorithm MatmulDecomposeFix is done (completion time is 00:00:00.00) [info] Finetune encoding skipped [info] Bias Correction skipped [info] Adaround skipped [info] Starting Quantization-Aware Fine-Tuning [info] Using dataset with 1024 entries for finetune Epoch 1/4 E0000 00:00:1761140926.636555 9 meta_optimizer.cc:966] layout failed: INVALID_ARGUMENT: Size of values 0 does not match size of permutation 4 @ fanin shape inStatefulPartitionedCall/SelectV2_2-2-TransposeNHWCToNCHW-LayoutOptimizer Traceback (most recent call last): File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/finetune/qft.py”, line 508, in run_qft self.main_train_summary_per_epoch = qft_distiller.fit( File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py”, line 122, in error_handler raise e.with_traceback(filtered_tb) from None File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/tensorflow/python/eager/execute.py”, line 53, in quick_execute tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, tensorflow.python.framework.errors_impl.ResourceExhaustedError: Graph execution error: Detected at node gradient_tape/yolo_v7_parly2_velizy2_mix_200ep_bs_1_3/conv69_1/bias_add_op_1/Cast_5/Cast defined at (most recent call last): File “/hailo_dataflow_compiler_venv/bin/hailo”, line 8, in File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/tools/cmd_utils/main.py”, line 111, in main File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/tools/cmd_utils/base_utils.py”, line 68, in run File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/tools/cmd_utils/base_utils.py”, line 89, in _run File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/tools/optimize_cli.py”, line 120, in run File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2206, in optimize File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2025, in _optimize File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1138, in full_quantization File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1376, in _full_acceleras_run File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 2029, in _optimization_flow_runner File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 239, in wrapper File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 357, in run File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py”, line 153, in parent_wrapper File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 382, in step2 File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 450, in post_quantization_optimization File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 748, in _finetune File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/optimization_algorithm.py”, line 55, in run File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/algorithm_base.py”, line 159, in run File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/finetune/qft.py”, line 387, in _run_int File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/finetune/qft.py”, line 508, in run_qft File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py”, line 117, in error_handler File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/keras/src/backend/tensorflow/trainer.py”, line 320, in fit File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/keras/src/backend/tensorflow/trainer.py”, line 121, in one_step_on_iterator File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/keras/src/backend/tensorflow/trainer.py”, line 108, in one_step_on_data File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/acceleras/model/distiller.py”, line 132, in train_step failed to allocate memory [[{{node gradient_tape/yolo_v7_parly2_velizy2_mix_200ep_bs_1_3/conv69_1/bias_add_op_1/Cast_5/Cast}}]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode. [Op:__inference_one_step_on_iterator_570536] During handling of the above exception, another exception occurred: Traceback (most recent call last): File “/hailo_dataflow_compiler_venv/bin/hailo”, line 8, in sys.exit(main()) File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/tools/cmd_utils/main.py”, line 111, in main ret_val = client_command_runner.run() File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/tools/cmd_utils/base_utils.py”, line 68, in run return self._run(argv) File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/tools/cmd_utils/base_utils.py”, line 89, in _run return args.func(args) File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/tools/optimize_cli.py”, line 120, in run self._runner.optimize(dataset, work_dir=args.work_dir) File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func return func(self, *args, **kwargs) File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2206, in optimize result = self._optimize( File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func return func(self, *args, **kwargs) File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2025, in _optimize checkpoint_info = self._sdk_backend.full_quantization( File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1138, in full_quantization new_checkpoint_info = self._full_acceleras_run( File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1376, in _full_acceleras_run new_checkpoint_info = self._optimization_flow_runner(optimization_flow, checkpoint_info) File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 2029, in _optimization_flow_runner optimization_flow.run() File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 239, in wrapper return func(self, *args, **kwargs) File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 357, in run step_func() File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py”, line 153, in parent_wrapper func(self, *args, **kwargs) File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 382, in step2 self.post_quantization_optimization() File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 450, in post_quantization_optimization self._finetune() File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 748, in _finetune algo.run() File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/optimization_algorithm.py”, line 55, in run return super().run() File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/algorithm_base.py”, line 159, in run self._run_int() File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/finetune/qft.py”, line 387, in _run_int self.run_qft(self._model_native, self._model, metrics=self.metrics) File “/hailo_dataflow_compiler_venv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/finetune/qft.py”, line 521, in run_qft raise AccelerasResourceError( hailo_model_optimization.acceleras.utils.acceleras_exceptions.AccelerasResourceError: GPU memory has been exhausted. Please try to use Quantization-Aware Fine-Tuning with lower batch size or run on CPU.
I run the DFC v 3.33.0 under this docker image nvidia/cuda:12.8.1-cudnn-devel-ubuntu22.04. I have no error or warning when running the parser.
The optimize error seems suspicious because I use a RTX 3080 TI with 12Go of VRAM. Is there something wrong in my process ? I checked the –help of hailo optimize but found nothing that could help me.
Thank you