converting yolov8m onnx to hef using model_zoo

General
1

I used yolov8.2.x on github to train a .pt model and converted it to an onnx model, but I encountered problems when converting it to hef using model_zoo

hailomz compile --ckpt  best.onnx  --calib-path /home/deepvision/deepvision/Pictures/相机设备1/imgs/val  --yaml /root/hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov8m.yaml

(hailo_venv) root@d3e3f2dcf76b:/home/deepvision/deepvision/Pictures/相机设备1/imgs/result/20241112-095619/weights# hailomz parse --hw-arch hailo8 --ckpt ./best.onnx yolov8m

<Hailo Model Zoo INFO> Start run for network yolov8m ...
<Hailo Model Zoo INFO> Initializing the runner...
[info] Translation started on ONNX model yolov8m
[info] Restored ONNX model yolov8m (completion time: 00:00:00.42)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.80)
[warning] ONNX shape inference failed: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:146 onnxruntime::Model::Model(onnx::ModelProto&&, const PathString&, const IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 9, max supported IR version: 8

[info] Unable to simplify the model: '/model.2/Split_output_0'
Traceback (most recent call last):
  File "/root/hailo_venv/bin/hailomz", line 33, in <module>
    sys.exit(load_entry_point('hailo-model-zoo', 'console_scripts', 'hailomz')())
  File "/root/hailo_model_zoo/hailo_model_zoo/main.py", line 122, in main
    run(args)
  File "/root/hailo_model_zoo/hailo_model_zoo/main.py", line 111, in run
    return handlers[args.command](args)
  File "/root/hailo_model_zoo/hailo_model_zoo/main_driver.py", line 202, in parse
    parse_model(runner, network_info, ckpt_path=args.ckpt_path, results_dir=args.results_dir, logger=logger)
  File "/root/hailo_model_zoo/hailo_model_zoo/core/main_utils.py", line 99, in parse_model
    model_name = translate_model(runner, network_info, ckpt_path, tensor_shapes=start_node_shapes)
  File "/root/hailo_model_zoo/hailo_model_zoo/utils/parse_utils.py", line 28, in translate_model
    runner.translate_onnx_model(
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/runner/client_runner.py", line 1158, in translate_onnx_model
    parser.translate_onnx_model(
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 242, in translate_onnx_model
    raise e from None
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 220, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 300, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 351, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/model_translator/translator.py", line 79, in convert_model
    self._create_layers()
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 34, in _create_layers
    self._add_direct_layers()
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 111, in _add_direct_layers
    self._layer_callback_from_vertex(vertex)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 346, in _layer_callback_from_vertex
    consumed_vertices = self._create_feature_split_layer(vertex)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1344, in _create_feature_split_layer
    split_sizes, output_shapes, groups = vertex.get_feature_split_info()
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 2480, in get_feature_split_info
    split_sizes = [
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 2481, in <listcomp>
    self.convert_nchw_to_nhwc(self._graph.output_shapes[output][0], self.output_format)[-1]
KeyError: '/model.2/Split_output_0'
2
3

using python3.9 + hailo_ai/yolo .pt model to .onnx, add opset and imgsz

yolo export ....   imgsz=640 format=onnx opset=11

Then can convert

Although the conversion speed is slow and it seems to be a 4-context model, it is successful

4

Hi @kok.bing, for less contexts make sure to enable the maximum optimization level for the compilation.

1 Like
5

When I tried to compile yolov8m’s hef using model zoo today, an error occurred. can you help me :grinning:

<Hailo Model Zoo INFO> Start run for network yolov8m ...
<Hailo Model Zoo INFO> Initializing the hailo8 runner...
[info] Translation started on ONNX model yolov8m
[info] Restored ONNX model yolov8m (completion time: 00:00:00.43)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:02.35)
[info] NMS structure of yolov8 (or equivalent architecture) was detected.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.22/cv2.0/cv2.0.2/Conv /model.22/cv3.0/cv3.0.2/Conv /model.22/cv2.1/cv2.1.2/Conv /model.22/cv3.1/cv3.1.2/Conv /model.22/cv2.2/cv2.2.2/Conv /model.22/cv3.2/cv3.2.2/Conv.
[info] Start nodes mapped from original model: 'images': 'yolov8m/input_layer1'.
[info] End nodes mapped from original model: '/model.22/cv2.0/cv2.0.2/Conv', '/model.22/cv3.0/cv3.0.2/Conv', '/model.22/cv2.1/cv2.1.2/Conv', '/model.22/cv3.1/cv3.1.2/Conv', '/model.22/cv2.2/cv2.2.2/Conv', '/model.22/cv3.2/cv3.2.2/Conv'.
[info] Translation completed on ONNX model yolov8m (completion time: 00:00:03.80)
[info] Saved HAR to: /home/deepvision/deepvision/Pictures/相机设备1/imgs/result/20241113-223939/weights/yolov8m.har
<Hailo Model Zoo INFO> Preparing calibration data...
[info] Loading model script commands to yolov8m from /root/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov8m.alls
[info] Loading model script commands to yolov8m from string
[info] Starting Model Optimization
[warning] Reducing optimization level to 1 (the accuracy won't be optimized and compression won't be used) because there's less data than the recommended amount (1024)
[info] Model received quantization params from the hn
[info] Starting Mixed Precision
[info] Mixed Precision is done (completion time is 00:00:00.88)
[info] Layer Norm Decomposition skipped
[info] Starting Stats Collector
[info] Using dataset with 64 entries for calibration
Calibration: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [00:27<00:00,  2.32entries/s]
[info] Stats Collector is done (completion time is 00:00:30.24)
[warning] The force_range command has been used, notice that its behavior was changed on this version. The old behavior forced the range on the collected calibration set statistics, but allowed the range to change during the optimization algorithms.
The new behavior forces the range throughout all optimization stages.
The old method could be restored by adding the flag weak_force_range_out=enabled to the force_range command on the following layers ['yolov8m/conv42', 'yolov8m/conv53', 'yolov8m/conv63']
Traceback (most recent call last):
  File "/root/hailo_venv/bin/hailomz", line 33, in <module>
    sys.exit(load_entry_point('hailo-model-zoo', 'console_scripts', 'hailomz')())
  File "/root/hailo_model_zoo/hailo_model_zoo/main.py", line 122, in main
    run(args)
  File "/root/hailo_model_zoo/hailo_model_zoo/main.py", line 111, in run
    return handlers[args.command](args)
  File "/root/hailo_model_zoo/hailo_model_zoo/main_driver.py", line 250, in compile
    _ensure_optimized(runner, logger, args, network_info)
  File "/root/hailo_model_zoo/hailo_model_zoo/main_driver.py", line 91, in _ensure_optimized
    optimize_model(
  File "/root/hailo_model_zoo/hailo_model_zoo/core/main_utils.py", line 326, in optimize_model
    runner.optimize(calib_feed_callback)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/runner/client_runner.py", line 2093, in optimize
    self._optimize(calib_data, data_type=data_type, work_dir=work_dir)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/runner/client_runner.py", line 1935, in _optimize
    self._sdk_backend.full_quantization(calib_data, data_type=data_type, work_dir=work_dir)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py", line 1045, in full_quantization
    self._full_acceleras_run(self.calibration_data, data_type)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py", line 1229, in _full_acceleras_run
    optimization_flow.run()
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/tools/orchestator.py", line 306, in wrapper
    return func(self, *args, **kwargs)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/flows/optimization_flow.py", line 316, in run
    step_func()
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/tools/orchestator.py", line 250, in wrapped
    result = method(*args, **kwargs)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py", line 111, in parent_wrapper
    raise SubprocessTracebackFailure(*child_messages)
hailo_model_optimization.acceleras.utils.acceleras_exceptions.SubprocessTracebackFailure: Subprocess failed with traceback

Traceback (most recent call last):
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py", line 73, in child_wrapper
    func(self, *args, **kwargs)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/flows/optimization_flow.py", line 335, in step1
    self.core_quantization()
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/tools/orchestator.py", line 250, in wrapped
    result = method(*args, **kwargs)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/flows/optimization_flow.py", line 385, in core_quantization
    self._create_encoding()
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/tools/orchestator.py", line 250, in wrapped
    result = method(*args, **kwargs)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/flows/optimization_flow.py", line 461, in _create_encoding
    algo.run()
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/algorithms/optimization_algorithm.py", line 50, in run
    return super().run()
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/algorithms/algorithm_base.py", line 151, in run
    self._run_int()
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/algorithms/create_encoding/create_io_encoding.py", line 70, in _run_int
    self._matching_algo.match_components_group(matching_component_group, training=False)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/algorithms/equiv_matching/matching_algo.py", line 518, in match_components_group
    self.zp_matching_group(matching_component_group)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/algorithms/equiv_matching/matching_algo.py", line 171, in zp_matching_group
    self._sources_zp_matching_group(layers_with_out_ind)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/algorithms/equiv_matching/matching_algo.py", line 182, in _sources_zp_matching_group
    new_scale = self.get_new_scale_vector(zp_old, zp, scale_vector, bins)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/algorithms/equiv_matching/matching_algo.py", line 440, in get_new_scale_vector
    ratio = cls._get_ratio(zp_old, zp, bins)
  File "/root/hailo_venv/lib/python3.9/site-packages/hailo_model_optimization/algorithms/equiv_matching/matching_algo.py", line 466, in _get_ratio
    raise MatchingAlgoError
hailo_model_optimization.acceleras.utils.acceleras_exceptions.MatchingAlgoError

my alls

normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
model_optimization_config(calibration, batch_size=2)
change_output_activation(conv58, sigmoid)
change_output_activation(conv71, sigmoid)
change_output_activation(conv83, sigmoid)
post_quantization_optimization(finetune, policy=enabled, learning_rate=0.000025)
nms_postprocess("../../postprocess_config/yolov8m_nms_config.json", meta_arch=yolov8, engine=cpu)
quantization_param([conv42, conv53, conv63], force_range_out=[0.0, 1.0])

yolov8m.yaml

base:
- base/yolov8.yaml
postprocessing:
  device_pre_post_layers:
    nms: true
  hpp: true
network:
  network_name: yolov8m
paths:
  network_path:
  - models_files/ObjectDetection/Detection-COCO/yolo/yolov8m/2023-02-02/yolov8m.onnx
  alls_script: yolov8m.alls
  url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov8m/2023-02-02/yolov8m.zip
parser:
  nodes:
  - null
  - - /model.22/cv2.0/cv2.0.2/Conv
    - /model.22/cv3.0/cv3.0.2/Conv
    - /model.22/cv2.1/cv2.1.2/Conv
    - /model.22/cv3.1/cv3.1.2/Conv
    - /model.22/cv2.2/cv2.2.2/Conv
    - /model.22/cv3.2/cv3.2.2/Conv
info:
  task: object detection
  input_shape: 640x640x3
  output_shape: 80x5x100
  operations: 78.93G
  parameters: 25.9M
  framework: pytorch
  training_data: coco train2017
  validation_data: coco val2017
  eval_metric: mAP
  full_precision_result: 50.08
  source: https://github.com/ultralytics/ultralytics
  license_url: https://github.com/ultralytics/ultralytics/blob/main/LICENSE
  license_name: GPL-3.0
6

Use opset=9, and try again. If it not work either, use ONNX load the yolov8m model and export it to onnx format with opset=9

8

opset=9 not works , i will try onnx reconvert later

9 
import onnx
import argparse
from onnx import version_converter

def convert_to_opset9(input_onnx_path, output_onnx_path):
    # 加载 ONNX 模型
    model = onnx.load(input_onnx_path)
    print(f"Original Opset version: {model.opset_import[0].version}")

    # 转换为 Opset 9
    converted_model = version_converter.convert_version(model, 9)
    print("Model successfully converted to Opset 9")

    # 保存新的 ONNX 模型
    onnx.save(converted_model, output_onnx_path)
    print(f"Converted model saved to: {output_onnx_path}")

if __name__ == "__main__":
    # 使用 argparse 接收命令行参数
    parser = argparse.ArgumentParser(description="Convert ONNX model to Opset 9.")
    parser.add_argument("input_model", type=str, help="Path to the input ONNX model.")
    parser.add_argument("output_model", type=str, help="Path to save the output ONNX model.")
    args = parser.parse_args()

    # 调用转换函数
    convert_to_opset9(args.input_model, args.output_model)

After I used this script to convert, I still couldn’t convert yolov8m.hef, and I still got the same error message