Facing issue for onnx to hef for custom trained model

General
1

Hello Team,

I am drilling through all the possible steps needed for converting my custom onnx model to hec file for running it on Halio AI chip on raspi5.

I have trained this model on YOLOv8_seg using using dataset of 4320x4320 resolution which was my requirement for higher precision.

I need to run this on raspi5 with higher processing time for this i am trying Hailo Ai chip.

For conversion I had tried Dataflow compiler steps but it was not been helpfull.

Now i am trying the compilation using hailo zoo for which i followed the steps needed for custom train data.
I am using my onnx file which was already trained using Clould as it had needed compute engine.

for compilation i am trying to follow the steps as explain in below link.
Tutorial of AI Kit with Raspberry Pi 5 about YOLOv8n object detection | Seeed Studio Wiki.
My calibration data is coco dataset as i was clueless about how to use my custom dataset as there are my calibration files.

while optimizing i am getting various error i have debug all of them still i am stuck with this below error. I don’t want to give up on this problem need your help to guide me as documentation is not able to help me to debug my issue.

below is the error i am getting.

hailo_model_optimization.acceleras.utils.acceleras_exceptions.NegativeSlopeExponentNonFixable: Quantization failed in layer yolov8n_seg/conv45 due to unsupported required slope. Desired shift is 10.0, but op has only 8 data bits. This error raises when the data or weight range are not balanced. Mostly happens when using random calibration-set/weights, the calibration-set is not normalized properly or batch-normalization was not used during training.
(hailoenv) nsl@DINRfyCn6h5:~/hailoenv/lib/python3.10/site-packages/hailo_model_zoo$

this are my config files

yolov8.yaml

  base:
  - base/yolo.yaml
  postprocessing:
    meta_arch: nanodet_v8
    anchors:
      scale_factors:
      - 0.5
      - 0.5
      regression_length: 15
    device_pre_post_layers:
      sigmoid: true
    nms_max_output_per_class: 300
    post_nms_topk: 300
    nms_iou_thresh: 0.7
    score_threshold: 0.001

YOLO.yaml

base:
- base/coco.yaml
preprocessing:
network_type: detection
input_shape:
- 4320
- 4320
- 3
meta_arch: yolo_v5
padding_color: 114
postprocessing:
nms_iou_thresh: 0.45
score_threshold: 0.01
meta_arch: yolo_v5
anchors:
strides:
- 8
- 16
- 32
sizes:
- - 10
- 13
- 16
- 30
- 33
- 23
- - 30
- 61
- 62
- 45
- 59
- 119
- - 116
- 90
- 156
- 198
- 373
- 326
hpp: false
info:
source:
Release v2.0 · ultralytics/yolov5 · GitHub
parser:
normalization_params:
normalize_in_net: true
mean_list:
- 0.0
- 0.0
- 0.0
std_list:
- 255.0
- 255.0
- 255.0
evaluation:
labels_offset: 1
classes: 80
dataset_name: coco_2017_detection

2

Hi @kewalrc19,
The Negative Slop error is usually caused from one (or more) of the following:

  1. The model was not trained properly (model did not converge).
  2. The model was not trained with Batch Normalization and thus the outputs from the activations “explode”, meaning the values range of the output is wider than what can be represented in 8 bits.
  3. The calibration process was done with a calibration set that is not representable subset of the training dataset (64 images).

Did you apply all the above?
BTW - a 4320x4320 input resolution is vey big. Even if you are able to compile the model, the performance, especially on the Raspie5 will not be very good. With the standard yolov8_seg model (640x640) it runs with the app here:

around 30 FPS. please take that into considerations.

Regards,