How to run inference using ONNX file

General
1

Hi,

I want to run inference using my custom trained yolov8. What i should do?

I have .pt file and onnx file.

Can I run inference by using onnxruntime:

or I must first convert onnx to hef file and then run inference by using i.e this code examples??

I try convert onnx to hef but i have some problems…
like:
raise FileNotFoundError(f"Couldn’t find dataset in {data_path}. Please refer to docs/DATA.rst.")
FileNotFoundError: Couldn’t find dataset in /home/usser/.hailomz/data/models_files/coco/2021-06-18/coco_calib2017.tfrecord. Please refer to docs/DATA.rst.

2

Welcome to the Hailo Community!

Yes, you first must convert the model into the HEF format.

The dataset mentioned here can be created using the instruction here:

GitHub - Hailo Model Zoo - DATA

However, you said

That means you need to use images from your own dataset. The Model Zoo will use the COCO dataset for this model. See

GitHub - Hailo Model Zoo - yolov8m.yaml

Check this page last section:

GitHub - Hailo Model Zoo - Training - Yolov8

If you have not done so yet, please also work trough the tutorials in the Hailo AI Software Suite about the model conversion workflow. Inside the docker simply type:

hailo tutorials
3

Thanks for your reply.

I read these documents, but I’m still confused.

To be clarify, my model is yolov8n, but I used 3 classes, and other input version (1024x1024), and this model was trained on my custom dataset.

So how can I convert it from onnx to hef ? I also need to do it with int16

After reading these docs, I get more questions:

  • start_node_names (list of str, optional): Name of the first ONNX node to parse.
  • end_node_names (list of str, optional): List of ONNX nodes, that the parsing can stop after all of them are parsed.

I understand that I can get these nodes names from my onnx file, but hm… in yolov8n I have:
Input [image, batch x channels x width x height] →

Node name: /model.0/conv/Conv
Input(s): images, model.0.conv.weight, model.0.conv.bias
Output(s): /model.0/conv/Conv_output_0
Op type: Conv

…
…
…

some convs, concat, split … etc

at the end:

…

Node name: /model.22/Div_1
Input(s): /model.22/Add_2_output_0, /model.22/Constant_11_output_0
Output(s): /model.22/Div_1_output_0
Op type: Div

Node name: /model.22/Sub_1
Input(s): /model.22/Add_1_output_0, /model.22/Sub_output_0
Output(s): /model.22/Sub_1_output_0
Op type: Sub

Node name: /model.22/Concat_4
Input(s): /model.22/Div_1_output_0, /model.22/Sub_1_output_0
Output(s): /model.22/Concat_4_output_0
Op type: Concat

Node name: /model.22/Constant_12
Input(s):
Output(s): /model.22/Constant_12_output_0
Op type: Constant

Node name: /model.22/Mul_2
Input(s): /model.22/Concat_4_output_0, /model.22/Constant_12_output_0
Output(s): /model.22/Mul_2_output_0
Op type: Mul

Node name: /model.22/Sigmoid
Input(s): /model.22/Split_output_1
Output(s): /model.22/Sigmoid_output_0
Op type: Sigmoid

Node name: /model.22/Concat_5
Input(s): /model.22/Mul_2_output_0, /model.22/Sigmoid_output_0
Output(s): output0
Op type: Concat

So… i find some instructitons when is describes that i should use configurations file ".yaml:

what is it???
anchors:
scale_factors:
- 0.5
- 0.5
regression_length: 15

normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
change_output_activation(conv42, sigmoid)
change_output_activation(conv53, sigmoid)
change_output_activation(conv63, sigmoid)
nms_postprocess(“…/…/postprocess_config/yolov8s_nms_config.json”, meta_arch=yolov8, engine=cpu)

Why is the normalization and output changing?
I saw on the graph after converting from onnx to har that my architecture is changing and I don’t know why.

Why do I have to use images from my dataset when using the zoo model?

Should I prepare the data in the same format as in the examples? Labels, images, folder tracking / val

And how to use config files?

…there is too much information out there, making me a little lost and I don’t know how to convert onnx to hef correctly

4

Here is also

but hm, when i change input shape i shoudl change eanchors values?? but how?