About Compiler Upcoming Features/Enhancement

[saurabh]

I have raspberry pi AI kit. But unable to use for any custom model. And also not able to compile the pretrained yolo models. Tried both with or without Hailo model zoo. And also inside Hailo Docker Suite.
I am not sure exactly about how to handle hailo related errors. Everyone is struggling to compile a custom model. I read other post. And most of them have issue in compiling. We can only use the provided compiled models. Which technically does not make sense for everyone.

I understand that, It’s relatively new in the market. And you will be definitely working hard to improve.

So May I know what’s the future plans for dataflow compiler?
What’s progress on it and when it will be available?
DFC studio can also be a great tool but currently it only saves har format. So When you releasing the new features quantization, optimization, compilation steps.

[omria]

Hey @saurabh,

Thanks for reaching out. To better assist you, could you provide more details about your situation?

1. Which specific model are you attempting to retrain?
2. What errors are you encountering during this process?

Regarding dfc-studio, we’re aiming to release full functionality in our next quarterly update.

We’re also here to help with alternatives:

1. We can suggest similar models that might suit your needs.
2. If there’s enough demand for a specific model, we can consider adding it to our model zoo by compiling it ourselves.

Please feel free to suggest any models you’re interested in seeing in our zoo.

Don’t hesitate to share more details about the errors you’re facing. The more information you provide, the better we can assist you in resolving any issues.

[saurabh]

Thank you so much for you’re response.
I have a custom trained model on yolov8s-pose. I trained and converted to onnx outside hailo env (hailo_model_zoo does not have training for pose model). And copied trained model and dataset into hailo env.

Next I run the compile command -

hailomz compile --ckpt trainedv8s-pose.onnx --calib-path keypoints-dataset/images/ --yaml hailo_model_zoo/cfg/networks/yolov8s_pose.yaml --classes 1

My dataset yaml

Once I run I get the error -

My Env:
Ubuntu 22.04(NO GPU)
Hailo Docker Suite (Latest)

Training Env
Colab
Using Ultralytics

[saurabh]

@omria
In the Previous I tried latest opset version.
Also Tried different onnx opset versions(11 and 13). Nothing changed. Same Error.

[saurabh]

@omria
Another question, do you provide paid support to businesses in creating models compatible to hailo? If yes, I will probably prefer to go with that.

[saurabh]

@omria
Hey I think I found where is the issue…
My training data contains images of resolution 640x360. And also my data contains 2 keypoints. And these things changes the output of the network. And I am considering those changes in yaml file. Probably this is the reason…

Yolov8s-pose

my trained yolov8s-pose

Another Similar Issue: Problem With Model Optimization

base:
- base/yolov8_pose.yaml
network:
  network_name: yolov8s_pose
paths:
  alls_script: yolov8s_pose.alls
  network_path:
  - models_files/PoseEstimation/yolov8/yolov8s/pretrained/2023-06-11/yolov8s_pose.onnx
  url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/PoseEstimation/yolov8/yolov8s/pretrained/2023-06-11/yolov8s_pose.zip
parser:
  nodes:
  - null
  - - /model.22/cv2.2/cv2.2.2/Conv
    - /model.22/cv3.2/cv3.2.2/Conv
    - /model.22/cv4.2/cv4.2.2/Conv
    - /model.22/cv2.1/cv2.1.2/Conv
    - /model.22/cv3.1/cv3.1.2/Conv
    - /model.22/cv4.1/cv4.1.2/Conv
    - /model.22/cv2.0/cv2.0.2/Conv
    - /model.22/cv3.0/cv3.0.2/Conv
    - /model.22/cv4.0/cv4.0.2/Conv
info:
  task: pose estimation
  input_shape: 640x640x3
  output_shape: 20x20x64, 20x20x1, 20x20x51, 40x40x64, 40x40x1, 40x40x51, 80x80x64,
    80x80x1, 80x80x51
  operations: 30.2G
  parameters: 11.6M
  framework: pytorch
  training_data: coco keypoints train2017
  validation_data: coco keypoints val2017
  eval_metric: mAP
  full_precision_result: 59.2
  source: https://github.com/ultralytics/ultralytics
  license_url: https://github.com/ultralytics/ultralytics/blob/main/LICENSE
  license_name: GPL-3.0

Can you help me updating the output_shape? How can adjust shape according to my data.

[saurabh]

I am able to solve the issues I have. I use the application examples from hailo.

Hailo-Application-Code-Examples/runtime/python/pose_estimation at main ·...

I Changed the key-points length from the post processing steps, where ever required. And it is working correctly.

If someone is facing issue with Yolov8 Pose I can try to help.

[prasaanthg]

hi @saurabh im training a multi class pose detection model by retraining yolov8spose
but no matter what change i do in alls script im not getting any predictions. when i observed the snr of each layer, the output layers are getting affected. can you please help me with this?

[saurabh]

@prasaanthg
Hi, I am new to ML and not much familiar of the terms.
Can you be tell me the what issue you are seeing in output frame?

Are you getting incorrect keypoints?

[saurabh]

if you have trained your custom model and compiled with default configurations you will most likely see keypoints issues in pose inference.

Take a look on this recent thread… Custom python Post processing function in gstreamer performance issues

[prasaanthg]

@saurabh i have some doubts.

1. what are the changes you have done during training yolov8pose]
2. what are the changes you have done during conversion to hailo(in onnx, model configs like alls,yaml)

the poset processing you are saying is after we get the .hef right?

i very sure i dont have any problems during inference but during porting to hailo.

could you please guide me?

[prasaanthg]

@saurabh im not getting any error. i compiled my model with 4 classes and when i do inference in pi 5 and not getting any predictions. im sure that it is something to do with training. can you help me out please?