hailomz compile - batch sizes

General
, , ,
1

I have been trying for a few weeks now to work towards an edge application involving a hailo 8 on an rpi5 that will serve as a proof of concept that I can scale.

I am happy with setting everything up on the pi, and I can run inference with my custom trained models but I’m getting almost nothing detected (I had a single low confidence detection of 1 of my 5 classes once…) so I don’t think my models are converted correctly so have a few questions.

Firstly, is there ant recommended best practice with onnx conversion? I’m using opset=11 and stating my input size of 1080P to ensure it sticks, but would love to know if there is anything else that would help at this stage.

Next, I’ve used hailomz compile for my conversion so far as it seems easier than all the separate dfc steps, but is this recommended? I’ve used:

hailomz compile yolov8s --ckpt=./models/parts.onnx --hw-arch hailo8 --classes 5 --calib-path ./data/train/parts/images

I’ve read about specifically stating the input and output nodes in other posts so i’ll do that next time i compile, but is there anything else about my approach that would cause almost zero detection capability. I used ~100 calibration images which were taken as a sample from my original training dataset.

Finally (for now), I had a few issues with hailomz compile around batch sizes, I assume because of the large image resolution. My training pc has an RTX4090 with 24gb of ram so i assumed it would be up to the task, but i occasionally got errors messages that said the GPU memory was full and I should use a smaller batch size. How should I avoid this moving forward? Can i manually state a smaller batch size or is there something else I can do about this?

Sorry this post is quite broad, but I really need help with this and have been living and breathing the troubleshooting foor a while now so would love some expert guidance.

Thanks!

1 Like
2

Hi @Liam_D

Welcome to the Hailo community. At DeGirum (a SW partner of Hailo), we developed a cloud compiler to help users convert their YOLO checkpoints to hef files. You can see more details here: Early Access to DeGirum Cloud Compiler

3

Thanks for that! I’ll deffinately request access.

I’d still like help on my questions though so i can gain knowledge and context, so hopefully somebody can help

4

So I now believe i should be using the DFC as my use case requires a fresh model rather than an amended model from the zoo?

I’m going to start trying to produce a hef model using the dfc, and if anybody could guide or advise me in anyway i’d appreciate it!

5

I spent a few hours today training various model types with my dataset, and the only combination I could succesfully compile was yolov8n with 320x320 input size. Even though i’m certain that this model wouldn’t be capable of detecting small objects for my use case, I tried it anyway and it gave very strange results. It couldn’t detect objects that the yolo model it was trained from could (I tested all the yolo models before conversion) and it kept detecting the entire frame as a ‘person’ class?

I don’t know how to proceed now, I know the Hailo 8 works so i’m sure that I just need to learn how to convert models from .pt to .hef.

I’m eagerly waiting feedback from this post and some others where I have commented.

I’d appreciate any help that anybody can provide!