Good day, I encountered an unusual issue in the benchmark for the Raspberry Pi. Here Benchmark on RPi5 and CM4 running yolov8s with rpi ai kit | Seeed Studio Wiki, it is stated that YOLOv8s reaches 80.7 FPS with a batch size of 1, and Pose reaches 65 FPS. However, in my case, although the FPS matches with a batch size of 8, it differs significantly with a batch size of 1—54 FPS for YOLOv8s and 46 FPS for Pose. Could this be related to the firmware version? I have version 4.20, while the benchmark used 4.17.
Have a look at the following post that explain some basics why your model may run slower.
Hailo Community - My model runs slower than expected
In your case you should check whether your PCIe is set to Gen 3.
Also when you compile a model there can be differences in speed between versions of the compiler and the compiler has a performance mode where it will try harder to achieve a higher throughput. This will increase the time needed to compile the model significantly. So, the same model can be compiled into HEF files that run at different speeds depending on the effort used to compile it.
Yes, I am using Gen 3. Moreover, I specifically used their model and repository, meaning I didn’t compile my own but tested theirs and got these results.
You can download a few HEF files from our Model Zoo and check whether you see something similar.