Here is a general topic that will explain a few things. Please have a look.
Hailo Community - My model runs slower than expected
That is on a x86 machine with 4 PCIe lanes and batch-size 8.
The Hailo device always uses integer operations. That is one part why our architecture is more power efficient.
Every bit helps. If you need a significant boos have a look at the yolov8n model. You can use the Model Explorer in the Developer Zone to compare models on accuracy vs speed.
See guide above about Multi Context models
Yes, on weak CPUs pre- and post-processing can be the bottleneck. The Hailo accelerators have been designed to do the heavy compute of inference at very high efficiency. But they cannot do everything.
We do have our application code examples and Raspberry Pi examples as a starting point. Have a look into our GitHub repositories.
With a batch-size of 2 I measured 49 FPS on a Raspberry Pi 5 with Hailo-8.