Good morning,
I was wondering what are the best hardware upgrades to make in order to reduce model compile times for the Hailo 8. Currently my setup takes 4-8 hours for ViT-small models and 2-3 hours for YoloV5 models.
My current server setup:
Intel(R) Core™ i7-5820K CPU @ 3.30GHz
128 GB Memory
GTX 1080 Ti
A total system upgrade is also possible in the near future, but I would still be interesting in knowing what to prioritize with that build.
I suspect that by compile you mean the entire model conversion workflow.
On my test machine, converting a YOLOv5m model takes approximately 25 minutes.
Roughly half of this time is spent on model optimization, which runs on the GPU. My system uses an RTX 4090, and I would expect this step to take significantly longer on a less powerful GPU. If that is indeed the case, upgrading the GPU would likely provide the best benefit.
The remaining time is spent in the compilation step. While my CPU is a bit faster, the compilation does not make use of many CPU cores. Therefore I would not expect substantial gains from a CPU upgrade.
Note that enabling the compilers performance mode can significantly increase compilation time, as the compiler will try harder to increase the utilization to achieve the best FPS.