Dear all members,
I have a question regarding the meaning of the Send/Recv rate when measuring performance using the hailortcli run *.hef
command-line tool.
I ran the MSPN.hef
model downloaded from Hailo’s official benchmark page.
However, when I executed it, the results looked like this:
Compared to the official benchmark numbers, the performance degraded significantly — from 2132 to 792, as shown in the image.
This may be due to our system setup: we are using a Raspberry Pi 5 with a PCIe 2.0 x1 lane, which might be the cause of the performance drop.
But my main question is: what exactly do the Send and Recv rates represent?
At first, I thought these rates simply reflected the bandwidth usage over the PCIe interface.
However, even though the model’s performance dropped significantly, the measured Send/Recv rates didn’t seem to reach the theoretical bandwidth limit of PCIe 2.0 x1 — which is 4000 Mbit/s (or 500 MB/s).
This leads me to believe that there might be other data being transmitted that isn’t fully captured in the current measurements.
If anyone has expertise in this area and can shed some light on it, I would really appreciate your input.
My current hypothesis is that bandwidth usage related to model weights or other metadata may not be included in the Send/Recv measurements. (Only BW rate for inference was measured…?)
If someone could suggest a method to validate this hypothesis, that would be incredibly helpful.
Thank you for reading, and I hope you have a wonderful day.
Best regards,
Jusung Kang
===============================================
===============================================