How to check Hailo Module Utilization and how many TOPS are being used

Community Projects
, ,
1

Hi everyone,

I’m running an object detection pipeline on a Hailo module with Raspberry Pi 5, and I’d like to monitor how much of the Hailo accelerator is actually being utilized. Specifically, I want to check:

  • The real-time utilization of the Hailo module when inference is running.

  • How many TOPS are being used by my object detection model.

Is there any recommended tool, SDK command, or API function that can report Hailo module utilization and TOPS usage during runtime?

Any guidance or example commands/scripts would be very helpful.

Thanks!

2

You can use the HailoRT CLI Monitor. I one terminal use the following command to start an htop like monitor.

hailortcli monitor

In second terminal first set the environment variable for the monitor and then start your app or command.

export HAILO_MONITOR=1
hailortcli run model.hef

You can create a profiler report for your model. Here is an example from our Model Zoo.

Hailo Model Zoo - Yolov5m_wo_spp_profiler_results_compiled

Run through the tutorials if you want to find out how to create the profiler report.

hailo tutorial
3

Thanks for guidance , I able to monitor the utilization.

4

can you elaborate some more deeper

5

Before running inference open terminal and type the following command
hailortcli monitor
And run your model in another terminal.
You can able to see utilization.