How can i use multi-chip in hailo?


I want to use multiple Hailo Chips


I checked that 4 chips are recognized.
Then, i was faced with two situation.

case 1 ) hailo profiler
case 2 ) hailortcli run

In case 1 ) hailo profiler

After checking the 4 chip and completing the SDK Conversion with the same process as 1 chip without a different setting.
I checked through the profiler and found that the number of devices is the same in the performance summary.

performance summary

Is there any additional settings I would make for Multi-Chips during SDK Conversion?

In case 2 ) hailortcli run

When i run though a hef file in hailortcli, is it possible to set up a multi-chip?

Maybe, I think this part is important to me for running Multi-Chip, how do i set it up?

Thanks, Hailo Team

1 Like

Hey @rhymus314

You can look at the following tappas examples that is running multi devices


i’ve checked that example a few times

i can run multi-chip in gstreamer pipeline

but, i just want to know about those two cases

The first case which is the hailo profiler , there is no such thing for running multiple devices .

The second case , you can run the following command :
hailortcli run {HEF} --device-count 4

1 Like

Hi @rhymus314,
There are a few ways to approachs that -

  1. Use the naive load balance by the hailort scheduler. This approach takes no previous knowledge on the setup/pipeline
  2. Manual splitting - in some cases, as a developer, you might want to allocate a certain physical device to one network, and have the other networks run on the rest of the devices. A typical case is if you have a ‘big’ detector at the beginning of the pipine, where the other parts of the pipeline are being activated dynamically based on the detections of this big model.

You can apply both approaches in GStreamer, C++, or Python. There’s no one-size-fits-all here, and as the system engineer you know best what suits your situation.

1 Like

Short answer, No.

The most versatile/scalable option is to convert a model for a single Hailo-8 and use multiple devices during runtime to get a higher throughput by running the same HEF on multiple Hailo-8. Using the HailoRT scheduler will allow you to also run other models with little management from your application.

This would even be true if your model runs in multiple contexts and you would also want to run other models.

As my colleague Nadav wrote only in some cases you may want to manually split a large model onto multiple devices. This may give you a bit more performance in some cases. However is less flexible and will make it harder to run additional models later.