How to Run Dual Camera with Two AI Models in Parallel on Raspberry Pi 5 + Hailo-8?

General
1

Hi everyone,

I’m currently working with a Raspberry Pi 5 combined with a Hailo-8 AI HAT+, and I need to run two cameras simultaneously, each using a different AI model (e.g., YOLOv6n + YOLOv8s). My goal:

  • Camera 1 runs model A
  • Camera 2 runs model B
  • Both run in real-time, in parallel, using the Hailo NPU for inference

I’ve found relevant code in the hailo-rpi5-examples repository (e.g., detection.py, multi_detection), and I’m able to detect both cameras. However, I’m not sure how to configure two parallel pipelines, each with a different model. Should I launch them in separate processes, or is there a recommended way to manage both in a single application?

My current setup:

  • 2 USB camera
  • Hailo SDK on Raspberry Pi 5
  • GStreamer pipelines using hailonet, hailofilter, hailooverlay, etc.
  • Models: YOLOv6n, YOLOv8s (.hef format)

Has anyone successfully run a similar multi-camera, multi-model pipeline using the Hailo SDK on RPi5? Any documentation, code samples, or tips would be greatly appreciated!

Thanks in advance!

2

Hi @The_Nguyen
You can use DeGirum PySDK for such use cases. Please see Running multiple models independently thread for an example.

3

Hey @The_Nguyen,

Welcome to the Hailo Community!

I’m curious , what’s the reason for running two different models? There might be a more efficient approach depending on what you’re trying to achieve.

For a good starting point, I’d suggest checking out this example: tappas/apps/h8/gstreamer/general/multistream_detection at master · hailo-ai/tappas · GitHub

It should give you a solid foundation to work from.

5

Hi @shashi
Thank you for your response and guidance. I followed your instructions with the code below; however, when I run it, two windows appear showing the two camera USB streams, but they immediately freeze and no frames are captured. The process stops right away, and CPU usage spikes to over 200%. I’m not sure where the issue lies. I’m using a Raspberry Pi 5 with 8GB RAM and a Hailo-8 (26 TOPS). I would greatly appreciate your feedback so that I can continue my research and further development. Here is my code:

import degirum as dg
import degirum_tools.streams as dgstreams
inference_host_address = “@local”
zoo_url = “degirum/models_hailort”
token = ‘’
configurations = [
{
“model_name”: “yolov8n_relu6_coco–640x640_quant_hailort_hailo8_1”,
“source”: 2,
“display_name”: “USB cam 1”,
},
{
“model_name”: “yolov8n_relu6_face–640x640_quant_hailort_hailo8_1”,
“source”: 0,
“display_name”: “USB cam 2”,
},
]
models = [
dg.load_model(cfg[“model_name”], inference_host_address, zoo_url, token)
for cfg in configurations
]
sources = [dgstreams.VideoSourceGizmo(cfg[“source”]) for cfg in configurations]
detectors = [dgstreams.AiSimpleGizmo(model) for model in models]
display = dgstreams.VideoDisplayGizmo(
[cfg[“display_name”] for cfg in configurations], show_ai_overlay=True, show_fps=True
)
pipeline = (
(source >> detector for source, detector in zip(sources, detectors)),
(detector >> display[di] for di, detector in enumerate(detectors)),
)
dgstreams.Composition(*pipeline).start()

6

Hi @The_Nguyen
Can you please try this simpler code and see what happens?

import degirum as dg, degirum_tools, cv2

# Camera sources: 0 and 2 represent connected webcam indices
sources = [2, 0]

# Hailo configuration
inference_host_address = "@local"  # Use IP if remote Hailo device
zoo_url = "degirum/models_hailort"
token = ""  # Leave empty if using public models

# Load the Hailo-compatible model using dg.load_model
model1 = dg.load_model(
    model_name="yolov8n_relu6_coco–640x640_quant_hailort_hailo8_1",
    inference_host_address=inference_host_address,
    zoo_url=zoo_url,
    token=token,    
)

model2 = dg.load_model(
    model_name="yolov8n_relu6_face–640x640_quant_hailort_hailo8_1",
    inference_host_address=inference_host_address
    zoo_url=zoo_url,
    token=token,
)

# Display and inference loop
with degirum_tools.Display("USB Cam 1") as display1, degirum_tools.Display("USB Cam 2") as display2:
    for result1, result2 in zip(
        degirum_tools.predict_stream(model1, sources[0]),
        degirum_tools.predict_stream(model2, sources[1])
    ):
        display1.show(result1)
        display2.show(result2)
1 Like
7

Hi @omria,
Sorry for the late reply. I’m currently experimenting with using 5 USB cameras with 5 different models on a Raspberry Pi 5 with AI HAT+ for my project. For now, I’m starting with 2 cameras and 2 models to observe performance. I’m not sure if the 26 TOPS from the AI HAT+ and the 8GB RAM of the Pi 5 will be enough to handle it. Thank you for your attention. If you have any information related to running multiple cameras and models simultaneously, please share it with me. Thank you!

8

Hi @shashi
This is a great solution—thank you very much for your enthusiastic support. The code works as intended, but after running for a while, it stops and throws the error:
“[HailoRT] [critical] Executing pipeline terminate failed with status HAILO_RPC_FAILED(77)”.
I’m not sure how to resolve this issue on the Pi 5 with 8GB RAM and AI HAT+ (26 TOPS). I would really appreciate your feedback. Thank you!

9

Hi @The_Nguyen
Glad to hear it is working for you. Regarding crashing after a while: can you let us know the kernel version? In the past few weeks, we saw users complaining that kernel 6.25 is causing stability issues.

10

Hi @shashi
Thanks you for your response. The kernel version is 6.12.25. if you have any suggestions for a more stable version, I will consider switching to ensure system stability. I look forward to your recommendation. Thanks you so much!

11

Hi @The_Nguyen
This is the official thread on the kernel issue and instructions: Raspberry Pi Kernel Compatibility Issue - Temporary Fix - General - Hailo Community

1 Like
12

Hi @shashi
Everything is working very well now. Thank you for your enthusiastic support.

1 Like
13

Hi @The_Nguyen
Glad to hear. You can adapt the code to your 5 camera, 5 model use case you mentioned as well. Let us know if you encounter any issues. Please note that there will be performance degradation due to switching the models. You may also run into weak host problem as RPi5 is not a very powerful host.

1 Like
14

Hi @shashi,
Sorry to bother you, but I would like to deploy 5 cameras along with 5 different models running on ROS 2 Humble. I have completely built ROS 2 Humble from source. However, I haven’t been able to find any documentation or resources about deploying Hailo-8 with ROS 2. Could you please provide me with some information or guidance on this? Thank you very much.

15

Hi @The_Nguyen
We are not familiar with the ROS 2 Humble ecosystem. Maybe the Hailo team can help.

1 Like