How to use two cameras with Yolo modules for RPI5 with HAT+ Hailo ?

General
1

Hello, Peers! I have connected two cameras to both ports of my RPI5 with HAT+ Hailo boards and a have well tested operation of Yolo modules separately with each camera. But how can I use Yolo modules for object detection with both cameras simultaneously? I woild like to see images from both cameras on my monitor. What should I change in the code for that? Thank you!

2

Hey @Igor_Zaitsev1 ,

You can combine two USB cameras in a single GStreamer pipeline while using the Hailo framework for inference. Here’s how:

Create a composite pipeline that:

  • Takes input from two cameras (/dev/video0 and /dev/video1)
  • Combines both streams using the compositor element
  • Runs the combined output through your inference pipeline

Implementation Example ( Not Tested !)

Modify your get_pipeline_string() method in the GStreamerDetectionApp class:

def get_pipeline_string(self):
    cam0 = "/dev/video0"
    cam1 = "/dev/video1"
    width, height = 640, 480  # Resolution for each camera

    # Camera 0 pipeline
    cam0_pipeline = (
        f'v4l2src device={cam0} ! image/jpeg, width={width}, height={height}, framerate=30/1 ! '
        f'decodebin ! videoconvert ! videoscale ! '
        f'video/x-raw, width={width}, height={height} ! '
        f'queue ! compositor.sink_0 '
    )

    # Camera 1 pipeline
    cam1_pipeline = (
        f'v4l2src device={cam1} ! image/jpeg, width={width}, height={height}, framerate=30/1 ! '
        f'decodebin ! videoconvert ! videoscale ! '
        f'video/x-raw, width={width}, height={height} ! '
        f'queue ! compositor.sink_1 '
    )

    # Compositor element
    compositor = (
        f'compositor name=compositor sink_0::xpos=0 sink_1::xpos={width} ! '
        f'videoconvert ! videoscale ! '
        f'video/x-raw, width={width*2}, height={height} ! '
    )

    # Add inference and display pipelines
    inference = INFERENCE_PIPELINE(...)
    display = DISPLAY_PIPELINE(...)

    return cam0_pipeline + cam1_pipeline + compositor + inference + display

Key Considerations

  • Resolution: Ensure the combined output resolution matches your model’s input requirements
  • Performance: Keep framerates equal for better synchronization
  • Bounding Boxes: Will appear across the combined frame with no separation between cameras
  • Layout: Adjust positions with sink_0::xpos and sink_1::xpos parameters

If you prefer switching between cameras rather than combining them, replace compositor with input-selector.

We will be adding Guides and more pipeline examples in the near future !

3

Dear Sir,
thank you so much for this detailed answer!
I will try to folow.
Best regards