Multiple RPi HQ Cameras Object Detection with Hailo and YOLOv8

Hi everyone,

I’m working on scaling up our object detection system using multiple Raspberry Pi High Quality Cameras. Our current setup works successfully with:

  • Raspberry Pi 5
  • Docker container
  • Custom trained YOLOv8 model
  • USB webcam

Based on the Hailo example: hailo-rpi5-examples/basic_pipelines/detection.py at main · hailo-ai/hailo-rpi5-examples · GitHub

I’m trying to implement a multi-camera setup using the RPi HQ Camera (specifically this model: https://www.raspberrypi.com/products/raspberry-pi-high-quality-camera/). I found the Accelerated Multi-Stream Pipeline example in the Tappas repository which seems promising: tappas/apps/h8/gstreamer/general/multistream_detection/README.rst at master · hailo-ai/tappas · GitHub

However, when running the multi_stream_detection.sh script after cloning the repo, I’m encountering an error.

root@raspberrypi:/home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection# ./multi_stream_detection.sh
Running yolov5
gst-launch-1.0 hailoroundrobin mode=0 name=fun ! queue name=hailo_pre_infer_q_0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailonet hef-path=/home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/yolov5m_wo_spp_60p.hef device-count=1 scheduling-algorithm=0 nms-score-threshold=0.3 nms-iou-threshold=0.45 output-format-type=HAILO_FORMAT_TYPE_FLOAT32 ! queue name=hailo_postprocess0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailofilter function-name=yolov5 so-path=/home/raspberry/tappas/apps/h8/gstreamer/libs/post_processes//libyolo_hailortpp_post.so qos=false ! queue name=hailo_draw0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailooverlay ! hailostreamrouter name=sid src_0::input-streams=“<sink_0>” src_1::input-streams=“<sink_1>” src_2::input-streams=“<sink_2>” src_3::input-streams=“<sink_3>” src_4::input-streams=“<sink_4>” src_5::input-streams=“<sink_5>” src_6::input-streams=“<sink_6>” src_7::input-streams=“<sink_7>” src_8::input-streams=“<sink_8>” src_9::input-streams=“<sink_9>” src_10::input-streams=“<sink_10>” src_11::input-streams=“<sink_11>” compositor name=comp start-time-selection=0 sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=640 sink_1::ypos=0 sink_2::xpos=1280 sink_2::ypos=0 sink_3::xpos=1920 sink_3::ypos=0 sink_4::xpos=0 sink_4::ypos=640 sink_5::xpos=640 sink_5::ypos=640 sink_6::xpos=1280 sink_6::ypos=640 sink_7::xpos=1920 sink_7::ypos=640 sink_8::xpos=0 sink_8::ypos=1280 sink_9::xpos=640 sink_9::ypos=1280 sink_10::xpos=1280 sink_10::ypos=1280 sink_11::xpos=1920 sink_11::ypos=1280 sink_12::xpos=0 sink_12::ypos=1920 sink_13::xpos=640 sink_13::ypos=1920 sink_14::xpos=1280 sink_14::ypos=1920 sink_15::xpos=1920 sink_15::ypos=1920 ! queue name=hailo_video_q_0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! videoconvert ! queue name=hailo_display_q_0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! fpsdisplaysink video-sink=xvimagesink name=hailo_display sync=false text-overlay=false uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection0.mp4 name=source_0 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_0 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_0 sid.src_0 ! queue name=comp_q_0 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_0 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection1.mp4 name=source_1 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_1 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_1 sid.src_1 ! queue name=comp_q_1 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_1 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection2.mp4 name=source_2 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_2 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_2 sid.src_2 ! queue name=comp_q_2 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_2 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection3.mp4 name=source_3 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_3 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_3 sid.src_3 ! queue name=comp_q_3 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_3 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection4.mp4 name=source_4 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_4 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_4 sid.src_4 ! queue name=comp_q_4 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_4 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection5.mp4 name=source_5 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_5 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_5 sid.src_5 ! queue name=comp_q_5 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_5 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection6.mp4 name=source_6 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_6 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_6 sid.src_6 ! queue name=comp_q_6 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_6 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection7.mp4 name=source_7 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_7 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_7 sid.src_7 ! queue name=comp_q_7 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_7 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection8.mp4 name=source_8 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_8 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_8 sid.src_8 ! queue name=comp_q_8 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_8 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection9.mp4 name=source_9 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_9 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_9 sid.src_9 ! queue name=comp_q_9 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_9 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection10.mp4 name=source_10 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_10 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_10 sid.src_10 ! queue name=comp_q_10 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_10 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection11.mp4 name=source_11 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_11 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_11 sid.src_11 ! queue name=comp_q_11 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_11
Running Pipeline…
Setting pipeline to PAUSED …
Could not load lib /home/raspberry/tappas/apps/h8/gstreamer/libs/post_processes//libyolo_hailortpp_post.so: cannot open shared object file: No such file or directory
Cannot load symbol: /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailotools.so: undefined symbol: yolov5
Caught SIGSEGV
#0 0x00007fff114982e4 in __GI___wait4 (pid=5963, stat_loc=0x7ffff000ee1c, options=0, usage=0x0) at …/sysdeps/unix/sysv/linux/wait4.c:30
#1 0x00007fff116447d0 in g_on_error_stack_trace () at /lib/aarch64-linux-gnu/libglib-2.0.so.0
#2 0x000055555cf0626c in ()
#3 0x00007fff119587b0 in ()
#4 _dl_close (_map=0x0) at ./elf/dl-close.c:755
#5 0x00007fff1150e9ec in __GI__dl_catch_exception (exception=exception@entry=0x7ffff0010380, operate=0x7fff11921d80 <_dl_close>, args=0x0) at ./elf/dl-error-skeleton.c:208
#6 0x00007fff1150eab0 in __GI__dl_catch_error (objname=0x7ffff00103f8, errstring=0x7ffff0010400, mallocedp=0x7ffff00103f7, operate=, args=) at ./elf/dl-error-skeleton.c:227
#7 0x00007fff1145a8e0 in _dlerror_run (operate=, args=) at ./dlfcn/dlerror.c:138
#8 0x00007fff1145a610 in __dlclose (handle=) at ./dlfcn/dlclose.c:31
#9 0x00007fff10c81f1c in () at /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailotools.so
#10 0x00007fff10b7b97c in () at /lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#11 0x00007fff10b7bbe4 in () at /lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#12 0x00007fff118162f8 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#13 0x00007fff11816a18 in gst_pad_set_active () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#14 0x00007fff117eba78 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#15 0x00007fff118041f8 in gst_iterator_fold () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#16 0x00007fff117ec16c in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#17 0x00007fff117ee808 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#18 0x00007fff117eeb94 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#19 0x00007fff117f10ec in gst_element_change_state () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#20 0x00007fff117f1788 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#21 0x00007fff117ca1e4 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#22 0x00007fff1181ee14 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#23 0x00007fff117f10ec in gst_element_change_state () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#24 0x00007fff117f1128 in gst_element_change_state () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#25 0x00007fff117f1788 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#26 0x000055555cf05d6c in ()
#27 0x00007fff11407740 in __libc_start_call_main (main=main@entry=0x55555cf038c0, argc=argc@entry=520, argv=argv@entry=0x7ffff0011098) at …/sysdeps/nptl/libc_start_call_main.h:58
#28 0x00007fff11407818 in __libc_start_main_impl (main=0x55555cf038c0, argc=520, argv=0x7ffff0011098, init=, fini=, rtld_fini=, stack_end=) at …/csu/libc-start.c:360
#29 0x000055555cf03930 in ()
Spinning. Please run ‘gdb gst-launch-1.0 5939’ to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

It looks like that we are supposed to run instal.sh script first but I’m getting an error running it as well:

dpkg: error: cannot access archive ‘/home/raspberry/tappas/hailort/hailort_*_amd64.deb’: No such file or directory

Has anyone successfully set up multiple RPi serial interface connector cameras with Hailo for object detection?

Any guidance on the correct process or alternative solutions would be greatly appreciated.

Thank you
Vadim

Hi Vadim,

I think the issue is with the installation of Tappas on RPI5.

Please run the following command:
./install.sh --skip-hailort --target-platform rpi5 --core-only

regards,

Victor

Hi Victor,

It doesn’t seemt to help

root@raspberrypi:/home/raspberry/tappas# ./install.sh --skip-hailort --target-platform rpi5 --core-only

No TAPPAS_WORKSPACE in environment found, using the default one /home/raspberry/tappas

Hailo Tappas was removed successfully.

To unset Tappas environment variables, please relogin or reboot the PC.

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

HailoRT sources directory not found (/home/raspberry/tappas/hailort/sources), Please follow our manual installation guide

Thank you,
Vadim

It also seems to be broken on a brand new RPI5 setup:

  1. git clone GitHub - hailo-ai/tappas: High-performance, optimized pre-trained template AI application pipelines for systems using Hailo devices
  2. cd tappas
  3. ./install.sh --skip-hailort --target-platform rpi5 --core-only
No TAPPAS_WORKSPACE in environment found, using the default one /home/user/tappas
Hailo Tappas was removed successfully.
To unset Tappas environment variables, please relogin or reboot the PC.

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

WARNING: Requirement x11-utils not found.
WARNING: Requirement python-gi-dev not found.
WARNING: Requirement libcairo2-dev not found.
WARNING: Requirement libgirepository1.0-dev not found.
WARNING: Requirement libgstreamer1.0-dev not found.
WARNING: Requirement cmake not found.
WARNING: Requirement libgstreamer-plugins-base1.0-dev not found.
WARNING: Requirement libzmq3-dev not found.
WARNING: Requirement libgstreamer-plugins-bad1.0-dev not found.
WARNING: Requirement gstreamer1.0-tools not found.
WARNING: Requirement opencv4 not found.
Package opencv4 was not found in the pkg-config search path.
Perhaps you should add the directory containing `opencv4.pc'
to the PKG_CONFIG_PATH environment variable
Package 'opencv4', required by 'virtual:world', not found
WARNING: Requirement opencv_imgproc not found.
WARNING: Requirement opencv_core not found.
WARNING: Requirement opencv_imgcodec not found.
WARNING: Requirement opencv_features2d not found.
WARNING: Requirement opencv_calib3d not found.
WARNING: Requirement opencv_flann not found.
WARNING: Requirement Gstreamer not found.

SYSTEM REQUIREMENTS REPORT

Component           Requirement                       Found       
==========          ==========                        ==========  ==========
Apt-Package         ffmpeg                            V           Required
Apt-Package         x11-utils                         X           Required
Apt-Package         python3-dev                       V           Required
Apt-Package         python3-pip                       V           Required
Apt-Package         python3-setuptools                V           Required
Apt-Package         libgirepository                   V           Required
Apt-Package         gcc-12                            V           Required
Apt-Package         g++-12                            V           Required
Apt-Package         python-gi-dev                     X           Required
Apt-Package         pkg-config                        V           Required
Apt-Package         libcairo2-dev                     X           Required
Apt-Package         libgirepository1.0-dev            X           Required
Apt-Package         libgstreamer1.0-dev               X           Required
Apt-Package         cmake                             X           Required
Apt-Package         libgstreamer-plugins-base1.0-dev  X           Required
Apt-Package         libzmq3-dev                       X           Required
Apt-Package         rsync                             V           Required
Apt-Package         git                               V           Required
Apt-Package         libgstreamer-plugins-bad1.0-dev   X           Required
Apt-Package         gstreamer1.0-plugins-base         V           Required
Apt-Package         gstreamer1.0-plugins-good         V           Required
Apt-Package         gstreamer1.0-plugins-bad          V           Required
Apt-Package         gstreamer1.0-libav                V           Required
Apt-Package         gstreamer1.0-tools                X           Required
Apt-Package         gstreamer1.0-x                    V           Required
Pkg_config-Package  opencv4                           X           Required
Pkg_config-Package  opencv_imgproc                    X           Required
Pkg_config-Package  opencv_core                       X           Required
Pkg_config-Package  opencv_imgcodec                   X           Required
Pkg_config-Package  opencv_features2d                 X           Required
Pkg_config-Package  opencv_calib3d                    X           Required
Pkg_config-Package  opencv_flann                      X           Required
Package             Gstreamer                         X           Required

See system_reqs_results.log for more information.

3. apps/h8/gstreamer/general/multistream_detection/multi_stream_detection.sh

/home/user/tappas/apps/h8/gstreamer/general/multistream_detection/../../../../../scripts/misc/checks_before_run.sh: line 128: xvinfo: command not found
ERROR: hailortcli was not found
  1. sudo apt install hailo-all

  2. hailortcli fw-control identify

[HailoRT] [error] CHECK failed - Driver version (4.18.0) is different from library version (4.20.0)
[HailoRT] [error] Driver version mismatch, status HAILO_INVALID_DRIVER_VERSION(76)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_INVALID_DRIVER_VERSION(76)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_INVALID_DRIVER_VERSION(76)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_INVALID_DRIVER_VERSION(76)
[HailoRT CLI] [error] CHECK_SUCCESS failed with status=HAILO_INVALID_DRIVER_VERSION(76)
  1. gst-inspect-1.0 hailotools
Name                     hailotools
  Description              hailo tools plugin
  Filename                 /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailotools.so
  Version                  3.31.0
  License                  unknown
  Source module            gst-hailo-tools
  Binary package           gst-hailo-tools
  Origin URL               https://hailo.ai/

  hailoaggregator: hailoaggregator - Cascading
  hailocounter: hailocounter - postprocessing element
  hailocropper: hailocropper
  hailoexportfile: hailoexportfile - export element
  hailoexportzmq: hailoexportzmq - export element
  hailofilter: hailofilter - postprocessing element
  hailogallery: Hailo gallery element
  hailograytonv12: hailograytonv12 - postprocessing element
  hailoimportzmq: hailoimportzmq - import element
  hailomuxer: Muxer pipeline merging
  hailonv12togray: hailonv12togray - postprocessing element
  hailonvalve: HailoNValve element
  hailooverlay: hailooverlay - overlay element
  hailoroundrobin: Input Round Robin element
  hailostreamrouter: Hailo Stream Router
  hailotileaggregator: hailotileaggregator
  hailotilecropper: hailotilecropper - Tiling
  hailotracker: Hailo object tracking element


  18 features:
  +-- 18 elements
  1. gst-inspect-1.0 hailo
Plugin Details:
  Name                     hailo
  Description              hailo gstreamer plugin
  Filename                 /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailo.so
  Version                  1.0
  License                  unknown
  Source module            hailo
  Binary package           GStreamer
  Origin URL               http://gstreamer.net/

  hailodevicestats: hailodevicestats element
  hailonet: hailonet element
  synchailonet: sync hailonet element

  3 features:
  +-- 3 elements

Can anybody please check and confirm if provided example isn’t broken.

Thank you,
Vadim

You are missing one step, which is to clone the Hailo-RT repo as explained here:

cd tappas_VERSION
mkdir hailort
git clone GitHub - hailo-ai/hailort: An open source light-weight and high performance inference framework for Hailo devices hailort/sources