Hi everyone,
I’m working on scaling up our object detection system using multiple Raspberry Pi High Quality Cameras. Our current setup works successfully with:
- Raspberry Pi 5
- Docker container
- Custom trained YOLOv8 model
- USB webcam
Based on the Hailo example: hailo-rpi5-examples/basic_pipelines/detection.py at main · hailo-ai/hailo-rpi5-examples · GitHub
I’m trying to implement a multi-camera setup using the RPi HQ Camera (specifically this model: https://www.raspberrypi.com/products/raspberry-pi-high-quality-camera/). I found the Accelerated Multi-Stream Pipeline example in the Tappas repository which seems promising: tappas/apps/h8/gstreamer/general/multistream_detection/README.rst at master · hailo-ai/tappas · GitHub
However, when running the multi_stream_detection.sh script after cloning the repo, I’m encountering an error.
root@raspberrypi:/home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection# ./multi_stream_detection.sh
Running yolov5
gst-launch-1.0 hailoroundrobin mode=0 name=fun ! queue name=hailo_pre_infer_q_0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailonet hef-path=/home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/yolov5m_wo_spp_60p.hef device-count=1 scheduling-algorithm=0 nms-score-threshold=0.3 nms-iou-threshold=0.45 output-format-type=HAILO_FORMAT_TYPE_FLOAT32 ! queue name=hailo_postprocess0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailofilter function-name=yolov5 so-path=/home/raspberry/tappas/apps/h8/gstreamer/libs/post_processes//libyolo_hailortpp_post.so qos=false ! queue name=hailo_draw0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailooverlay ! hailostreamrouter name=sid src_0::input-streams=“<sink_0>” src_1::input-streams=“<sink_1>” src_2::input-streams=“<sink_2>” src_3::input-streams=“<sink_3>” src_4::input-streams=“<sink_4>” src_5::input-streams=“<sink_5>” src_6::input-streams=“<sink_6>” src_7::input-streams=“<sink_7>” src_8::input-streams=“<sink_8>” src_9::input-streams=“<sink_9>” src_10::input-streams=“<sink_10>” src_11::input-streams=“<sink_11>” compositor name=comp start-time-selection=0 sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=640 sink_1::ypos=0 sink_2::xpos=1280 sink_2::ypos=0 sink_3::xpos=1920 sink_3::ypos=0 sink_4::xpos=0 sink_4::ypos=640 sink_5::xpos=640 sink_5::ypos=640 sink_6::xpos=1280 sink_6::ypos=640 sink_7::xpos=1920 sink_7::ypos=640 sink_8::xpos=0 sink_8::ypos=1280 sink_9::xpos=640 sink_9::ypos=1280 sink_10::xpos=1280 sink_10::ypos=1280 sink_11::xpos=1920 sink_11::ypos=1280 sink_12::xpos=0 sink_12::ypos=1920 sink_13::xpos=640 sink_13::ypos=1920 sink_14::xpos=1280 sink_14::ypos=1920 sink_15::xpos=1920 sink_15::ypos=1920 ! queue name=hailo_video_q_0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! videoconvert ! queue name=hailo_display_q_0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! fpsdisplaysink video-sink=xvimagesink name=hailo_display sync=false text-overlay=false uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection0.mp4 name=source_0 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_0 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_0 sid.src_0 ! queue name=comp_q_0 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_0 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection1.mp4 name=source_1 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_1 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_1 sid.src_1 ! queue name=comp_q_1 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_1 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection2.mp4 name=source_2 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_2 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_2 sid.src_2 ! queue name=comp_q_2 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_2 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection3.mp4 name=source_3 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_3 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_3 sid.src_3 ! queue name=comp_q_3 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_3 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection4.mp4 name=source_4 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_4 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_4 sid.src_4 ! queue name=comp_q_4 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_4 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection5.mp4 name=source_5 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_5 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_5 sid.src_5 ! queue name=comp_q_5 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_5 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection6.mp4 name=source_6 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_6 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_6 sid.src_6 ! queue name=comp_q_6 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_6 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection7.mp4 name=source_7 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_7 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_7 sid.src_7 ! queue name=comp_q_7 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_7 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection8.mp4 name=source_8 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_8 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_8 sid.src_8 ! queue name=comp_q_8 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_8 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection9.mp4 name=source_9 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_9 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_9 sid.src_9 ! queue name=comp_q_9 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_9 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection10.mp4 name=source_10 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_10 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_10 sid.src_10 ! queue name=comp_q_10 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_10 uridecodebin3 uri=file:///home/raspberry/tappas/apps/h8/gstreamer/general/multistream_detection/resources/detection11.mp4 name=source_11 ! videorate ! video/x-raw,framerate=30/1 ! queue name=hailo_preprocess_q_11 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! videoscale method=0 add-borders=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1 ! fun.sink_11 sid.src_11 ! queue name=comp_q_11 leaky=downstream max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! comp.sink_11
Running Pipeline…
Setting pipeline to PAUSED …
Could not load lib /home/raspberry/tappas/apps/h8/gstreamer/libs/post_processes//libyolo_hailortpp_post.so: cannot open shared object file: No such file or directory
Cannot load symbol: /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailotools.so: undefined symbol: yolov5
Caught SIGSEGV
#0 0x00007fff114982e4 in __GI___wait4 (pid=5963, stat_loc=0x7ffff000ee1c, options=0, usage=0x0) at …/sysdeps/unix/sysv/linux/wait4.c:30
#1 0x00007fff116447d0 in g_on_error_stack_trace () at /lib/aarch64-linux-gnu/libglib-2.0.so.0
#2 0x000055555cf0626c in ()
#3 0x00007fff119587b0 in ()
#4 _dl_close (_map=0x0) at ./elf/dl-close.c:755
#5 0x00007fff1150e9ec in __GI__dl_catch_exception (exception=exception@entry=0x7ffff0010380, operate=0x7fff11921d80 <_dl_close>, args=0x0) at ./elf/dl-error-skeleton.c:208
#6 0x00007fff1150eab0 in __GI__dl_catch_error (objname=0x7ffff00103f8, errstring=0x7ffff0010400, mallocedp=0x7ffff00103f7, operate=, args=) at ./elf/dl-error-skeleton.c:227
#7 0x00007fff1145a8e0 in _dlerror_run (operate=, args=) at ./dlfcn/dlerror.c:138
#8 0x00007fff1145a610 in __dlclose (handle=) at ./dlfcn/dlclose.c:31
#9 0x00007fff10c81f1c in () at /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailotools.so
#10 0x00007fff10b7b97c in () at /lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#11 0x00007fff10b7bbe4 in () at /lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#12 0x00007fff118162f8 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#13 0x00007fff11816a18 in gst_pad_set_active () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#14 0x00007fff117eba78 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#15 0x00007fff118041f8 in gst_iterator_fold () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#16 0x00007fff117ec16c in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#17 0x00007fff117ee808 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#18 0x00007fff117eeb94 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#19 0x00007fff117f10ec in gst_element_change_state () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#20 0x00007fff117f1788 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#21 0x00007fff117ca1e4 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#22 0x00007fff1181ee14 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#23 0x00007fff117f10ec in gst_element_change_state () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#24 0x00007fff117f1128 in gst_element_change_state () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#25 0x00007fff117f1788 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#26 0x000055555cf05d6c in ()
#27 0x00007fff11407740 in __libc_start_call_main (main=main@entry=0x55555cf038c0, argc=argc@entry=520, argv=argv@entry=0x7ffff0011098) at …/sysdeps/nptl/libc_start_call_main.h:58
#28 0x00007fff11407818 in __libc_start_main_impl (main=0x55555cf038c0, argc=520, argv=0x7ffff0011098, init=, fini=, rtld_fini=, stack_end=) at …/csu/libc-start.c:360
#29 0x000055555cf03930 in ()
Spinning. Please run ‘gdb gst-launch-1.0 5939’ to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.
It looks like that we are supposed to run instal.sh script first but I’m getting an error running it as well:
dpkg: error: cannot access archive ‘/home/raspberry/tappas/hailort/hailort_*_amd64.deb’: No such file or directory
Has anyone successfully set up multiple RPi serial interface connector cameras with Hailo for object detection?
Any guidance on the correct process or alternative solutions would be greatly appreciated.
Thank you
Vadim