Problems with hailo_roundrobin using 11 cameras or more

Hello,

I’ve been working in a project using Hailo 8 with multiple cameras. For that, I used hailo_roundrobin, and this successfully worked using 10 cameras. However when I try 11 or more, a receive the error “gst_parse_error: could not link videoconvert_q_2 to fun (3)”.
Is there any limitation on the number o sink pads for hailo_roundrobin?

The pipeline is below, note that the error always happen in the sink pad after the sink pad number 11.

The GStreamer pipeline that I’m currently using:

hailomuxer name=hmux hailoroundrobin mode=0 name=fun ! tee name=t ! queue name=bypass_queue max-size-buffers=20 max-size-bytes=0 max-size-time=0 ! hmux.sink_0 t. ! videoscale ! video/x-raw,width=640,height=360 ! videobox border-alpha=0 top=-140 bottom=-140 ! video/x-raw,width=640,height=640 ! queue name=hailo_pre_infer_q_0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailonet hef-path=/var/velsis/velsis_hailo/models/yolov8s_v2_300epochs_h8l.hef batch-size=8 nms-score-threshold=0.2 nms-iou-threshold=0.7 output-format-type=HAILO_FORMAT_TYPE_FLOAT32 multi-process-service=true device-count=1 vdevice-key=1 ! queue name=queue_hailofilter max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailofilter function-name=yolov8s so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libyolo_hailortpp_post.so qos=false ! queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! queue name=queue_hailotracker max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailotracker name=hailo_tracker class-id=-1 kalman-dist-thr=0.7 iou-thr=0.8 init-iou-thr=0.9 keep-tracked-frames=3 keep-new-frames=3 keep-lost-frames=3 ! queue name=queue_hmuc max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! hmux.sink_1 hmux. ! queue name=queue_hailopython max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! identity name=identity_callback ! queue name=queue_hailooverlay max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailooverlay ! queue name=hailo_video_q_0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! videoconvert ! queue name=hailo_display_q_0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! fpsdisplaysink video-sink=fakesink name=hailo_display sync=false signal-fps-measurements=true rtspsrc location=rtsp://192.168.2.62:554/ISAPI/streaming/channels/102 name=src_1 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_1 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=1 qos=false ! queue name=queue_rtsp_1 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_1 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_1 ! fun.sink_1 rtspsrc location=rtsp://192.168.2.63:554/ISAPI/streaming/channels/102 name=src_10 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_10 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=10 qos=false ! queue name=queue_rtsp_10 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_10 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_10 ! fun.sink_10 rtspsrc location=rtsp://192.168.2.62:554/ISAPI/streaming/channels/102 name=src_11 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_11 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=11 qos=false ! queue name=queue_rtsp_11 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_11 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_11 ! fun.sink_11 rtspsrc location=rtsp://192.168.2.62:554/ISAPI/streaming/channels/102 name=src_2 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_2 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=2 qos=false ! queue name=queue_rtsp_2 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_2 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_2 ! fun.sink_2 rtspsrc location=rtsp://192.168.2.62:554/ISAPI/streaming/channels/102 name=src_3 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_3 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=3 qos=false ! queue name=queue_rtsp_3 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_3 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_3 ! fun.sink_3 rtspsrc location=rtsp://192.168.2.62:554/ISAPI/streaming/channels/102 name=src_4 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_4 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=4 qos=false ! queue name=queue_rtsp_4 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_4 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_4 ! fun.sink_4 rtspsrc location=rtsp://192.168.2.62:554/ISAPI/streaming/channels/102 name=src_5 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_5 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=5 qos=false ! queue name=queue_rtsp_5 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_5 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_5 ! fun.sink_5 rtspsrc location=rtsp://192.168.2.62:554/ISAPI/streaming/channels/102 name=src_6 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_6 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=6 qos=false ! queue name=queue_rtsp_6 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_6 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_6 ! fun.sink_6 rtspsrc location=rtsp://192.168.2.63:554/ISAPI/streaming/channels/102 name=src_7 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_7 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=7 qos=false ! queue name=queue_rtsp_7 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_7 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_7 ! fun.sink_7 rtspsrc location=rtsp://192.168.2.63:554/ISAPI/streaming/channels/102 name=src_8 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_8 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=8 qos=false ! queue name=queue_rtsp_8 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_8 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_8 ! fun.sink_8 rtspsrc location=rtsp://192.168.2.63:554/ISAPI/streaming/channels/102 name=src_9 latency=0 protocols=tcp retry=1000 retry-timeout=5000000 timeout=2000000 message-forward=true ! rtpbin ! queue ! hailofilter name=set_id_9 so-path=/workspace/tappas_v3.29.1/apps/h8/gstreamer/libs/post_processes/libstream_id_tool.so config-path=9 qos=false ! queue name=queue_rtsp_9 max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! decodebin ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! video/x-raw, format=I420 ! queue name=hailo_preprocess_q_9 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert name=videoconvert_q_9 ! fun.sink_9

There is no explicit hard limit on the number of sources that hailoroundrobin can handle. The element uses dynamic sink pads with the template sink_%u , meaning it can theoretically accept an unlimited number of input sources.

I think that next step would be to try to run it with GST_DEBUG=3 or GST_DEBUG=*:4 to catch element linking errors or caps negotiation issues early.

Thank you, that helped.
Indeed there is no limitation in hailo_roundrobin.

I was declaring the streamings pipeline for hailo_roundrobin in the following sequence: “1,10,11,2,3,4,5,6,7,8,9”. The right way to declare it is “1,2,3,4,5,6,7,8,9,10,11”, changing this all worked fine. But it is a mystery for me why “1,10,2,3,4,5,6,7,8,9” worked…