Hailo 8L Raspberry pi gstreamer parallel inference

General
,
1

Hello,
i tried to implement example of running two networks in parallel as specified under
this link : tappas/docs/pipelines/parallel_networks.rst at master · hailo-ai/tappas · GitHub

so i modified GStreamerPoseEstimationApp in terms of creating pipeline like this:

    self.POSE_DET_PIPELINE = f"queue name=queue_hailonet1 ! \
        videoconvert n-threads=3 ! \
        hailonet hef-path={self.pose_hef_path} is-active=true batch-size={self.batch_size} force-writable=true ! \
        queue name=queue_hailofilter1 ! \
        hailofilter function-name={self.post_function_name} so-path={self.pose_default_postprocess_so} qos=false !"
   
    self.OBJ_DET_PIPELINE = (
         QUEUE("queue_hailonet2")
        + "videoconvert n-threads=3 ! "
        f"hailonet hef-path={self.objdet_hef_path} is-active=true batch-size={self.batch_size} {self.thresholds_str} force-writable=true ! "
        + QUEUE("queue_hailofilter2")
        + f"hailofilter so-path={self.objdet_default_postprocess_so} {self.labels_config} qos=false ! "
      
    )

    pipeline_string = (
        "hailomuxer name=hmux "
        + source_element
        + "tee name=t ! "
        + QUEUE("bypass_queue", max_size_buffers=20)
        + "hmux.sink_0 "
        + "t. ! "
        + self.OBJ_DET_PIPELINE
        + QUEUE("queue_hmuc")
        + "hmux.sink_1 "
        + "hmux. ! "
        + QUEUE("queue_hailo_python")
        + QUEUE("queue_hailooverlay")
        + "hailooverlay ! "
        + QUEUE("queue_videoconvert")
        + "videoconvert n-threads=3 qos=false ! "
        + QUEUE("queue_hailo_display")
        + f"fpsdisplaysink video-sink={self.video_sink} name=hailo_display sync={self.sync} text-overlay={self.options_menu.show_fps} signal-fps-measurements=true "
        + "t. ! "
        + self.POSE_DET_PIPELINE
        + QUEUE("queue_hailo_python2")
        + QUEUE("queue_hailooverlay2")
        + "hailooverlay ! "
        + QUEUE("queue_videoconvert2")
        + "videoconvert n-threads=3 qos=false ! "
        + QUEUE("queue_hailo_display2")
        + f"fpsdisplaysink video-sink={self.video_sink} name=hailo_display2 sync={self.sync} text-overlay={self.options_menu.show_fps} "
    )

here is error it gives:
CHECK_EXPECTED failed with status=74
[HailoRT] [error] CHECK failed - Failed to create vdevice. there are not enough free devices. requested: 1, found: 0
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_OUT_OF_PHYSICAL_DEVICES(74)

Could you please suggest what is wrong ?

2

Hey @kyurrii ,

When encountering the error:

CHECK_EXPECTED failed with status=74
[HailoRT] [error] CHECK failed - Failed to create vdevice. there are not enough free devices. requested: 1, found: 0
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_OUT_OF_PHYSICAL_DEVICES(74)

Root Causes:

  1. A Hailo device might be busy with previous operations or hung processes
  2. Multiple networks attempting to run in parallel without proper resource allocation
  3. Insufficient virtual device configuration
  4. Memory resource constraints due to high batch sizes
  5. Improper network scheduling

Solution Steps:

  1. Initial Troubleshooting:

    • Use htop to identify any lingering Hailo applications
    • Kill any stuck processes using kill -9 <PID>
    • Verify device availability using hailoctl list

  2. Pipeline Modifications:

    a) Initialize Virtual Device:
    Before running the pipeline, set up virtual device allocation:

    hailo_vdevice --device-count=2

    b) Update Pipeline Configuration:
    Modify your GStreamer pipeline with these key changes:

    hailonet hef-path={self.pose_hef_path} \
is-active=true \
batch-size=1 \
force-writable=true \
device-count=2 \
scheduling-algorithm=ROUND_ROBIN !

  3. Complete Pipeline Example for Parallel Processing:

    pipeline_string = (
    "hailomuxer name=hmux "
    + source_element
    + "tee name=t ! "
    + QUEUE("bypass_queue", max_size_buffers=20)
    + "hmux.sink_0 "
    + "t. ! "
    + "queue name=queue_hailonet1 ! "
    + "videoconvert n-threads=3 ! "
    + f"hailonet hef-path={self.pose_hef_path} is-active=true batch-size=1 force-writable=true device-count=2 scheduling-algorithm=ROUND_ROBIN ! "
    + QUEUE("queue_hailofilter1")
    + f"hailofilter so-path={self.pose_default_postprocess_so} qos=false ! "
    + "hmux.sink_1 "
    # ... Additional pipeline components for object detection network
)

  4. Key Pipeline Optimizations:

    • Set batch-size=1 to prevent memory issues
    • Enable device-count=2 for parallel execution
    • Implement scheduling-algorithm=ROUND_ROBIN for efficient resource sharing
    • Use proper queue management with QUEUE() functions
    • Configure videoconvert with multiple threads (n-threads=3)

  5. Resource Management:

    • Ensure proper linking between hailonet and hailofilter
    • Use hailomuxer for managing multiple network outputs
    • Implement proper queue management to prevent bottlenecks
    • Configure display settings with fpsdisplaysink

Please try implementing these changes in your pipeline and let me know if you encounter any errors or issues - I’ll be happy to help troubleshoot them further.

3

Hi @omria,
thanks for fast answer. Can you suggest how to set up virtual machine on Raspberry pi5 Bookworm. I found only this guide: Enabling Virtual Machine to access Hailo-8 devices but obviousely it is not applicable in this case.

I tried command you provided, but get this error:

(venv_hailo_rpi5_examples) pi@raspberrypi:~ $ hailo_vdevice --device-count=2
bash: hailo_vdevice: command not found

4

Hi @omria, so i followed the suggestion and when i add to parameters to hailonet element f"hailonet hef-path={self.pose_hef_path} is-active=true batch-size=1 force-writable=true device-count=2 scheduling-algorithm=ROUND_ROBIN ! " , i get error:
gst_parse_error: could not set property “scheduling-algorithm” in element “hailonet” to “ROUND_ROBIN” (4). Please advice solution, thanks .