Processing live video stream from opencv on raspberry pi 5

General
,
1

I can’t process frames from the web camera via hailo 8 on raspberry pi 5. The web camera itself works, I checked it via qv4l2. I get a segmentation fault error when calling the get_roi_from_buffer function

if name == “main”:
user_data = user_app_callback_class()
app = GStreamerDetectionApp(app_callback, user_data)
#app.run()

cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()

    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    frame = frame.astype('uint8')

    roi = hailo.get_roi_from_buffer(frame) # error: segmentation fault
    
    cv2.imshow('frame', frame)
    if cv2.waitKey(1) == ord('q'):
        break
2

Hey @sherinkonstantiv,

Welcome to the Hailo Community! Great to have you here.

Regarding the segmentation fault you’re encountering, it’s important to note that the Hailo Raspberry Pi 5 examples are specifically designed to work with GStreamer pipelines for handling video streams. When you use OpenCV’s cv2.VideoCapture, the frames are processed as NumPy arrays, which are not compatible with Hailo’s hailo.get_roi_from_buffer() function. On the other hand, GStreamer pipelines provide Gst.Buffer objects that integrate seamlessly with the Hailo Raspberry Pi 5 examples.

To address this issue, you have two potential solutions:

  1. Use our GStreamer-based app and extract the frames from GStreamer. You can find an example of this approach in our Hailo Raspberry Pi 5 examples repository: GitHub - hailo-ai/hailo-rpi5-examples

  2. Alternatively, you can use our Python-based example, which you can find in the Hailo Application Code Examples repository: Hailo-Application-Code-Examples/runtime/python at main · hailo-ai/Hailo-Application-Code-Examples · GitHub

Both of these options should help you resolve the segmentation fault and enable you to successfully process video streams using the Hailo Raspberry Pi 5.

Let me know if you have any further questions or need additional assistance!

Best Regards,
Omria

3

Hi @sherinkonstantiv,
We developed a python SDK that makes working with Hailo8/Hailo8L easy. You find instructions and tutorials at: DeGirum/hailo_examples.

For your specific issue, our PySDK code looks as below. The code runs a yolov8n model on WebCam with index 0. If you have HAILO8 device instead of HAILO8L, just change model_name=yolov8n_relu6_coco--640x640_quant_hailort_hailo8_1

import degirum as dg, degirum_tools

model_name = "yolov8n_relu6_coco--640x640_quant_hailort_hailo8l_1"
video_source = 0

# load AI model
model = dg.load_model(
    model_name=model_name,
    inference_host_address=inference_host_address,
    zoo_url=zoo_url,
    token=token
)

# Run inference on video_source
with degirum_tools.Display("AI Camera") as output_display:
    for inference_result in degirum_tools.predict_stream(model, video_source):
        output_display.show(inference_result)

Please let u sknow if this helps or if you encounter any difficulties in running the above code.