Streaming object detection video from Raspberry Pi to web page using GStreamer and Flask

General
1

Hello everyone,

Im working on the hailo rpi5 example for object detection GitHub - hailo-ai/hailo-rpi5-examples at 0c70ac577677d997c0c0b38290670466ed3fe9a2

For the moment im able to launch the detection with my raspberry pi module 2 camera. So a window is opening with the video feedback and the detection.

But i would like to have this detection on a webpage using flask. I think that i have to change the display sink in the display_pipeline. Thats why i changed the line

f'fpsdisplaysink name={name} video-sink={video_sink} sync={sync} text-overlay={show_fps} signal-fps-measurements=true '```


by the line :

f’jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000’

in this code :

def DISPLAY_PIPELINE(video_sink=‘autovideosink’, sync=‘true’, show_fps=‘false’, name=‘hailo_display’):
“”"
Creates a GStreamer pipeline string for displaying the video.
It includes the hailooverlay plugin to draw bounding boxes and labels on the video.

Args:
    video_sink (str, optional): The video sink element to use. Defaults to 'autovideosink'.
    sync (str, optional): The sync property for the video sink. Defaults to 'true'.
    show_fps (str, optional): Whether to show the FPS on the video sink. Should be 'true' or 'false'. Defaults to 'false'.
    name (str, optional): The prefix name for the pipeline elements. Defaults to 'hailo_display'.

Returns:
    str: A string representing the GStreamer pipeline for displaying the video.
"""
# Construct the display pipeline string
display_pipeline = (
    f'{OVERLAY_PIPELINE(name=f"{name}_overlay")} ! '
    f'{QUEUE(name=f"{name}_videoconvert_q")} ! '
    f'videoconvert name={name}_videoconvert n-threads=2 qos=false ! '
    f'{QUEUE(name=f"{name}_q")} ! '
    f'jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000'
    #f'fpsdisplaysink name={name} video-sink={video_sink} sync={sync} text-overlay={show_fps} signal-fps-measurements=true '
)

return display_pipeline


So i think that by doing this when i launch the code using python detection.py --input rpi i create a udp server host=127.0.0.1 port=5000 instead of creating a video output.

So to have the video on the web page using flask i need to connect my flask webpage as a client to this udp host server. 

But i dont know where to integrate this in the code of detection.py or on of the other script that are use in this example.

I asked to chatgpt and he proposed to create a stream_server.py :

from flask import Flask, Response, render_template
import cv2

app = Flask(name)

def generate_frames():
cap = cv2.VideoCapture(“udp://127.0.0.1:5000”, cv2.CAP_FFMPEG)
while True:
success, frame = cap.read()
if not success:
print(“Impossible de lire une image.”)
break
else:
print(“Image lue avec succès.”)
ret, buffer = cv2.imencode(‘.jpg’, frame)
frame = buffer.tobytes()
yield (b’–frame\r\n’
b’Content-Type: image/jpeg\r\n\r\n’ + frame + b’\r\n’)

@app.route(‘/’)
def index():
return render_template(‘index.html’)

@app.route(‘/video_feed’)
def video_feed():
return Response(generate_frames(), mimetype=‘multipart/x-mixed-replace; boundary=frame’)

if name == ‘main’:
app.run(host=‘0.0.0.0’, port=8080)


and a template html :
Object Detection Stream

Detection d'objets

```

and then to launch detection.py and after stream_server.py but it doesnt work for me. Does anyone know why or have a suggestion to display the video on a flask web page? thanks

2

after searching i changed the line :

f’jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000’

by

f’videoconvert ! openh264enc ! mpegtsmux ! rtpmp2tpay ! udpsink host=127.0.0.1 port=6976’

and it works !

1 Like