Hi,
I’ve modified the gstreamer pipeline to provide arguments for class label and json filepath for detecting a specific class and saving the results to a json file.
The data is getting saved when I run detection.sh, but I further wanna know the frame number as well, i.e. in which frame the detections have happened.
Like first 10 detections in frame 1, next 15 in frame 2 and like that.
Is there a way to do it using the gstreamer pipeline
1 Like
this is from my detection.py
for detection in detections:
label = detection.get_label()
bbox = detection.get_bbox()
confidence = detection.get_confidence()
if label == “Madsen”:
# Get track ID
track_id = 0
track = detection.get_objects_typed(hailo.HAILO_UNIQUE_ID)
if len(track) == 1:
track_id = track[0].get_id()
string_to_print += (f"Detection: ID: {track_id} Label: {label} Confidence: {confidence:.2f} Detection count: {detection_count}\n")
and
user_data.increment()
string_to_print = f"Frame count: {user_data.get_count()}\n"
You’re using model zoo or the tappas gstreamer pipelines, cuz I was trying to figure out if its possible to do it via the latter one.
This ones seems a model zoo implementation.
yes it is the rpi 5 examples in virtual inv.
the real magic happens here hailo-rpi5-examples/venv_hailo_rpi5_examples/lib/python3.11/site-packages/hailo_apps_infra/gstreamer_app.py
It just run 30 fps (camera speed) without any trouble. I am sending the string data to a named pipe. This makes sure that i am not slowing the hailo8 . It also make future upgrades easy and ofc. my c++ program runs in another thread. just reading the pipe data. I dont need frame data for anything other than control af the hailo8.
Good,
I’m actually using a MINI PC and was testing the detection.sh file in the tappas, in that I managed to change the original post processing code to give arguments for specific class detection and saving the metadata as well, couldn’t find any functionality to tell me the frame number or frame id to verify my post processing results like if there were 3 people in 15th frame, we have received detection metadata for 3 people only, if not then the detection might be getting compromised and needs retraining, something like that.
Now is there a way to do something like this, where I can save the frame number to verify my results from metadata, it could be then used for rtsp as well as mp4 videos.
I am building my own detection model.
I have made some small tools to take pictures out of a video frame (opencv).
You have to retrain your model. Standard model are only for general use and not accurate. I am also learning here . If you are having specific needs you have to retrain the model. It is difficult to find info about what we can get out of the hailo frame . I hobe that they will do some documentet som day.
there is a little look here. →
Okay, I’ve got some idea regarding the gstreamer buffer that is storing the frames, will see if some changes can be made to have a count of the processed frames along with their detections, also If the timestamp for the post processing can be saved, then I might be able to track down the frame number.
1 Like
gstreamer_app.py on my rpi 5
At the button of the page it just shows it.
But look at the callback class
def increment(self):
self.frame_count += 1
it is in gstreamer.py all the magic happens. you can change the default behavior there.
I have got it working on my rpi .
The user frame DID not show the bounding boxes ,so I have to write them again. ???
I got my on class with my dog only.
And the frame rate drops from 69 to 39 
python3 basic_pipelines/madsen.py --hef-path resources/madsens.hef --input rpi --labels-json resources/madsen.json --input madsentest.mp4 --disable-sync --show-fps --use-frame
--use-frame show user frame.
import gi
gi.require_version('Gst', '1.0')
from gi.repository import Gst, GLib
import os
import numpy as np
import cv2
import hailo
import sys
from hailo_apps_infra.hailo_rpi_common import (
get_caps_from_pad,
get_numpy_from_buffer,
app_callback_class,
)
from hailo_apps_infra.detection_pipeline import GStreamerDetectionApp
# -----------------------------------------------------------------------------------------------
# User-defined class to be used in the callback function
# -----------------------------------------------------------------------------------------------
# Inheritance from the app_callback_class
class user_app_callback_class(app_callback_class):
def __init__(self):
super().__init__()
self.new_variable = 42 # New variable example
def new_function(self): # New function example
return "The meaning of life is: "
# -----------------------------------------------------------------------------------------------
# User-defined callback function
# -----------------------------------------------------------------------------------------------
# This is the callback function that will be called when data is available from the pipeline
def app_callback(pad, info, user_data):
# Get the GstBuffer from the probe info
buffer = info.get_buffer()
# Check if the buffer is valid
if buffer is None:
return Gst.PadProbeReturn.OK
# Using the user_data to count the number of frames
user_data.increment()
#user_data.use_frame = True
string_to_print = f"Frame count: {user_data.get_count()}\n"
# Get the caps from the pad
format, width, height = get_caps_from_pad(pad)
# If the user_data.use_frame is set to True, we can get the video frame from the buffer
frame = None
if user_data.use_frame and format is not None and width is not None and height is not None:
# Get video frame
frame = get_numpy_from_buffer(buffer, format, width, height)
string_to_print += " frame ok "
# Get the detections from the buffer
roi = hailo.get_roi_from_buffer(buffer)
detections = roi.get_objects_typed(hailo.HAILO_DETECTION)
# Parse the detections
detection_count = 0
for detection in detections:
detection_count += 1
label = detection.get_label()
bbox = detection.get_bbox()
confidence = detection.get_confidence()
#if label != None:
# Get track ID
track_id = 0
track = detection.get_objects_typed(hailo.HAILO_UNIQUE_ID)
if len(track) == 1:
track_id = track[0].get_id()
string_to_print += (f"Label: {label} ID: {track_id} Confidence: {confidence:.2f} Detection count: {detection_count}\n")
bbox = detection.get_bbox()
# Call the coordinate methods
x_min = bbox.xmin()
y_min = bbox.ymin()
box_width = bbox.width()
box_height = bbox.height()
# Calculate max coordinates
x_max = x_min + box_width
y_max = y_min + box_height
# Calculate center point (these are normalized 0-1)
center_x = x_min + (box_width / 2)
center_y = (y_min + (box_height / 2) - 0.22) * 1.83
# Debug print for coordinates
string_to_print += (f"Position: center=({center_x:.2f}, {center_y:.2f}) "
f"Bounds: xmin={x_min:.2f}, ymin={y_min:.2f}, xmax={x_max:.2f}, ymax={y_max:.2f}\n")
cv2.rectangle(frame, ((int)(x_min * width),(int) (y_min* height)), ((int)(x_max * width),(int) (y_max * height)), (255, 255, 255),1)
cv2.putText(frame, label,((int)(x_min * width),(int) ((y_min* height) - 10)), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 1)
if user_data.use_frame:
# Note: using imshow will not work here, as the callback function is not running in the main thread
# Let's print the detection count to the frame
cv2.putText(frame, f"Label: {label} ID: {track_id} Confidence: {confidence:.2f} Detection count: {detection_count}", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
# Example of how to use the new_variable and new_function from the user_data
# Let's print the new_variable and the result of the new_function to the frame
#cv2.putText(frame, f"{user_data.new_function()} {user_data.new_variable}", (10, 60), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
# Convert the frame to BGR
frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
user_data.set_frame(frame)
print(string_to_print)
sys.stdout.flush()
return Gst.PadProbeReturn.OK
if __name__ == "__main__":
# Create an instance of the user app callback class
user_data = user_app_callback_class()
app = GStreamerDetectionApp(app_callback, user_data)
app.run()
using the standard youlov11s and example.mp4 and my madsen.py the result is the same frame rate drops 50%
python3 basic_pipelines/madsen.py --hef-path resources/yolov11s.hef --input rpi --input example.mp4 --disable-sync --show-fps --use-frame
A error free version of madsen.py (My Danish Swedish hound dog got the family last name 
)
import gi
gi.require_version('Gst', '1.0')
from gi.repository import Gst, GLib
import os
import numpy as np
import cv2
import hailo
import sys
from hailo_apps_infra.hailo_rpi_common import (
get_caps_from_pad,
get_numpy_from_buffer,
app_callback_class,
)
from hailo_apps_infra.detection_pipeline import GStreamerDetectionApp
# -----------------------------------------------------------------------------------------------
# User-defined class to be used in the callback function
# -----------------------------------------------------------------------------------------------
# Inheritance from the app_callback_class
class user_app_callback_class(app_callback_class):
def __init__(self):
super().__init__()
self.new_variable = 42 # New variable example
def new_function(self): # New function example
return "The meaning of life is: "
# -----------------------------------------------------------------------------------------------
# User-defined callback function
# -----------------------------------------------------------------------------------------------
# This is the callback function that will be called when data is available from the pipeline
def app_callback(pad, info, user_data):
# Get the GstBuffer from the probe info
buffer = info.get_buffer()
# Check if the buffer is valid
if buffer is None:
return Gst.PadProbeReturn.OK
# Using the user_data to count the number of frames
user_data.increment()
#user_data.use_frame = True
string_to_print = f"Frame count: {user_data.get_count()}\n"
# Get the caps from the pad
format, width, height = get_caps_from_pad(pad)
# If the user_data.use_frame is set to True, we can get the video frame from the buffer
frame = None
label = None
if user_data.use_frame and format is not None and width is not None and height is not None:
# Get video frame
frame = get_numpy_from_buffer(buffer, format, width, height)
#string_to_print += " frame ok "
# Get the detections from the buffer
roi = hailo.get_roi_from_buffer(buffer)
detections = roi.get_objects_typed(hailo.HAILO_DETECTION)
# Parse the detections
detection_count = 0
for detection in detections:
detection_count += 1
label = detection.get_label()
if label != None:
bbox = detection.get_bbox()
confidence = detection.get_confidence()
# Get track ID
track_id = 0
track = detection.get_objects_typed(hailo.HAILO_UNIQUE_ID)
if len(track) == 1:
track_id = track[0].get_id()
string_to_print += (f"Label: {label} ID: {track_id} Confidence: {confidence:.2f} Detection count: {detection_count}\n")
bbox = detection.get_bbox()
# Call the coordinate methods
x_min = bbox.xmin()
y_min = bbox.ymin()
box_width = bbox.width()
box_height = bbox.height()
# Calculate max coordinates
x_max = x_min + box_width
y_max = y_min + box_height
# Calculate center point (these are normalized 0-1)
center_x = x_min + (box_width / 2)
center_y = (y_min + (box_height / 2) - 0.22) * 1.83
# Debug print for coordinates
string_to_print += (f"Position: center=({center_x:.2f}, {center_y:.2f}) "
f"Bounds: xmin={x_min:.2f}, ymin={y_min:.2f}, xmax={x_max:.2f}, ymax={y_max:.2f}\n")
if user_data.use_frame:
cv2.rectangle(frame, ((int)(x_min * width),(int) (y_min* height)), ((int)(x_max * width),(int) (y_max * height)), (255, 255, 255),1)
cv2.putText(frame, label,((int)(x_min * width),(int) ((y_min* height) - 10)), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 1)
if user_data.use_frame and label != None:
# Note: using imshow will not work here, as the callback function is not running in the main thread
# Let's print the detection count to the frame
cv2.putText(frame, f"Label: {label} ID: {track_id} Confidence: {confidence:.2f} Detection count: {detection_count}", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
# Example of how to use the new_variable and new_function from the user_data
# Let's print the new_variable and the result of the new_function to the frame
#cv2.putText(frame, f"{user_data.new_function()} {user_data.new_variable}", (10, 60), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
# Convert the frame to BGR
frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
user_data.set_frame(frame)
else :
user_data.set_frame(frame)
print(string_to_print)
sys.stdout.flush()
return Gst.PadProbeReturn.OK
if name == “main”:
# Create an instance of the user app callback class
user_data = user_app_callback_class()
app = GStreamerDetectionApp(app_callback, user_data)
app.run()
1 Like
Hey @Kim_Madsen,
This is an amazing project!
If you’d like, you can open it as a community project and push it to our RPI Examples Community Projects repo — we’d love to have it there!
Regarding the documentation:
We’re actively working on it as part of a big update across all our repos.
The goal is to expand the tools, improve the pipelines, add UI options, and enable direct buffer drawing — making it much easier to develop, visualize, and deploy projects with Hailo.
Yes but later i AM having a poject i have to solve now . 1 mdr from now i Got the time 
tirs. 29. apr. 2025 18.38 skrev omria via Hailo Community <notifications@hailo.discoursemail.com>: