Hey @Ole_Jakob_Opdal-Sell,
Welcome to the Hailo Community!
Thanks for the feedback!
Just wanted to address your points:
-
Python support is actually available through the hailo-all installer. If you run pip list | grep hailo on your RPi, you’ll see hailort is installed. For examples on how to use it directly, check out our repo here: Hailo-Application-Code-Examples/runtime/python at main · hailo-ai/Hailo-Application-Code-Examples · GitHub
-
Like I mentioned, hailort can be accessed directly with Python!
-
GStreamer can extract all the data you need. Here’s the callback for detection - it shows how to extract everything:
# This is the callback function that will be called when data is available from the pipeline
def app_callback(pad, info, user_data):
# Get the GstBuffer from the probe info
buffer = info.get_buffer()
# Check if the buffer is valid
if buffer is None:
hailo_logger.warning("Received None buffer | frame=%s", user_data.get_count())
return Gst.PadProbeReturn.OK
# Using the user_data to count the number of frames
user_data.increment()
frame_idx = user_data.get_count()
string_to_print = f"Frame count: {user_data.get_count()}\n"
# Get the caps from the pad
format, width, height = get_caps_from_pad(pad)
hailo_logger.debug("Frame=%s | caps fmt=%s %sx%s", frame_idx, format, width, height)
# If the user_data.use_frame is set to True, we can get the video frame from the buffer
frame = None
if user_data.use_frame and format is not None and width is not None and height is not None:
# Get video frame
frame = get_numpy_from_buffer(buffer, format, width, height)
# Get the detections from the buffer
roi = hailo.get_roi_from_buffer(buffer)
detections = roi.get_objects_typed(hailo.HAILO_DETECTION)
# Parse the detections
detection_count = 0
for detection in detections:
label = detection.get_label()
bbox = detection.get_bbox()
confidence = detection.get_confidence()
if label == "person":
# Get track ID
track_id = 0
track = detection.get_objects_typed(hailo.HAILO_UNIQUE_ID)
if len(track) == 1:
track_id = track[0].get_id()
string_to_print += (
f"Detection: ID: {track_id} Label: {label} Confidence: {confidence:.2f}\n"
)
hailo_logger.debug(
"Frame=%s | Detection person | id=%s conf=%.2f bbox=(x=%.1f,y=%.1f,w=%.1f,h=%.1f)",
frame_idx,
track_id,
confidence,
bbox.xmin(),
bbox.ymin(),
bbox.width(),
bbox.height(),
)
detection_count += 1
if user_data.use_frame:
# Note: using imshow will not work here, as the callback function is not running in the main thread
# Let's print the detection count to the frame
cv2.putText(
frame,
f"Detections: {detection_count}",
(10, 30),
cv2.FONT_HERSHEY_SIMPLEX,
1,
(0, 255, 0),
2,
)
# Example of how to use the new_variable and new_function from the user_data
# Let's print the new_variable and the result of the new_function to the frame
cv2.putText(
frame,
f"{user_data.new_function()} {user_data.new_variable}",
(10, 60),
cv2.FONT_HERSHEY_SIMPLEX,
1,
(0, 255, 0),
2,
)
# Convert the frame to BGR
frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
user_data.set_frame(frame)
print(string_to_print)
hailo_logger.info(string_to_print.strip())
return Gst.PadProbeReturn.OK
-
All the resources you’re looking for are in the config. If you run into any specific issues or bugs, just let me know and I’ll be happy to help fix them!
-
Regarding the apps - I’ve shared some already, but we’re also working on integrating everything into a single hailo-apps package for easier access.
Hope this helps you get started and build what you’re working on! Don’t hesitate to reach out if you need anything else.