Reading metadata from Hailonet - no detections

[Peter_Carpenter]

Hi,

I’m trying to set up a pipeline using hailonet in gstreamer and wanting to read the results. I can confirm the card / model / videos work fine as I can use the python sample object-detection to produce a video with objects highlighted as expected.

However, if I try and do the same thing using gstreamer & c++, I’m not able to get any metadata/tensors from the hailo_filter.cpp code (taken from tappas library).

Here’s what I am doing:

1. Setting up a gstreamer pipeline and inserting the hailo_filter as below:

	pipeline = gst_parse_launch("filesrc location=D:/test/video.mp4 ! \
								decodebin ! \
								videoscale qos=false n-threads=2 ! \
								videoconvert qos=false n-threads=2 ! \
								hailonet name=hailonet hef-path=D:/models/yolov8n.hef is-active=true batch-size=1 nms-score-threshold=0.3 nms-iou-threshold=0.45 output-format-type=3 ! \
								autovideosink name=autoSink sync=false", NULL);
	GstElement* hailonet = gst_bin_get_by_name(GST_BIN(pipeline), "hailonet");
	GstElement* hailoFilter = (GstElement*)g_object_new(GST_TYPE_HAILO_FILTER, NULL);
	GstElement* autoSink = gst_bin_get_by_name(GST_BIN(pipeline), "autoSink");
	gst_bin_add(GST_BIN(pipeline), hailoFilter);
	gst_element_unlink(hailonet, autoSink);
	gst_element_link(hailonet, hailoFilter);
	gst_element_link(hailoFilter, autoSink);
	gst_element_sync_state_with_parent(hailoFilter);

2. The pipeline runs and I can see debug information showing that the hailo_filter’s gst_hailofilter_transform_ip function is being called as expected. The gsthailofilter.cpp class and it’s dependencies have been copied straight from the tappas repository - with the only modification to remove the call to the .so file. (I’m running on Windows 11 and using VS - otherwise I’d supply the .so as designed.)

3. Everything compiles & runs ok - however there are no objects detected. The same video with the same .hef file detects heaps using python.

4. If I add debug to the “get_tensors_from_meta(buffer, hailo_roi)” function - then I can see that “gst_buffer_get_meta(pmeta->buffer, g_type_from_name(TENSOR_META_API_NAME)” is returning false.
   It does contain data though - the tensor name is “yolov8n/yolov8_nms_postprocess”.

5. If I add debug to the “get_hailo_main_roi(buffer, true)” function - then I can see that gst_buffer_get_hailo_meta(buffer) returns NULL.

6. Hailort.log is below:

[2025-10-22 18:09:34.955] [9828] [HailoRT] [info] [vdevice.cpp:536] [hailort::VDevice::create] Creating vdevice with params: device_count: 1, scheduling_algorithm: ROUND_ROBIN, multi_process_service: false
[2025-10-22 18:09:34.957] [9828] [HailoRT] [info] [control.cpp:117] [hailort::control__parse_identify_results] firmware_version is: 4.22.0
[2025-10-22 18:09:34.957] [9828] [HailoRT] [info] [vdevice.cpp:682] [hailort::VDeviceBase::create] VDevice Infos: 0000:02:00.0
[2025-10-22 18:09:34.966] [9828] [HailoRT] [info] [hef.cpp:1995] [hailort::Hef::Impl::get_network_group_and_network_name] No name was given. Addressing all networks of default network_group: yolov8n
[2025-10-22 18:09:34.966] [9828] [HailoRT] [info] [hef.cpp:1995] [hailort::Hef::Impl::get_network_group_and_network_name] No name was given. Addressing all networks of default network_group: yolov8n
[2025-10-22 18:09:35.154] [4528] [HailoRT] [warning] [device_internal.cpp:800] [hailort::DeviceBase::check_clock_rate_for_hailo8] HEF was compiled assuming clock rate of 400 MHz, while the device clock rate is 200 MHz. FPS calculations might not be accurate.
[2025-10-22 18:09:35.156] [4528] [HailoRT] [warning] [device_internal.cpp:800] [hailort::DeviceBase::check_clock_rate_for_hailo8] HEF was compiled assuming clock rate of 400 MHz, while the device clock rate is 200 MHz. FPS calculations might not be accurate.
[2025-10-22 18:09:35.175] [4528] [HailoRT] [info] [internal_buffer_manager.cpp:75] [hailort::InternalBufferManager::print_execution_results] Default Internal buffer planner failed to meet requirements
[2025-10-22 18:09:35.175] [4528] [HailoRT] [info] [internal_buffer_manager.cpp:86] [hailort::InternalBufferManager::print_execution_results] Default Internal buffer planner executed successfully
[2025-10-22 18:09:35.192] [4528] [HailoRT] [info] [device_internal.cpp:57] [hailort::DeviceBase::configure] Configuring HEF took 36.3505 milliseconds
[2025-10-22 18:09:35.192] [4528] [HailoRT] [info] [vdevice.cpp:780] [hailort::VDeviceBase::configure] Configuring HEF on VDevice took 37.7274 milliseconds
[2025-10-22 18:09:35.192] [4528] [HailoRT] [info] [infer_model.cpp:419] [hailort::InferModelBase::configure] Configuring network group ‘yolov8n’ with params: batch size: 1, power mode: PERFORMANCE, latency: NONE
[2025-10-22 18:09:35.192] [4528] [HailoRT] [info] [multi_io_elements.cpp:754] [hailort::AsyncHwElement::create] Created (AsyncHwEl)
[2025-10-22 18:09:35.193] [4528] [HailoRT] [info] [queue_elements.cpp:450] [hailort::AsyncPushQueueElement::create] Created (EntryPushQEl0yolov8n/input_layer1 | timeout: 10s)
[2025-10-22 18:09:35.193] [4528] [HailoRT] [info] [filter_elements.cpp:101] [hailort::PreInferElement::create] Created (PreInferEl3yolov8n/input_layer1 | Reorder - src_order: RGB4, src_shape: (640, 640, 3), dst_order: NHCW, dst_shape: (640, 640, 3))
[2025-10-22 18:09:35.193] [4528] [HailoRT] [info] [queue_elements.cpp:450] [hailort::AsyncPushQueueElement::create] Created (PushQEl3yolov8n/input_layer1 | timeout: 10s)
[2025-10-22 18:09:35.193] [4528] [HailoRT] [info] [multi_io_elements.cpp:135] [hailort::NmsPostProcessMuxElement::create] Created (NmsPPMuxEl0YOLOV8-Post-Process | Op YOLOV8, Name: YOLOV8-Post-Process, Score threshold: 0.300, IoU threshold: 0.45, Classes: 80, Max bboxes per class: 100, Image height: 640, Image width: 640)
[2025-10-22 18:09:35.194] [4528] [HailoRT] [info] [queue_elements.cpp:942] [hailort::MultiPushQueue::create] Created (MultiPushQEl0YOLOV8-Post-Process | timeout: 10s)
[2025-10-22 18:09:35.194] [4528] [HailoRT] [info] [edge_elements.cpp:187] [hailort::LastAsyncElement::create] Created (LastAsyncEl0NmsPPMuxEl0YOLOV8-Post-Process)
[2025-10-22 18:09:35.194] [4528] [HailoRT] [info] [pipeline.cpp:891] [hailort::PipelineElement::print_deep_description] EntryPushQEl0yolov8n/input_layer1 | inputs: user | outputs: PreInferEl3yolov8n/input_layer1(running in thread_id: 5972)
[2025-10-22 18:09:35.194] [4528] [HailoRT] [info] [pipeline.cpp:891] [hailort::PipelineElement::print_deep_description] PreInferEl3yolov8n/input_layer1 | inputs: EntryPushQEl0yolov8n/input_layer1[0] | outputs: PushQEl3yolov8n/input_layer1
[2025-10-22 18:09:35.194] [4528] [HailoRT] [info] [pipeline.cpp:891] [hailort::PipelineElement::print_deep_description] PushQEl3yolov8n/input_layer1 | inputs: PreInferEl3yolov8n/input_layer1[0] | outputs: AsyncHwEl(running in thread_id: 7192)
[2025-10-22 18:09:35.194] [4528] [HailoRT] [info] [pipeline.cpp:891] [hailort::PipelineElement::print_deep_description] AsyncHwEl | inputs: PushQEl3yolov8n/input_layer1[0] | outputs: MultiPushQEl0YOLOV8-Post-Process MultiPushQEl0YOLOV8-Post-Process MultiPushQEl0YOLOV8-Post-Process MultiPushQEl0YOLOV8-Post-Process MultiPushQEl0YOLOV8-Post-Process MultiPushQEl0YOLOV8-Post-Process
[2025-10-22 18:09:35.194] [4528] [HailoRT] [info] [pipeline.cpp:891] [hailort::PipelineElement::print_deep_description] MultiPushQEl0YOLOV8-Post-Process | inputs: AsyncHwEl[0] AsyncHwEl[1] AsyncHwEl[2] AsyncHwEl[3] AsyncHwEl[4] AsyncHwEl[5] | outputs: NmsPPMuxEl0YOLOV8-Post-Process(running in thread_id: 5960)
[2025-10-22 18:09:35.194] [4528] [HailoRT] [info] [pipeline.cpp:891] [hailort::PipelineElement::print_deep_description] NmsPPMuxEl0YOLOV8-Post-Process | inputs: MultiPushQEl0YOLOV8-Post-Process[0] | outputs: LastAsyncEl0NmsPPMuxEl0YOLOV8-Post-Process
[2025-10-22 18:09:35.195] [4528] [HailoRT] [info] [pipeline.cpp:891] [hailort::PipelineElement::print_deep_description] LastAsyncEl0NmsPPMuxEl0YOLOV8-Post-Process | inputs: NmsPPMuxEl0YOLOV8-Post-Process[0] | outputs: user
[2025-10-22 18:09:35.205] [4528] [HailoRT] [info] [hef.cpp:1995] [hailort::Hef::Impl::get_network_group_and_network_name] No name was given. Addressing all networks of default network_group: yolov8n

So my questions are: Why is the hailonet element not providing the detections in the metadata? How can I debug this further?

Thanks!!

[Peter_Carpenter]

Actually I’ll make this question a bit broader. Is there any sample code anywhere that demonstrates that the gstreamer hailonet component actually works under Windows? Ie: Takes a video or a live source in and provides detections out - dumping to console or a file is fine…

I’d like to think that I just don’t have the video source correct - however setting the capability as “video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1,format=RGB” didn’t seem to make any difference. I notice that the hailort.log says it’s reordering from RGB4 to NHCW. Is this what is supposed to be happening?

[Peter_Carpenter]

Another interesting observation: If I dump the stream to a file before & after the hailonet element, the data is the same (with the exception that the before looks like it contains an extra frame at the very end).

gst-launch-1.0 filesrc location=“D:/test/video.mp4” ! decodebin ! videoscale qos=false n-threads=2 ! videoconvert n-threads=2 qos=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1,format=RGB ! hailonet hef-path=D:/models/yolov8n.hef batch-size=1 nms-score-threshold=0.3 nms-iou-threshold=0.45 ! filesink location=after.txt

vs

gst-launch-1.0 filesrc location=“D:/test/video.mp4” ! decodebin ! videoscale qos=false n-threads=2 ! videoconvert n-threads=2 qos=false ! video/x-raw,width=640,height=640,pixel-aspect-ratio=1/1,format=RGB ! filesink location=before.txt

So it does appear that the hailonet element is not appending any metadata to the stream at all. What am I missing?

Release notes for HailoRT v4.22.0 state:

Added a new struct hailo_tensor_metadata_t to be used with GStreamer output tensor instead of hailo_vstream_info_t.

Perhaps this has broken the gstreamer hailonet plugin?

(Just tested v4.23.0 with it’s newer gsthailo.lib & dll, and still no tensors).

** Since determined that metadata is not written to disk via filesink - so this observation is irrelevant.

[Peter_Carpenter]

I’m about ready to abandon all hope of using this product with Windows/C++/GStreamer. Although advertised, it’s clearly not a path well travelled.

My recent efforts have been to compile the hailonet gstreamer plugin using the instructions found here: https://hailo.ai/developer-zone/documentation/hailort-v5-1-0/?sp_referrer=install/install.html#compiling-from-sources.

Whilst it compiles without errors, if I copy the resultant dll/lib files into the gstreamer lib directory, they don’t work. Strangely - if I run “gst-inspect-1.0 hailonet” it crashes out halfway through printing the element properties.

I’m using 2019 VS Build tools as specified. Do I have to use a specific older version of GStreamer? (I’m using v1.26.5)

[Peter_Carpenter]

So I now understand why I wasn’t seeing any detections in the data extracted by hailo filter class. My understanding was that this class would provide access to each of the individual detection bounding boxes and classifications - and it does - but you need to put them there first.

The filter class creates the roi object and populates it with the raw tensor. It then calls your .so code to do the post processing. Whilst the roi object contains a list of detections, this list isn’t populated by the filter - the intent of the filter is to allow you to write the custom code to populate/convert (and also filter) them yourself.

The tappas guides show examples of iterating over this list to get the detections - and it also shows ways to add the detections (using hardcoded values) - however it would have been very useful to me if it highlighted this greater purpose. Perhaps it could be more appropriately named…

I hope this helps any future readers who may attempt to go down a similar path.

[omria]

Hey @Peter_Carpenter,

Great work compiling Tappas on Windows! Since we don’t officially support Windows, this is impressive. The community would love to hear about your process!

Regarding the errors you encountered:

1. Windows Support Limitation: We don’t officially support Tappas and GStreamer on Windows, which is why Windows-specific examples aren’t available. While it’s possible to compile from source (as you did) using GitHub - hailo-ai/hailo-apps-infra, we recommend using the direct API for Windows instead.

2. The Core Issue: When using hailonet with output-format-type=3 (HAILO_FORMAT_TYPE_AUTO), the NMS post-processing happens internally, outputting processed detections rather than raw tensors in metadata. This means hailonet doesn’t attach tensor metadata in the format that hailofilter expects to consume.

   Note: HailoRT v4.22.0 introduced a new hailo_tensor_metadata_t struct for GStreamer output tensors, but this doesn’t apply when using the NMS post-processing format built into your yolov8n.hef.

Note: I’ll forward your request to create more guides on working with and rewriting pipelines for new HEF files.

Hope this helps! if you run into anything please let me know!