output frame does not match dimensions

laurens.eiroasattler · October 13, 2024, 5:35am

Hi, i am trying to develop my own script for CV suing my hailo 8l. I already did several tutorials but i don’t want to copy paste all this code in to my own applications. Therefore, I am trying to make a script as simple as possible (where the highest performance is not necessary) and afterwards start trying more advance methodologies.
Following the hailort user guide for python inference (5.5. Python Inference Tutorial - Single Model). Which for me looks the easiest since you don’t need to control the scheduler.
My sctript looks as follows:

import cv2
import numpy as np
from PIL import Image
from multiprocessing import Process
from hailo_platform import (HEF, VDevice, HailoStreamInterface, InferVStreams,ConfigureParams, HailoSchedulingAlgorithm,
                            InputVStreamParams, OutputVStreamParams, InputVStreams, OutputVStreams,
                            FormatType)
# The target can be used as a context manager (”with” statement) to ensure it's released on time.
# Here it's avoided for the sake of simplicity
def preprocess_frame(frame,image_height, image_width, channels):
    print(f"chanels: {channels}\nframe_shape: {frame.shape}")
    input_data = cv2.resize(frame, (image_height, image_width))
    input_data = input_data.astype(np.float32) / 255.0
    #input_data = np.transpose(input_data, (2, 0, 1))  # Change to CHW format
    input_data = np.expand_dims(input_data, axis=0)  # Add batch dimension
    return input_data

target = VDevice()

# Loading compiled HEFs to device:
model_name = 'yolov6n'
hef_path = '{}.hef'.format(model_name)
hef = HEF(hef_path)

# Configure network groups
configure_params = ConfigureParams.create_from_hef(hef=hef,interface=HailoStreamInterface.PCIe)
network_groups = target.configure(hef, configure_params)
network_group = network_groups[0]
network_group_params = network_group.create_params()

# Create input and output virtual streams params
input_vstreams_params = InputVStreamParams.make(network_group, format_type=FormatType.FLOAT32)
output_vstreams_params = OutputVStreamParams.make(network_group, format_type=FormatType.FLOAT32)

# Define dataset params
input_vstream_info = hef.get_input_vstream_infos()[0]
output_vstream_info = hef.get_output_vstream_infos()[0]
image_height, image_width, channels = input_vstream_info.shape


# Generate random dataset
image = cv2.imread("Input/im0.jpg")
input_frame = preprocess_frame(image,image_height, image_width, channels)

print(f"input stream shape: {input_vstream_info.shape}, frame shape{input_frame.shape}\n desired output shape {output_vstream_info.shape}")
#Infer
with InferVStreams(network_group, input_vstreams_params, output_vstreams_params) as infer_pipeline:
    input_data = {input_vstream_info.name: input_frame}
    with network_group.activate(network_group_params):
        infer_results = infer_pipeline.infer(input_data)
        print('Stream output is {}'.format(infer_results[output_vstream_info.name]))

Running this, I obtain the following result:

(custom_camera_cv_env) rpi@raspberrypi:~/Scripts/hailo_process_frame $ python3 test2.py 
chanels: 3
frame_shape: (480, 640, 3)
input stream shape: (640, 640, 3), frame shape(1, 640, 640, 3)
 desired output shape (80, 5, 100)
Stream output is [[array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5), dtype=float64)]]

Which obiously does not make much sense.

Inspecting the hailort.log i find:

[2024-10-12 12:44:20.306] [4278] [HailoRT] [info] [device.cpp:46] [Device] OS Version: Linux 6.6.51+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.51-1+rpt3 (2024-10-08) aarch64
[2024-10-12 12:44:20.306] [4278] [HailoRT] [info] [control.cpp:100] [control__parse_identify_results] firmware_version is: 4.18.0
[2024-10-12 12:44:20.306] [4278] [HailoRT] [info] [vdevice.cpp:652] [create] VDevice Infos: 0000:04:00.0
[2024-10-12 12:44:20.388] [4278] [HailoRT] [info] [internal_buffer_manager.cpp:202] [print_execution_results] Default Internal buffer planner failed to meet requirements
[2024-10-12 12:44:20.388] [4278] [HailoRT] [info] [internal_buffer_manager.cpp:212] [print_execution_results] Default Internal buffer planner executed successfully
[2024-10-12 12:44:20.399] [4278] [HailoRT] [info] [device_internal.cpp:57] [configure] Configuring HEF took 15.794048 milliseconds
[2024-10-12 12:44:20.399] [4278] [HailoRT] [info] [vdevice.cpp:751] [configure] Configuring HEF on VDevice took 16.244344 milliseconds
[2024-10-12 12:44:20.399] [4278] [HailoRT] [info] [hef.cpp:1844] [get_network_group_and_network_name] No name was given. Addressing all networks of default network_group: yolov6n
[2024-10-12 12:44:20.399] [4278] [HailoRT] [info] [hef.cpp:1844] [get_network_group_and_network_name] No name was given. Addressing all networks of default network_group: yolov6n
[2024-10-12 12:44:20.415] [4278] [HailoRT] [info] [edge_elements.cpp:52] [create] Created (HwWriteEl3yolov6n/input_layer1 | hw_frame_size: 1228800)
[2024-10-12 12:44:20.415] [4278] [HailoRT] [info] [queue_elements.cpp:255] [create] Created (PushQEl3yolov6n/input_layer1 | timeout: 10s)
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [filter_elements.cpp:101] [create] Created (PreInferEl3yolov6n/input_layer1 | Quantization - src_type: FLOAT32, dst_type UINT8, limvals_min: 0, limvals_max: 255 | Reorder - src_order: NHWC, src_shape: (640, 640, 3), dst_order: NHCW, dst_shape: (640, 640, 3))
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [vstream.cpp:754] [InputVStreamImpl] Creating yolov6n/input_layer1...
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [vstream_builder.cpp:102] [create_inputs] Input pipeline 'yolov6n/input_layer1': (PreInferEl3yolov6n/input_layer1 | Quantization - src_type: FLOAT32, dst_type UINT8, limvals_min: 0, limvals_max: 255 | Reorder - src_order: NHWC, src_shape: (640, 640, 3), dst_order: NHCW, dst_shape: (640, 640, 3)) >> (PushQEl3yolov6n/input_layer1 | timeout: 10s) >> (HwWriteEl3yolov6n/input_layer1 | hw_frame_size: 1228800) >> HW
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [multi_io_elements.cpp:159] [create] Created (NmsPPMuxEl0YOLOX-Post-Process | Op YOLOX, Name: YOLOX-Post-Process, Score threshold: 0.200, IoU threshold: 0.65, Classes: 80, Cross classes: false, Max bboxes per class: 100, Image height: 640, Image width: 640)
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [edge_elements.cpp:287] [create] Created (HwReadEl2yolov6n/conv36_70 | hw_frame_size: 512000)
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms2yolov6n/conv36_70 | timeout: 10s)
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [edge_elements.cpp:287] [create] Created (HwReadEl1yolov6n/conv37_70 | hw_frame_size: 25600)
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms1yolov6n/conv37_70 | timeout: 10s)
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [edge_elements.cpp:287] [create] Created (HwReadEl0yolov6n/conv38_70 | hw_frame_size: 6400)
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms0yolov6n/conv38_70 | timeout: 10s)
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [edge_elements.cpp:287] [create] Created (HwReadEl9yolov6n/conv47_70 | hw_frame_size: 128000)
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms9yolov6n/conv47_70 | timeout: 10s)
[2024-10-12 12:44:20.418] [4278] [HailoRT] [info] [edge_elements.cpp:287] [create] Created (HwReadEl6yolov6n/conv48_70 | hw_frame_size: 6400)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms6yolov6n/conv48_70 | timeout: 10s)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [edge_elements.cpp:287] [create] Created (HwReadEl7yolov6n/conv49_70 | hw_frame_size: 1600)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms7yolov6n/conv49_70 | timeout: 10s)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [edge_elements.cpp:287] [create] Created (HwReadEl4yolov6n/conv57_70 | hw_frame_size: 38400)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms4yolov6n/conv57_70 | timeout: 10s)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [edge_elements.cpp:287] [create] Created (HwReadEl8yolov6n/conv58_70 | hw_frame_size: 1920)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms8yolov6n/conv58_70 | timeout: 10s)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [edge_elements.cpp:287] [create] Created (HwReadEl5yolov6n/conv59_70 | hw_frame_size: 480)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms5yolov6n/conv59_70 | timeout: 10s)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [queue_elements.cpp:756] [create] Created (UserBufQEl_post_infer0yolov6n/yolox_nms_postprocess | timeout: 10s)
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [vstream.cpp:1107] [OutputVStreamImpl] Creating yolov6n/yolox_nms_postprocess...
[2024-10-12 12:44:20.419] [4278] [HailoRT] [info] [vstream_builder.cpp:627] [create_output_post_process_nms] Output pipeline 'yolov6n/yolox_nms_postprocess': HW >> (HwReadEl2yolov6n/conv36_70 | hw_frame_size: 512000) >> (PullQEl_nms2yolov6n/conv36_70) >> (HwReadEl1yolov6n/conv37_70 | hw_frame_size: 25600) >> (PullQEl_nms1yolov6n/conv37_70) >> (HwReadEl0yolov6n/conv38_70 | hw_frame_size: 6400) >> (PullQEl_nms0yolov6n/conv38_70) >> (HwReadEl9yolov6n/conv47_70 | hw_frame_size: 128000) >> (PullQEl_nms9yolov6n/conv47_70) >> (HwReadEl6yolov6n/conv48_70 | hw_frame_size: 6400) >> (PullQEl_nms6yolov6n/conv48_70) >> (HwReadEl7yolov6n/conv49_70 | hw_frame_size: 1600) >> (PullQEl_nms7yolov6n/conv49_70) >> (HwReadEl4yolov6n/conv57_70 | hw_frame_size: 38400) >> (PullQEl_nms4yolov6n/conv57_70) >> (HwReadEl8yolov6n/conv58_70 | hw_frame_size: 1920) >> (PullQEl_nms8yolov6n/conv58_70) >> (HwReadEl5yolov6n/conv59_70 | hw_frame_size: 480) >> (PullQEl_nms5yolov6n/conv59_70) >> (NmsPPMuxEl0YOLOX-Post-Process | Op YOLOX, Name: YOLOX-Post-Process, Score threshold: 0.200, IoU threshold: 0.65, Classes: 80, Cross classes: false, Max bboxes per class: 100, Image height: 640, Image width: 640) >> (UserBufQEl_post_infer0yolov6n/yolox_nms_postprocess | timeout: 10s)
[2024-10-12 12:44:20.420] [4278] [HailoRT] [info] [core_op.cpp:156] [activate] Activating yolov6n took 0.729222 milliseconds. Note that the function is asynchronous and thus the network is not fully activated yet.
[2024-10-12 12:44:20.420] [4278] [HailoRT] [info] [core_op.cpp:156] [activate] Activating yolov6n took 0.740055 milliseconds. Note that the function is asynchronous and thus the network is not fully activated yet.
[2024-10-12 12:44:20.437] [4304] [HailoRT] [info] [vdevice_native_stream.cpp:331] [read_impl] Failed read from stream HAILO_STREAM_NOT_ACTIVATED(72) (device: 0000:04:00.0)
[2024-10-12 12:44:20.437] [4305] [HailoRT] [info] [vdevice_native_stream.cpp:331] [read_impl] Failed read from stream HAILO_STREAM_NOT_ACTIVATED(72) (device: 0000:04:00.0)
[2024-10-12 12:44:20.437] [4309] [HailoRT] [info] [vdevice_native_stream.cpp:331] [read_impl] Failed read from stream HAILO_STREAM_NOT_ACTIVATED(72) (device: 0000:04:00.0)
[2024-10-12 12:44:20.437] [4302] [HailoRT] [info] [vdevice_native_stream.cpp:331] [read_impl] Failed read from stream HAILO_STREAM_NOT_ACTIVATED(72) (device: 0000:04:00.0)
[2024-10-12 12:44:20.437] [4301] [HailoRT] [info] [vdevice_native_stream.cpp:331] [read_impl] Failed read from stream HAILO_STREAM_NOT_ACTIVATED(72) (device: 0000:04:00.0)
[2024-10-12 12:44:20.437] [4303] [HailoRT] [info] [vdevice_native_stream.cpp:331] [read_impl] Failed read from stream HAILO_STREAM_NOT_ACTIVATED(72) (device: 0000:04:00.0)
[2024-10-12 12:44:20.437] [4307] [HailoRT] [info] [vdevice_native_stream.cpp:331] [read_impl] Failed read from stream HAILO_STREAM_NOT_ACTIVATED(72) (device: 0000:04:00.0)
[2024-10-12 12:44:20.437] [4306] [HailoRT] [info] [vdevice_native_stream.cpp:331] [read_impl] Failed read from stream HAILO_STREAM_NOT_ACTIVATED(72) (device: 0000:04:00.0)
[2024-10-12 12:44:20.437] [4308] [HailoRT] [info] [vdevice_native_stream.cpp:331] [read_impl] Failed read from stream HAILO_STREAM_NOT_ACTIVATED(72) (device: 0000:04:00.0)
[2024-10-12 12:44:20.437] [4278] [HailoRT] [info] [core_op.cpp:186] [deactivate] Deactivating took 0.567851 ms
[2024-10-12 12:44:20.437] [4278] [HailoRT] [info] [core_op.cpp:186] [deactivate] Deactivating took 0.582555 ms
[2024-10-12 12:44:20.437] [4302] [HailoRT] [info] [queue_elements.cpp:690] [run_in_thread] Shutdown event was signaled in run_pull of queue element PullQEl_nms1yolov6n/conv37_70!
[2024-10-12 12:44:20.437] [4305] [HailoRT] [info] [queue_elements.cpp:690] [run_in_thread] Shutdown event was signaled in run_pull of queue element PullQEl_nms6yolov6n/conv48_70!
[2024-10-12 12:44:20.438] [4307] [HailoRT] [info] [queue_elements.cpp:690] [run_in_thread] Shutdown event was signaled in run_pull of queue element PullQEl_nms4yolov6n/conv57_70!
[2024-10-12 12:44:20.438] [4301] [HailoRT] [info] [queue_elements.cpp:690] [run_in_thread] Shutdown event was signaled in run_pull of queue element PullQEl_nms2yolov6n/conv36_70!
[2024-10-12 12:44:20.438] [4306] [HailoRT] [info] [queue_elements.cpp:690] [run_in_thread] Shutdown event was signaled in run_pull of queue element PullQEl_nms7yolov6n/conv49_70!
[2024-10-12 12:44:20.438] [4303] [HailoRT] [info] [queue_elements.cpp:690] [run_in_thread] Shutdown event was signaled in run_pull of queue element PullQEl_nms0yolov6n/conv38_70!
[2024-10-12 12:44:20.438] [4304] [HailoRT] [info] [queue_elements.cpp:690] [run_in_thread] Shutdown event was signaled in run_pull of queue element PullQEl_nms9yolov6n/conv47_70!
[2024-10-12 12:44:20.438] [4308] [HailoRT] [info] [queue_elements.cpp:690] [run_in_thread] Shutdown event was signaled in run_pull of queue element PullQEl_nms8yolov6n/conv58_70!
[2024-10-12 12:44:20.438] [4309] [HailoRT] [info] [queue_elements.cpp:690] [run_in_thread] Shutdown event was signaled in run_pull of queue element PullQEl_nms5yolov6n/conv59_70!
[2024-10-12 12:44:20.438] [4310] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element UserBufQEl_post_infer0yolov6n/yolox_nms_postprocess is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element UserBufQEl_post_infer0yolov6n/yolox_nms_postprocess has 0 frames in his Queue on destruction
[2024-10-12 12:44:20.438] [4309] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element PullQEl_nms5yolov6n/conv59_70 is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element PullQEl_nms5yolov6n/conv59_70 has 0 frames in his Queue on destruction
[2024-10-12 12:44:20.438] [4308] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element PullQEl_nms8yolov6n/conv58_70 is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element PullQEl_nms8yolov6n/conv58_70 has 0 frames in his Queue on destruction
[2024-10-12 12:44:20.438] [4307] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element PullQEl_nms4yolov6n/conv57_70 is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element PullQEl_nms4yolov6n/conv57_70 has 0 frames in his Queue on destruction
[2024-10-12 12:44:20.438] [4306] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element PullQEl_nms7yolov6n/conv49_70 is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element PullQEl_nms7yolov6n/conv49_70 has 0 frames in his Queue on destruction
[2024-10-12 12:44:20.438] [4305] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element PullQEl_nms6yolov6n/conv48_70 is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element PullQEl_nms6yolov6n/conv48_70 has 0 frames in his Queue on destruction
[2024-10-12 12:44:20.438] [4304] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element PullQEl_nms9yolov6n/conv47_70 is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element PullQEl_nms9yolov6n/conv47_70 has 0 frames in his Queue on destruction
[2024-10-12 12:44:20.438] [4303] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element PullQEl_nms0yolov6n/conv38_70 is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element PullQEl_nms0yolov6n/conv38_70 has 0 frames in his Queue on destruction
[2024-10-12 12:44:20.438] [4302] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element PullQEl_nms1yolov6n/conv37_70 is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element PullQEl_nms1yolov6n/conv37_70 has 0 frames in his Queue on destruction
[2024-10-12 12:44:20.438] [4301] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element PullQEl_nms2yolov6n/conv36_70 is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element PullQEl_nms2yolov6n/conv36_70 has 0 frames in his Queue on destruction
[2024-10-12 12:44:20.438] [4300] [HailoRT] [info] [queue_elements.cpp:54] [operator()] Thread in element PushQEl3yolov6n/input_layer1 is not running anymore, exiting..
[2024-10-12 12:44:20.438] [4278] [HailoRT] [info] [queue_elements.cpp:43] [~BaseQueueElement] Queue element PushQEl3yolov6n/input_layer1 has 0 frames in his Queue on destruction

Could someone help me address the issue. Not sure what is going on.

Regards,
Laurens

omria · October 13, 2024, 2:09pm

Hey @laurens.eiroasattler

Welcome to the Hailo Community !

It looks like you’re facing an issue where the inference output doesn’t make sense, and the logs indicate that the stream isn’t fully activated.

Based on your code and the output logs, here are a few potential causes and steps you can take to address this:

1. Check the Preprocessing

The input data seems to be resized and normalized correctly, but the channel order might not align with what your model expects.
Hailo models typically expect inputs in NHWC format (batch, height, width, channels). However, some models might use NCHW format (batch, channels, height, width). Your current code uses NHWC, but it’s worth verifying by checking the model’s HEF metadata or documentation.
If your model expects NCHW format, you can uncomment the line to transpose the input data:
```
input_data = np.transpose(input_data, (0, 3, 1, 2))  # From NHWC to NCHW format
```

2. Post-Processing or Output Format

The output format may not match your expectations. From the log:

Stream output is [[array([], shape=(0, 5), dtype=float64), ... ]]

This indicates the output is either empty or not processed correctly. YOLO models typically output bounding boxes, confidences, and class predictions. You might need to apply a post-processing step (like Non-Maximum Suppression) to extract valid predictions.

Check your model’s documentation to make sure the output format is handled correctly. Many YOLO models output multi-scale predictions, so your post-processing should account for this.

3. Failed Activation (HAILO_STREAM_NOT_ACTIVATED)

The log shows that the network isn’t fully activated:

Failed read from stream HAILO_STREAM_NOT_ACTIVATED

Ensure that the network is fully activated before running inference. You can add a short delay after activating the network group to allow it enough time to initialize before starting inference.
Also, double-check the network_group.activate() block to make sure everything is synchronized properly.

4. Buffer Management Issues

There is also a mention of the buffer planner failing, although it proceeds:

Default Internal buffer planner failed to meet requirements

This could indicate an issue with memory allocation or the input/output buffer sizes. You may want to specify the buffer size more explicitly or double-check the configuration.

5. Check HEF Metadata for Input/Output Dimensions

You can use the following command to inspect your model’s input/output specifications:

hailo_hef_viewer your_model.hef

This will help you confirm the input/output shapes and formats expected by your model.

Suggested Debugging Steps:

Verify the model’s input format (NHWC vs. NCHW) and adjust preprocessing if needed.
Ensure proper post-processing: For YOLO models, make sure you apply Non-Maximum Suppression (NMS) to the output.
Ensure network activation: Add a slight delay after activation to ensure the network is fully ready before inference.
Check HEF metadata to confirm input/output buffer sizes and formats.
Review buffer management: Make sure the buffer sizes and memory allocations match what the model expects.

Let me know if any of these steps help or if you need more clarification on any of them!

Best regards,
Omri

Topic		Replies	Views
Yolov8 output boxes not making any sense General raspberry-pi , yolov8	4	68	November 6, 2024
[HailoRT] [error] CHECK failed - Input buffer size 0 is different than expected 602112 for input 'yolov8m/input_layer1' General hailort , raspberry-pi , hailo8	6	181	October 9, 2024
Memory size frame count mismatch General raspberry-pi	9	296	August 26, 2024
Facing error in trensor matching while using hailo libraries. in our custom code General pcie , hailort	4	195	August 28, 2024
Looking for a solution to yolov8n/yolov8_nms_postprocess related errors General dfc , hailort , hailo8 , error , yolov8	0	14	November 17, 2024