Hailo_timeout(4)

ShingHin.Hung · June 10, 2024, 10:41am

I am trying to run inference on Hailo 8 hardware with custom trained yolov8n model, code as shown:

model_name = 'yolov8n'
compiled_model_har_path = f'{model_name}_compiled_model.har'
runner = ClientRunner(hw_arch='hailo8', har=compiled_model_har_path)

with runner.infer_context(InferenceContext.SDK_HAILO_HW) as ctx:
    nms_output = runner.infer(ctx, test_dataset_new)

It gives following error msg:

[HailoRT] [error] CHECK failed - UserBuffQEl13yolov8n_official/conv41 (D2H) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
[HailoRT] [error] Failed waiting for threads with status HAILO_TIMEOUT(4)
[HailoRT] [error] CHECK failed - UserBuffQEl6yolov8n_official/conv63 (D2H) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
[HailoRT] [error] CHECK failed - UserBuffQEl15yolov8n_official/conv52 (D2H) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
[HailoRT] [error] CHECK failed - UserBuffQEl12yolov8n_official/conv42 (D2H) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
[HailoRT] [error] Failed waiting for threads with status HAILO_TIMEOUT(4)
[HailoRT] [error] Failed waiting for threads with status HAILO_TIMEOUT(4)
[HailoRT] [error] CHECK failed - UserBuffQEl11yolov8n_official/conv62 (D2H) failed with status=HAILO_TIMEOUT(4) (timeout=10000ms)
[HailoRT] [error] Failed waiting for threads with status HAILO_TIMEOUT(4)
[HailoRT] [error] Failed waiting for threads with status HAILO_TIMEOUT(4)
Backend TkAgg is interactive backend. Turning interactive mode on.

What is the issue here?

omria · June 10, 2024, 10:54am

hey @ShingHin.Hung ,

you have to run it using the “yolov8n.hef” not the .har .

if you don’t have the hef , you can extract it from the ‘{model_name}_compiled_model.har’.

Hope this fixes the issue, please let me know.
Regards,
Omri

ShingHin.Hung · June 10, 2024, 11:36am

Hi @omria , I have the .hef file but how do I deploy it onto the Hailo 8 hardware?

To my knowledge ClientRunner does not take hef as an argument.

I also tried this:

runner = ClientRunner(hw_arch='hailo8', har=compiled_model_har_path)
runner.compile()

but it doesn’t work.

Could you provide a simple sample code for me?

Thank you!

omria · June 10, 2024, 1:02pm

hey @ShingHin.Hung

you can check full examples in here :

there are several good examples for yolov8n , like object detection or detection with tracker.

I would recommend you begin by looking at this inference :

github.com

hailo-ai/Hailo-Application-Code-Examples/blob/main/runtime/python/detection_with_tracker/hailo_inference.py

from hailo_platform import (HEF, VDevice, HailoStreamInterface, InferVStreams, ConfigureParams,
                InputVStreamParams, OutputVStreamParams, FormatType)

from loguru import logger
import numpy as np

class HailoInference():
    def __init__(self, hef_path):
        """
        Initialize the HailoInference class with the provided HEF model file path.

        Args:
            hef_path (str): Path to the HEF model file.
        """
        self.hef = HEF(hef_path)
        self.target = VDevice()
        self.network_group = self._configure_and_get_network_group(self.hef, self.target)
        self.network_group_params = self.network_group.create_params()
        self.input_vstreams_params, self.output_vstreams_params = self._create_vstream_params(self.network_group)
        self.input_vstream_info, self.output_vstream_info = self._get_and_print_vstream_info(self.hef)

This file has been truncated. show original

Hope this fixes the issue, please let me know.

ShingHin.Hung · June 11, 2024, 10:46pm

@omria thanks for the sample codes, I believe this what I need.

However I ran into a version issue. In the “requirement.txt”, it states that the package supervision == 0.19.0 is required:

During installation, the system shows that it requires pillow >= 9.4 to install supervision 0.19.0, and I am needed to upgrade pillow version:

However, after the upgrade, Hailo Model Zoo crashed because it requires pillow <= 9.2.0:

Isnt’t it a contradiction of versions?

ShingHin.Hung · June 12, 2024, 7:55am

But anyway, I downgrade the pillow back to 9.2.0 and everything seems working.

nina-vilela · June 13, 2024, 8:12am

Hi @ShingHin.Hung,

To avoid compatibility issues, it’s best to leave the DFC/MZ venv for their tools. So once you have a compiled model, you can use the hef on a separate venv for the inference.

Topic		Replies	Views
HAILO_FW_CONTROL_FAILURE(18) error General hailo8 , error	4	205	July 9, 2024
HAILO_TIMEOUT Error General hef , compiler , hailo8	3	50	November 7, 2024
Model compilation run-time error General compiler , hailo8 , error	8	151	September 11, 2024
CHECK_SUCCESS failed with status=HAILO_NOT_FOUND(61) after period of time General hailort , error	1	13	December 10, 2024
Segmentation fault / std::bad_function_call while idling General	3	64	August 20, 2024

Hailo_timeout(4)

Related topics