How do I increase the max queue size for HailoRT?

General
,
4

Hey @tarmily.wen,

Here’s a clearer explanation of running inference with HailoRT:

import numpy as np
from hailo_platform import (
    HEF, VDevice, HailoStreamInterface, InferVStreams,
    ConfigureParams, InputVStreamParams, OutputVStreamParams, FormatType
)

def run_inference(hef_path, num_images=10):
    # Load model
    hef = HEF(hef_path)
    
    # Setup device and configure model
    with VDevice() as device:
        # Configure network
        config_params = ConfigureParams.create_from_hef(
            hef=hef, 
            interface=HailoStreamInterface.PCIe
        )
        network_group = device.configure(hef, config_params)[0]
        
        # Setup streams
        input_params = InputVStreamParams.make(
            network_group, 
            format_type=FormatType.FLOAT32
        )
        output_params = OutputVStreamParams.make(
            network_group, 
            format_type=FormatType.UINT8
        )
        
        # Prepare input data
        input_info = hef.get_input_vstream_infos()[0]
        h, w, c = input_info.shape
        dataset = np.random.rand(num_images, h, w, c).astype(np.float32)
        
        # Run inference
        with network_group.activate():
            with InferVStreams(network_group, input_params, output_params) as pipeline:
                results = pipeline.infer({input_info.name: dataset})
                return results

# Example usage
results = run_inference('path/to/your_model.hef')

Key components:

  1. Load HEF model
  2. Configure device and network
  3. Set up input/output streams
  4. Prepare data
  5. Run inference using context managers

For more examples, check: Hailo-Application-Code-Examples/runtime/python/utils.py at main · hailo-ai/Hailo-Application-Code-Examples · GitHub

1 Like