Custom Model with sensor data input

General
1

we are ramping up our knowledge on the Hailo8, and have few questions.
While reading the HailoRT User guide, on Chapter 6 the supported inference stages and data preparation are described.
This with a strong focus on images as being the input for the network and the inference calculation ( NHWC, NCHW, NHCW, N12, N21,etc…)

Now, say we have a custom neural network where the input data consists of 1D timeseries of data coming from sensors. Is there any API or function or example we can refer to pack up the input data for the network that does not resemble an image RGB? we could not find this explained in the docs.

much appreciated
-d

2

Hey @ds1 ,

HailoRT does support custom tensor inputs, including 1D time-series data from sensors. However, to ensure compatibility with the hardware, the data needs to be properly formatted and quantized before being sent for inference.

Here is the recommended way to do so :

  1. Reshape your sensor data to match the expected tensor format (e.g., NHWC, NCHW, or NHCW). Treat the data as a single-channel “image” where:

    • Height = 1
    • Width = sequence length (e.g., 1000 samples)
    • Channels = 1 (or more, depending on input features)

    For example:

    reshaped_data = sensor_data.reshape(1, 1, 1000, 1)  # (Batch=1, H=1, W=1000, C=1)

  2. Use the HailoRT API to create an input virtual stream:

    hailo_status status;
hailo_input_vstream input_vstream;
status = hailo_create_input_vstreams(configured_network_group, inputs_params, 1, &input_vstream);

  3. If your sensor data is in floating point format, use hailo_input_transform_context to apply quantization before sending the data:

    hailo_input_transform_context transform_context;
status = hailo_input_transform_context_create(&input_vstream, &transform_context);

    This step scales the raw sensor data into a suitable numerical range for Hailo inference.

  4. Send the reshaped and quantized 1D sensor data to the device:

    status = hailo_vstream_write_raw_buffer(input_vstream, reshaped_data, buffer_size);

  5. Read the output data from the device:

    status = hailo_vstream_read_raw_buffer(output_vstream, output_buffer, buffer_size);

Note that if you’re using raw input streams, you can handle the quantization manually before sending the data. Also, the NHCW format is preferred for optimal performance on Hailo hardware.

Let me know if you have any further questions!

3

Thanks for your support, much appreciated.
We are trying out your suggestion and will be reverting back if we have issues.

1 Like