Background Removal with Hailo Accelerator & ONNX Runtime

General
1

Hi Hailo Community,

I’m currently running a Raspberry Pi 5 (8GB) with a Python web server that removes image backgrounds using the rembg library. This library relies on ONNX Runtime, and I’m using the U2Net ONNX model for background removal.

I came across this example: Hailo ONNX Runtime Example, where ONNX inference is performed using the OpenVINOExecutionProvider. Initially, I thought this might support Hailo devices, but I now realize that isn’t the case.

Most examples I’ve found seem to focus on real-time video inference using HEF models. My question is:

  • Is there an ONNX Runtime execution provider that can leverage the Hailo accelerator for image processing tasks, such as background removal?
  • If not, is there an example or a recommended approach for using the Hailo device for this kind of non-real-time task?

Any guidance would be greatly appreciated!

Thank you!

2

Hey @Cameron

Welcome to the Hailo Community!

Here’s a summary of the current situation and possible approaches:

  1. Hailo doesn’t currently offer an ONNX Runtime Execution Provider. Hailo solutions primarily use HEF models optimized for real-time video inference and embedded AI tasks.

  2. For your background removal task:

    • Convert your ONNX model (U2Net) to HEF format using the Hailo SDK’s conversion tools.
    • This may require retraining or optimizing the model to meet Hailo’s architecture requirements.
    • Once converted, deploy the model using the Hailo runtime environment.

  3. Recommended approach:

    • Use the Hailo SDK to convert and optimize your ONNX model to HEF.
    • Develop a custom Python pipeline to use the Hailo accelerator for batch-processing images.

While there’s no direct ONNX Runtime support for Hailo, adapting your model with the Hailo SDK can still leverage the accelerator for your task. This approach requires some effort in optimization but could provide efficient execution on Hailo devices.

Let me know if you need more detailed guidance on model conversion or SDK setup!

Best regards,
Omri

3

Hey Omri,

Thank you for the fast and informative reply.

I’ll get started on converting and optimizing the model with the Hailo SDK. I’ll reach out if I have any questions along the way.

Best regards,
Cameron

4

Will the Hailo ever support the ONNX runtime?
what would be the limitations with this approach?
Is this on your roadmap?

Regards,

5

Hey @avi_ron,

There are several limitations to directly supporting ONNX Runtime with Hailo. One key limitation is that the Hailo hardware and SDK aren’t fully optimized for ONNX Runtime, which could lead to confusion and suboptimal performance. However, there are other ways to work with ONNX models using Hailo, as shown in this example repository:

Hailo ONNX Runtime Example

Feel free to explore this for an alternative approach to integrating ONNX models with Hailo’s hardware.

Regards,
Omri

1 Like