How to get a machine learning application to utilise a Hailo accelerator?

General
1

Hello, I would like to understand how a machine learning application “knows” to use a Hailo accelerator rather than running on the CPU, or more generally how any machine learning application is accelerated with any NPU (since I’m assuming it doesn’t just happen automatically!).
I’ve been trying out the demo apps for the raspberry pi 5 AI kit and they are working well, but looking at the code I can’t figure out how I can utilise the Hailo accelerator for my own projects. Say I have some program using Sklearn or TensorFlow or PyTorch or whatever, how can I get the program to use the Hailo accelerator?

2

Hey @zaczn,

Welcome to the Hailo Community!

The application knows how to use the Hailo Neural Processing Unit (NPU) by utilizing the Hailo API (C++/Python).

If you have a model in PyTorch or TensorFlow, the first step is to convert it to Hailo’s internal representation of a model, called a Hailo Executable Format (HEF), using the Hailo DFC (Data Flow Compiler).

Once you have converted your model to the HEF format, you can refer to this example of Hailo usage in C++:

Best regards