"Preferred" way to use multiple hef files on a single device

[Marc_Jasner]

Thanks for all of the help so far! I’m making good progress with my project, but I wanted to ask a question about what the preferred method in the Hailo ecosystem is for doing something.

My application uses pose detection and face detection/face landmark detection. When building the face detection part of the project I ran into a problem because I have a hailo8l (pi AI Hat) and after loading the face detection hef (scrfd_2.5) I can’t load the landmark detector without getting an error about no available devices. So I guess that means I need to load the face detector, do the inference, unload it, load the landmark detector, unload it, etc. Once I integrate with this the pose detector I’ll need to do the same for that as well (I’m using async inference).

That seems inefficient. Is there a better/preferred way to have multiple hefs available for inference on a Hailo8l? Is it not possible to do parallel inference of 2 hefs?

[KlausK]

@Marc_Jasner

Is there a better/preferred way to have multiple hefs available for inference on a Hailo8l?

Have a look at the HailoRT User Guide section Model scheduler. The scheduler will load and unload the model for you based on different parameters that help optimizing the switching overhead.

@Marc_Jasner

Is it not possible to do parallel inference of 2 hefs?

Only if you have two small models that fit into a single Hailo device, you can join them. However this is not scalable and should only be used when you never want to change the two models. When you want to extend the app or change the models it is better to create individual HEF files and use the scheduler.