Runtime adjustment of input resize

General
1

Hello,

I am trying to integrate a Hailo 8 module into an existing (windows) application, using the C library, using an existing 320x320 (yolo3) network. I need to interact with different input streams, using different input resolutions. I am currently performing a bilinear resampling in software, and this is slow.

I know I can perform input resampling within the Hailo module by adding a model script containing a resize layer. However, from what I understand I need to add this script prior to the optimization and compilation steps. This means my .hef file is hard-coded to a specific input resolution; if I want to support multiple resolutions I would end up with multiple .HEFs, which are individually optimized. This isn’t optimal.

  1. Is there a way to change the input resolution of a (v)stream at runtime, using only the libhailort library? I don’t think there is, but there’s no harm in asking.
  2. Is there a way to separate the input resampling from the actual inference, so that I can use the same inference network (with the same quantisation!) on multiple resolutions. I feel like network groups were intended for this exact purpose, but they seem to require individual hefs for every network.
  3. How can I create a HEF which performs only resampling?

I’m probably asking for something obvious, but I’m not a data scientist and am getting confused by the documentation. If there is an existing example how to achieve something like this, I would appreciate it if someone pointed it out.

2

Yes, HEF files have a fixed input size.

Resizing should be done on the host. Since you are running on Windows I would expect your system to have enough performance to do this easily. Maybe you can find a better library.
Resizing on the Hailo-8 is only useful in rare circumstances. You are wasting PCIe bandwidth by sending images that are larger than required. So, unless your host is very weak and has enough PCIe bandwidth this is not the best option.

You would have to create a network e.g., in PyTorch that does not modify the image. I do not know whether it is possible to create this type of network and then convert it with the Hailo Dataflow Compiler.

You can try ChatGPT. It can probably create a simple network in PyTorch for you.

1 Like
3

Thank you for your response.

I noticed that the performance of the network took a significant hit when I added a resampling layer; I went from 1700 FPS with a latency of ~1.5 ms (which is much better than I really need) to 83 FPS with a latency of 14.7 ms. Input bandwidth didn’t go up all that much (from 150 MB/s to 182 MB/s); I don’t think I should be hitting any PCIe limitations at this point but maybe I am.

4

I don’t recommend adding a resize layer to your model.hef. It costs a lot of computation.

Useing the CPU to perform input image resize. it’s cost small than hailo.