Understanding NMS and nms_postprocess

General
1

Hi !
I am trying to understand exactly the meaning and behaviors of the NMS and nms_postprocess() function available in the .alls files.

I was not able to find any clear documentation about that. For instance, what is the difference of this :
nms_postprocess( meta_arch=yolov5, engine=cpu)
and this :
nms_postprocess( meta_arch=yolov5, engine=nn_core)
?

In addition, the fact that the .yaml file contains this:

postprocessing:
  device_pre_post_layers:
    nms: true

Is there a documentation on that topic or … ?

Thanks !

2

Hi @sylvain.fabre,

Welcome to the Hailo Community!

I am trying to understand exactly the meaning and behaviors of the NMS and nms_postprocess() function available in the .alls files.
I was not able to find any clear documentation about that.

The documentation about the nms_postprocess command is available in the Hailo DataFlow Compiler User Guide, section 5.1.5. NMS Post Processing.
The NMS algorithm is a post-processing operation to filter the predictions of object detectors models. The NMS is not part of the neural network itself, and it usually contains operations that are not supported by the Hailo-8 accelerator. However, for some of the most common models, Hailo offers the possibility to execute the NMS algorithm as part of the inference API, either on the host CPU (engine=cpu) or on a dedicated post-processing cluster available on the Hailo-8 (engine=nn_core).
The nms_postprocess command is used to instruct the compiler that the NMS must be performed after the model inference, in a way that is completely transparent to the user. In this way, when running the model using the HailoRT API, the obtained results will be the bounding boxes after NMS.

3

Thanks,
I was aware of what is NMS, but the way of processing it was unclear for me.
So, to summarize (to be sure I understand well :slight_smile: ) :

  • If engine=cpu then the NMS is performed inside the Hailo framework BUT in the CPU
  • if engine=nn_core then the NM is performed inside the Hailo framework BUT inside the HAILO8 chip.

If this is correct, then what is the purpose of this in the YAML file :

postprocessing:
  device_pre_post_layers:
    nms: true

Thanks !

4

Hi @sylvain.fabre,
The postprocessing section of the YAML file is necessary just for the Hailo Model Zoo evaluation pipeline. The hailomz tool needs to know whether the NMS is already part of HailoRT (nms: true), or if it needs to use Model Zoo postprocessing.
In this case is set to true, since the nms_postprocess command has been added to the model script.