Class agnostic NMS post processing for Nanodet

General
1

Hi,

i converted a nanodet model to .hef format with NMS in the model script. however, i want to use class agnostic NMS (not by class). therefore i tried to use bbox_decoding_only in my modelscript and use a custom NMS logic.

but when using the parameter bbox_decoding_only i don’t get any bounding boxes. the exact same pipeline works if i don’t set this parameter.

nms_postprocess('/nms_config.json', meta_arch=yolov8,bbox_decoding_only=True)\n"

my NMS config is:

{
  "nms_scores_th": 0.45,
  "nms_iou_th": 0.4,
  "image_dims": [
    288,
    512
  ],
  "max_proposals_per_class": 100,
  "classes": 3,
  "regression_length": 11,
  "background_removal": false,
  "background_removal_index": 0,
  "bbox_decoders": [
    {
      "name": "bbox_decoder1",
      "stride": 8,
      "combined_layer": ""
    },
    {
      "name": "bbox_decoder2",
      "stride": 16,
      "combined_layer": ""
    },
    {
      "name": "bbox_decoder3",
      "stride": 32,
      "combined_layer": ""
    }
  ]
}
2

Hi @josef.gugglberger,

Could you please check what output do you get from the model? You can check the model with hailortcli parse-hef HEF-PATH.

The outputs of a model with NMS and a model with “BBOX decoding only” are slightly different. If you take a YoloV8 trained on COCO as reference, you will see the following:

    Network name: yolov8s/yolov8s
        VStream infos:
            Input  yolov8s/input_layer1 UINT8, NHWC(640x640x3)
            Output yolov8s/yolov8_nms_postprocess FLOAT32, NHWC(1x8400x84)
            Operation:
                Op YOLOV8
                Name: YOLOv8Bbox-Only-Post-Process
                Classes: 80
                Image height: 640
                Image width: 640

Where 8400 is the number of predicted boxes and 84 is the number of features per prediction:

  • 4 box coordinates
  • 1 confidence score
  • 80 class score

Do you get any number as output of the model with decoding, or all zeros?

1 Like
3

hi,

thanks for your answer. i was able to fix it with the information from hailortcli:

Architecture HEF was compiled for: HAILO8L
Network group name: model, Multi Context - Number of contexts: 2
    Network name: model/model
        VStream infos:
            Input  model/input_layer1 UINT8, NHWC(288x512x3)
            Output model/yolov8_nms_postprocess FLOAT32, NHWC(1x3024x7)
            Operation:
                Op YOLOV8
                Name: YOLOv8Bbox-Only-Post-Process
                Classes: 3
                Image height: 288
                Image width: 512

i wasn’t realizing that the output changed and was reading the wrong data type (UINT instead of FLOAT32).

PS: i think in your example there should be 4 (box coords.) + 80 (class scores) = 84. your features add up to 85