Adding NMS Layer to custom model

General
1

Hello. I have my own onnx model and tried to quantize/compile it to a .hef file. However when i run the basic example with this model

python basic_pipelines/detection.py --input resources/detection0.mp4 --hef test/fish.hef

i get the error

NMS score threshold is set, but there is no NMS output in this model.
CHECK_SUCCESS failed with status=6

What is the problem here?

I looked at lot in the doku and tried the steps in the tutorial “Adding Non-Maximum Suppression (NMS) Layer Through Model Script Commands” in hopes this would solve above error. However, my runner also throws errors

AllocatorScriptParserException: Cannot infer bbox conv layers automatically. Please specify the bbox layer in the json configuration file

thanks in advance for help

3

Hi @kamper,
It’s difficult to say without seeing your model, but in principal the detection.py example works only for models that are compiled with Hailo-NMS. The are a number of supported detection models that have this feature enabled and they can be found here:

If your ONNX have the same architecture and structure as one of the models in the link I provided, then during the compilation process to a HEF file you’ll have the option to add the Hailo-NMS and have your model work successfully with the detection.py example. If not, then the example as won’t work out of the box for your model.

Can you specify exactly from which git repo have you taken this example?

Regards,

4

actually I used YOLOv8n and pytorch to retrain a default yolov8n.pt model. I then exported the trained model to onnx format. And now i wanted to convert it so i can run it fast on the Raspberry Pi Ai Kit.

from ultralytics import YOLO

model = YOLO("yolov8n.pt")

model.train(...)  # with annotated data

model.export(format="onnx")

With this onnx model I tried to convert to .hef. Is this not a valid workflow? I know you can use models from the model zoo as a basis and finetune them somehow, but since I already have a working model on PC (and on the Pi but 130ms per frame is slow) I thought it would be the easiest to convert this one

5

This flow is also okay.
The easiest way is to use the model-zoo retraining dockers.
I believe that what happens here, is that some layer names is changed, and with this the default config on the NMS layer is not able to find the right layers. So you should identify the end nodes of the model, and place it in the json mentioned by @Omer :slight_smile:
image

7

@nadav, @Omer - this was the closest post I could find to creating custom nms configs, hopefully you can help. I’m attempting to compile yolov5-p2 to .hef

My end nodes are like so:

parsing code 
runner = ClientRunner(hw_arch=chosen_hw_arch)
hn, npz = runner.translate_onnx_model(
    onnx_path,
    onnx_model_name,
    start_node_names=["/model.0/conv/Conv"],
    end_node_names=["/model.31/m.3/Conv", 
                   "/model.31/m.2/Conv",
                   "/model.31/m.1/Conv",
                   "/model.31/m.0/Conv"],
    net_input_shapes={"/model.0/conv/Conv": [1, 3, 640, 640]},
)

and I inspected the parsed model layers to check the renmaed nodes:

har output layers 
('yolov5np2_visdrone/output_layer1',
              OrderedDict([('type', 'output_layer'),
                           ('input', ['yolov5np2_visdrone/conv89']),
                           ('output', []),
                           ('input_shapes', [[-1, 160, 160, 18]]),
                           ('output_shapes', [[-1, 160, 160, 18]]),
                           ('original_names', ['out']),
                           ('compilation_params', {}),
                           ('quantization_params', {}),
                           ('transposed', False),
                           ('engine', 'nn_core'),
                           ('io_type', 'standard')])),
             ('yolov5np2_visdrone/output_layer2',
              OrderedDict([('type', 'output_layer'),
                           ('input', ['yolov5np2_visdrone/conv99']),
                           ('output', []),
                           ('input_shapes', [[-1, 80, 80, 18]]),
                           ('output_shapes', [[-1, 80, 80, 18]]),
                           ('original_names', ['out']),
                           ('compilation_params', {}),
                           ('quantization_params', {}),
                           ('transposed', False),
                           ('engine', 'nn_core'),
                           ('io_type', 'standard')])),
             ('yolov5np2_visdrone/output_layer3',
              OrderedDict([('type', 'output_layer'),
                           ('input', ['yolov5np2_visdrone/conv111']),
                           ('output', []),
                           ('input_shapes', [[-1, 40, 40, 18]]),
                           ('output_shapes', [[-1, 40, 40, 18]]),
                           ('original_names', ['out']),
                           ('compilation_params', {}),
                           ('quantization_params', {}),
                           ('transposed', False),
                           ('engine', 'nn_core'),
                           ('io_type', 'standard')])),
             ('yolov5np2_visdrone/output_layer4',
              OrderedDict([('type', 'output_layer'),
                           ('input', ['yolov5np2_visdrone/conv121']),
                           ('output', []),
                           ('input_shapes', [[-1, 20, 20, 18]]),
                           ('output_shapes', [[-1, 20, 20, 18]]),
                           ('original_names', ['out']),
                           ('compilation_params', {}),
                           ('quantization_params', {}),
                           ('transposed', False),
                           ('engine', 'nn_core'),
                           ('io_type', 'standard')]))])

So to create the nms config, I used one of the yolov5 configs but added an extra bbox_decoder and updated the anchors:

custom model nms config 
{
    "nms_scores_th": 0.001,
    "nms_iou_th": 0.6,
    "image_dims": [
        640,
        640
    ],
    "max_proposals_per_class": 100,
    "background_removal": false,
    "classes": 1,
    "bbox_decoders": [
        {
            "name": "bbox_decoder89",
            "w": [
                2.01172,
                2.68945,
                4.41016
            ],
            "h": [
                3.97266,
                5.95312,
                5.50000
            ],
            "stride": 8,
            "encoded_layer": "conv89"
        },
        {
            "name": "bbox_decoder99",
            "w": [
                3.53125,
                5.31641,
                5.08594
            ],
            "h": [
                8.80469,
                8.64062,
                12.39062
            ],
            "stride": 16,
            "encoded_layer": "conv99"
        },
        {
            "name": "bbox_decoder111",
            "w": [
                8.03906,
                6.73438,
                9.27344
            ],
            "h": [
                10.12500,
                15.82812,
                17.54688
            ],
            "stride": 32,
            "encoded_layer": "conv111"
        },
        {
            "name": "bbox_decoder121",
            "w": [
                11.79688,
                15.61719,
                34.09375
            ],
            "h": [
                21.31250,
                29.06250,
                36.06250
            ],
            "stride": 64,
            "encoded_layer": "conv121"
        }
    ]
}

This will optimize without error but the model is unable to detect objects, just plots rectangles around nothing, even if I only use full precision

alls =  """
normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
change_output_activation(sigmoid)
model_optimization_config(calibration, batch_size=8, calibset_size=64)
post_quantization_optimization(finetune, policy=enabled, learning_rate=0.00001, epochs=4, batch_size=8, dataset_size=1024)
nms_postprocess("/local/shared_with_docker/visdrone/postprocess_config/yolov5np2v6_nms_config_custom.json", yolov5, engine=cpu)
performance_param(compiler_optimization_level=max)
allocator_param(width_splitter_defuse=disabled)

# """

runner.load_model_script(alls)
runner.optimize_full_precision()

Any suggestions as to where I’m going wrong here?