[Python API][RPI5] Coherent inference boxes but not their positions

General
, , ,
1

Hi,

I’ve managed to use my model in python thanks to version 4.18, I get a result with coherent boxes but badly positioned, I’ve tried lots of things, but nothing that corrects this, on the spot I wonder what I’ve done wrong.

I create my model, get the dimensions it manages:

self.hailo_inference = HailoInference(os.getenv('MODEL_PATH'))
self.height, self.width, _ = self.hailo_inference.get_input_shape()

I convert my image (CV2 => PIL) and put it in square format with black margins (my image is well transformed):

image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
image = Image.fromarray(image)
processed_image = preprocess(image, self.width, self.height)

I send the image to inference:

raw_detections = self.hailo_inference.run(np.array(processed_image))

And here are the results I get:

[0.4765124 , 0.91401637, 0.52448344, 0.9630216 , 0.8998809 ]

From what I understand, the table returns :
y_min, x_min, y_max, x_max, and the score

When I try to convert these results into “YOLO” equivalents, I get this, gor example, for the first result :

Class : 0
Score : 0.8998808860778809
x_center : 0.9385189712047577
y_center : 0.5004979223012924
width : 0.049005210399627686
height : 0.04797104001045227

And in pixels on the image it looks like this :

Image width : 1024
Image height : 1024
X min : 936 
Y min : 488 
X max : 986 
Y max : 537

And what I should get (result of the same inference but on my original model with ultralytics, YOLO and my pytorch model):

Class : 0
Score : 0.95
x_center : 0.5867246627807617
y_center : 0.16733388547544126
width : 0.030229949951171876
height : 0.05325857091833044

And in pixels on the image it looks like this :

Image width : 1920
Image height : 1080
X min : 1097 
Y min : 151 
X max : 1155 
Y max : 209

Thanks for help

2

Hi @alkemist,

This could be due to quantization noise. What is the model architecture and how did you compile it?

3

I made a post where I explained my complications with compiling the template :

And I finally succeeded with :slight_smile:

hailomz compile --ckpt guinea-pig-chons-v12.onnx --hw-arch hailo8l --calib-path data --yaml hef_config.yaml --classes 4 --performance

this file hef_config.yaml

- base/yolov8.yaml
postprocessing:
  device_pre_post_layers:
    nms: true
  hpp: true
network:
  network_name: yolov8n
paths:
  network_path:
  - models_files/ObjectDetection/Detection-COCO/yolo/yolov8n/2023-01-30/yolov8n.onnx
  alls_script: yolov8n.alls
  url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov8n/2023-01-30/yolov8n.zip
parser:
  nodes:
  - null
  - - /model.22/cv2.0/cv2.0.2/Conv
    - /model.22/cv3.0/cv3.0.2/Conv
    - /model.22/cv2.1/cv2.1.2/Conv
    - /model.22/cv3.1/cv3.1.2/Conv
    - /model.22/cv2.2/cv2.2.2/Conv
    - /model.22/cv3.2/cv3.2.2/Conv
info:
  task: object detection
  input_shape: 640x640x3
  output_shape: 80x5x100
  operations: 8.74G
  parameters: 3.2M
  framework: pytorch
  training_data: coco train2017
  validation_data: coco val2017
  eval_metric: mAP
  full_precision_result: 37.23
  source: https://github.com/ultralytics/ultralytics
  license_url: https://github.com/ultralytics/ultralytics/blob/main/LICENSE
  license_name: GPL-3.0

I’ve just noticed that the input_shape is different from mine (my model is trained on 1024x1024).

1 Like
4

Awesome, really nice job!

5

When I said I’d succeeded, it was at the end of this post in converting the model.

On the other hand, the problem of incorrect inference came later, and it’s still there.

Any idea where it might be coming from?

Should the “input_shape” in the config file correspond to my model (and therefore 1024x1024) or does it only correspond to the “yolov8n” vase model and therefore I shouldn’t change it?
The same goes for the “output_shape”, I’ve got the impression it’s the 80 corresponding to the number of classes (and I’ve only got 4 in my model).

Here are the 2 other files used for compilation

yolov8n.alls

normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
change_output_activation(conv42, sigmoid)
change_output_activation(conv53, sigmoid)
change_output_activation(conv63, sigmoid)
nms_postprocess(“/local/shared_with_docker/followchon_back/models/config/yolov8n_nms_config.json”, meta_arch=yolov8)

And yolov8n_nms_config.json

{
  “nms_scores_th": 0.2,
  “nms_iou_th": 0.7,
  “image_dims": [
    1024,
    1024
  ],
  “max_proposals_per_class": 100,
  “classes": 4,
  “regression_length": 16,
  “background_removal": false,
  “bbox_decoders": [
    {
      “name": “yolov8n/bbox_decoder41”,
      “stride": 8,
      “reg_layer": “yolov8n/conv41”,
      “cls_layer": “yolov8n/conv42”
    },
    {
      “name": “yolov8n/bbox_decoder52”,
      “stride": 16,
      “reg_layer": “yolov8n/conv52”,
      “cls_layer": “yolov8n/conv53”
    },
    {
      “name": “yolov8n/bbox_decoder62”,
      “stride": 32,
      “reg_layer": “yolov8n/conv62”,
      “cls_layer": “yolov8n/conv63”
    }
  ]
}
6

Thanks for clarifying.

There are some known issues with the quantization of yolov8 re-trained with few classes. You can try adding this solution to your model script.

Besides that, you can also try using some post-quant algorithms such as adaround. This can also be enabled with a command on the model script, for example:

post_quantization_optimization(adaround, policy=enabled)

You can read more about adaround and other post-quant algs here.

7

Well, it was just a problem of image size in the config file, replace 640 by 1024 ^^”

Thanks for the help

1 Like