Improvig Segformer B0

General
1

Hello,
I have a compiled hef B0 segformer. I manage to build the application arround it. However, the results of the inference looks very bad. I am attaching here an image showing the inference results. How can I improve the results? From where to start?

segmentation_output0
segmentation_output0848×480 285 KB

2

Hello @dudibs,

The image you uploaded indicates that the segmentation result is noisy, with inaccurate and fragmented predictions overlaid on the original image. To improve the inference results of SegFormer B0, you can start by checking the following:

1. Input Preprocessing

  • Ensure the image is resized correctly (e.g., 512x512).
  • Confirm the image is normalized using the same mean and standard deviation as during model training.
  • Verify the channel order (RGB or BGR) and data type.

2. Model and Runtime

  • Verify that the .har file has been correctly converted before transforming the model into a .hef file. You can use the hailo visualizer command to inspect the .har file. Detailed instructions for using this command are available in the RidgeRun Wiki - Hailo Visualizer.
  • If the model is quantized, ensure that accuracy loss during quantization is not significant.

3. Post-Processing

  • Ensure you’re applying the correct post-processing.

4. Dataset and Input Alignment

  • Compare your input data distribution with the dataset used for training.
  • Check if there’s a mismatch in resolution or object types.

5. Testing

  • Test with images from the training dataset to verify the converted model and the pipeline used.

I hope this helps you to start improving your results.

Regards,

Oscar Mendez
Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.ai
Developers wiki: Hailo AI Platform Guide
Website: www.ridgerun.ai

3

Thanks a lot Oscar. This is a b0 segformer model that i have trained on a single channel images. then i compiled it to hef model. hence there is no need to resize to 512512 but rather to 480848. I did find problems in the preprocessing step but it didnt improve the results and the green artifact coming out from the model as in the image i attached is still there. I suspect its from the model. how do i know that the hef model has been generated adequately?

4

Hi @dudibs,

Were you able to check if the conversion of your model to HAR was successful using the command I shared? Here’s the link with the usage explanation: Hailo Visualizer Command Usage. It’s a good idea to confirm this because, if the model wasn’t properly converted to HAR, the issues might carry over when converting it to HEF.

Let me know if you need any help!

Best Regards,

Oscar Mendez
Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.ai
Developers wiki: Hailo AI Platform Guide
Website: www.ridgerun.ai