Paser onnx model failed

General
1

I am attempting to convert a pp-ocr model to a hailo hef model. First, the pp-ocr model is converted to an onnx model. Then, the onnx model is converted to the hef format. However, I encounter an error when parsing the onnx model by DFC:

hailo_sdk_client.model_translator.exceptions.ParsingWithRecommendationException: Parsing failed. The errors found in the graph are:
UnsupportedSoftmaxLayerError in op p2o.Softmax.0: Unsupported softmax
UnsupportedSoftmaxLayerError in op p2o.Softmax.1: Unsupported softmax
Please try to parse the model again, using these end node names: p2o.MatMul.2, p2o.Squeeze.2, p2o.AveragePool.0, p2o.Transpose.0

How can I handle this case?

2

Hey @davis.bian ,

Welcome to the Hailo Community!

You can use the Netron app to visualize and analyze your ONNX model. It will help you identify the unsupported Softmax layers and plan the necessary modifications.

To resolve the error and make your model compatible with the Hailo Dataflow Compiler (DFC), you have two options:

  1. Remove the Softmax Layers:

    • Use the end nodes suggested in the error message (p2o.MatMul.2, p2o.Squeeze.2, etc.) to truncate the model.
    • Utilize tools like ONNX Graph Surgeon (ONNX-GS) or ONNX utilities to remove the Softmax layers and redefine the model’s output.

  2. Replace the Softmax Layers:

    • Replace the Softmax layers with equivalent operations that are supported by the Hailo DFC, such as custom normalization layers.

If the removed Softmax layers are essential for inference (e.g., to obtain probability distributions), you can perform the softmax operation during post-processing on the CPU or another supported platform.

  • Add the Softmax function in your application code after retrieving the raw outputs from the Hailo device:

    import numpy as np

def softmax(logits):
    exps = np.exp(logits - np.max(logits, axis=-1, keepdims=True))
    return exps / np.sum(exps, axis=-1, keepdims=True)

# Example usage
output = hailo_model.run_inference(input_data)
probabilities = softmax(output)

By applying these modifications, you should be able to compile your model successfully with the Hailo DFC.

3

Hey @omria
Thank you very much for the information. I will try those methods and share the results.

2 Likes
4

hey @davis.bian
I meet the same problem when dealing with ppocr-rec, could you teach me how to solve it?