Error converting onnx model using dfc

General
1

Hi, i’m tring to convert openpilot’s supercombo.onnx to hef, but something wrong.

My dfc_script.py:

import numpy as np
import os

from hailo_sdk_client import ClientRunner

#Define model information
model_name = 'supercombo'
onnx_path = '/tmp/fp16_to_fp32.onnx'
start_node = ["input_imgs","big_input_imgs", "desire",
        "traffic_convention","lateral_control_params",
        "prev_desired_curv", "features_buffer"
        ]
end_node = "outputs"
input_shape = {
        "input_imgs": [1,12,128,256],
        "big_input_imgs": [1,12,128,256],
        "desire": [1,100,8],
        "traffic_convention": [1,2],
        "lateral_control_params": [1,2],
        "prev_desired_curv": [1,100,1],
        'features_buffer': [1, 99,512]}
output_shape = {
        "outputs": [1, 6504]
        }
chosen_hw_arch = 'hailo8'

#alls_lines = [
#   'model_optimization_flavor(optimization_level=0, compression_level=1)\n',
#    'resources_param(max_control_utilization=1.0, max_compute_utilization=1.0,max_memory_utilization=1.0)\n',
#    'performance_param(fps=250)\n'
#]

#Parsing
runner = ClientRunner(hw_arch=chosen_hw_arch)
hn, npz = runner.translate_onnx_model(onnx_path, model_name, start_node_names=start_node, end_node_names=end_node, net_input_shapes=input_shape)

parsed_model_har_path = f'{model_name}_parsed_model.har'
runner.save_har(parsed_model_har_path)

#Optimize
#calibData = np.random.randint(0, 255, (1024, input_height, input_width, input_ch))

#runner.load_model_script(''.join(alls_lines))
#runner.optimize(calibData)

#quantized_model_har_path = f'{model_name}_quantized_model.har'
#runner.save_har(quantized_model_har_path)

#Compile
hef = runner.compile()

file_name = f'{model_name}.hef'
with open(file_name, 'wb') as f:
    f.write(hef)

compiled_model_har_path = f'{model_name}_compiled_model.har'
runner.save_har(compiled_model_har_path)

Logs:

bird@minipc:~$ python3.10 dfc_script.py
[warning] Cannot use graphviz, so no visualizations will be created
[info] Translation started on ONNX model supercombo
[info] Restored ONNX model supercombo (completion time: 00:00:00.32)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.58)
[info] Simplified ONNX model for a parsing retry attempt (completion time: 00:00:04.27)
Traceback (most recent call last):
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 179, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(onnx_model, valid_net_name, start_node_names,
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 237, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(onnx_model, None, net_name, start_node_names, end_node_names,
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 263, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 76, in convert_model
    self._create_layers()
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 28, in _create_layers
    self._add_direct_layers()
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 138, in _add_direct_layers
    raise ParsingWithRecommendationException(
hailo_sdk_client.model_translator.exceptions.ParsingWithRecommendationException: Parsing failed. The errors found in the graph are:
 UnexpectedNodeError in op /Unsqueeze_1: Unexpected node /Unsqueeze_1 (Unsqueeze)
 UnsupportedReduceMeanLayerError in op /_attention/layer_norm/ReduceMean: Reduce mean layer /_attention/layer_norm/ReduceMean has unsupported axis -1 (must be over one spatial dimension only).
 UnsupportedReduceMeanLayerError in op /_mlp/layer_norm/ReduceMean: Reduce mean layer /_mlp/layer_norm/ReduceMean has unsupported axis -1 (must be over one spatial dimension only).
 UnsupportedReduceMeanLayerError in op /_attention/layer_norm/ReduceMean_1: Reduce mean layer /_attention/layer_norm/ReduceMean_1 has unsupported axis -1 (must be over one spatial dimension only).
 UnsupportedReduceMeanLayerError in op /_mlp/layer_norm/ReduceMean_1: Reduce mean layer /_mlp/layer_norm/ReduceMean_1 has unsupported axis -1 (must be over one spatial dimension only).
 UnsupportedFeatureSplitterLayerError in op /_attention/Split: Feature splitter vertex /_attention/Split is splitting input over unsupported axis 2
 UnexpectedNodeError in op /Unsqueeze: Unexpected node /Unsqueeze (Unsqueeze)
Please try to parse the model again, using these end node names: /_extra_in/_extra_in.1/Relu, /summarizer/Div

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bird/dfc_script.py", line 35, in <module>
    hn, npz = runner.translate_onnx_model(onnx_path, model_name, start_node_names=start_node, end_node_names=end_node, net_input_shapes=input_shape)
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 876, in translate_onnx_model
    parser.translate_onnx_model(model=model, net_name=net_name, start_node_names=start_node_names,
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 211, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(simplified_model, valid_net_name,
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 237, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(onnx_model, None, net_name, start_node_names, end_node_names,
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 263, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 76, in convert_model
    self._create_layers()
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 28, in _create_layers
    self._add_direct_layers()
  File "/home/bird/.local/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 138, in _add_direct_layers
    raise ParsingWithRecommendationException(
hailo_sdk_client.model_translator.exceptions.ParsingWithRecommendationException: Parsing failed. The errors found in the graph are:
 UnexpectedNodeError in op /Unsqueeze_1: Unexpected node /Unsqueeze_1 (Unsqueeze)
 UnsupportedReduceMeanLayerError in op /_attention/layer_norm/ReduceMean: Reduce mean layer /_attention/layer_norm/ReduceMean has unsupported axis -1 (must be over one spatial dimension only).
 UnsupportedReduceMeanLayerError in op /_mlp/layer_norm/ReduceMean: Reduce mean layer /_mlp/layer_norm/ReduceMean has unsupported axis -1 (must be over one spatial dimension only).
 UnsupportedReduceMeanLayerError in op /_attention/layer_norm/ReduceMean_1: Reduce mean layer /_attention/layer_norm/ReduceMean_1 has unsupported axis -1 (must be over one spatial dimension only).
 UnsupportedReduceMeanLayerError in op /_mlp/layer_norm/ReduceMean_1: Reduce mean layer /_mlp/layer_norm/ReduceMean_1 has unsupported axis -1 (must be over one spatial dimension only).
 UnsupportedFeatureSplitterLayerError in op /_attention/Split: Feature splitter vertex /_attention/Split is splitting input over unsupported axis 2
 UnexpectedNodeError in op /Unsqueeze: Unexpected node /Unsqueeze (Unsqueeze)
Please try to parse the model again, using these end node names: /_extra_in/_extra_in.1/Relu, /summarizer/Div

What should i do now, thanks.:heart:

2

About this model, you can find at github
commaai/openpilot/blob/master/selfdrive/modeld/models/supercombo.onnx

3

Hi,
From some point in the model, the dimensions are reduced to 3, which is a problem:
image
Been able to parse (after simplifier) up untli there:
hailo parser onnx supercombo-sim.onnx --end-node-names Conv_200 --start-node-names Conv_0

4

More ops could be run on Hailo, if you change the model to use 4-D tensors all along the way.

5

Does it mean that Hailo does not support changes in the number of dimensions? How can reshape and transport be replaced？

6

@Nadav is it possible to get an answer to the previous question?

7

Support for transpose and reshape are not generic, and depends on the context. You can get deeper dive on that in the “supported layers” in the DFC user guide.

2 Likes