Error converting segormer b0 onnx model using DFC

General
1
  1. I install the DFC. Works fine converting yolo model.
  2. I have segformer b0 onnx model. weight 12 mb.
  3. I am getting this error from the DFC:

/home/nuchailo1/anaconda3/envs/hailocompiler/bin/python /home/nuchailo1/PycharmProjects/hailocompiler/hcompiler.py
[info] Translation started on ONNX model mapcoresegformer_small
[info] Restored ONNX model mapcoresegformer_small (completion time: 00:00:00.05)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.71)
[info] Start nodes mapped from original model: ‘input’: ‘mapcoresegformer_small/input_layer1’.
[info] End nodes mapped from original model: ‘/backbone/layers.1.0/Transpose’.
[info] Translation completed on ONNX model mapcoresegformer_small (completion time: 00:00:01.49)
[warning] hw_arch parameter not given, using the default hw_arch hailo8.
If another device is the target, please run again using one of hailo8, hailo8r, hailo8l, hailo15h, hailo15m
Traceback (most recent call last):
File “/home/nuchailo1/PycharmProjects/hailocompiler/hcompiler.py”, line 12, in
hn, npz = runner.translate_onnx_model(onnx_path, onnx_model_name,start_node_names=start_node,end_node_names=end_node)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 881, in translate_onnx_model
return self._finalize_parsing(parser.return_data)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 937, in _finalize_parsing
self.set_hn(return_data[‘hn_data’])
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 1356, in set_hn
self._sdk_backend = self._generate(hn)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 136, in _generate
hn = self._load_hn(hn)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 1363, in _load_hn
return HailoNN.from_hn(hn)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_common/hailo_nn/hailo_nn.py”, line 1370, in from_hn
return HNImporter().from_hn(hn_json)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_common/hailo_nn/hailo_nn.py”, line 1602, in from_hn
return self.from_parsed_hn(json.loads(ensure_str(hn_json)))
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_common/hailo_nn/hailo_nn.py”, line 1585, in from_parsed_hn
self._add_layers()
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_common/hailo_nn/hailo_nn.py”, line 1643, in _add_layers
layer_parsed = self.create_layer(layer_hn, layer_name)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_common/hailo_nn/hailo_nn.py”, line 1653, in create_layer
layer_parsed = cls.TYPE_TO_CLASS[layer_hn[‘type’]].from_hn(layer_hn)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_common/hailo_nn/hn_layers/fused_conv2d.py”, line 246, in from_hn
layer = super(FusedConv2DLayer, cls).from_hn(hn)
File “/home/nuchailo1/anaconda3/envs/hailocompiler/lib/python3.10/site-packages/hailo_sdk_common/hailo_nn/hn_layers/conv2d.py”, line 399, in from_hn
raise UnsupportedModelError(
hailo_sdk_common.hailo_nn.exceptions.UnsupportedModelError: Input features and output features must be a multiply of groups for conv layer mapcoresegformer_small/conv14 (translated from /backbone/layers.1.0/Transpose)

Specifically the error is coming from the following check:

** if hn[‘type’] in [LayerType.conv.value, LayerType.deconv.value]:**
** group_sizes_sum = sum(layer.group_sizes)**
** if layer.kernel_shape[2] % group_sizes_sum != 0 or layer.kernel_shape[3] % group_sizes_sum != 0:**
** raise UnsupportedModelError(**
** f’Input features and output features must be a multiply of groups for {layer.full_name_msg}'**
** )**

in conv2d.py file

How I can overcome this error?
I can simply comment this check and the har file is created but i guess it is important since the hef model created from this file is not working after commenting these lines.

Any Ideas?

All Best…
David

2

You would probably not be able to compile the model.
I think that the error message, in this case, suggests a solution:

Input features and output features must be a multiply of groups for conv layer

The question is do you think that you can comply with that?

3

thanks for the reply Nadav.
I am not sure how I can comply with that. I cannot change the model. I can delete this check from the DFC to create the har file, but i am failing in the next step - in the quantization.

4

Hi @dudibs,

Have you re-trained the model? If not, you can download an already compiled version from our ModelZoo collection. That model version has BatchNorm instead of LayerNorm, but that will change in the next release.

5

Hi Marina,
Thanks a lot for the info. I did re install the DFC and manage to create the har file for the segformer without changing it to batch norm.
Now i wish to create the hef file.
I run the simple optimization process using this command:
$ hailo optimize mapcoresegformer_small_hailo_model_2.har –use-random-calib-set

and got the quantized model :slight_smile:

next i run the command:

$ hailo compiler mapcoresegformer_small_optimized.har

and I am getting the an error that the mapping is failed.

Following is the stack trace:

(hcompiler2) nuchailo1@nuchailo1:~/PycharmProjects/hcompiler2$ hailo compiler mapcoresegformer_small_optimized.har

info] Current Time: 15:10:03, 08/08/24
[info] CPU: Architecture: x86_64, Model: 13th Gen Intel(R) Core™ i7-1360P, Number Of Cores: 16, Utilization: 0.4%
[info] Memory: Total: 15GB, Available: 8GB
[info] System info: OS: Linux, Kernel: 6.5.0-44-generic
[info] Hailo DFC Version: 3.28.0
[info] HailoRT Version: Not Installed
[info] PCIe: 0000:57:00.0: Number Of Lanes: 1, Speed: 8.0 GT/s PCIe
[info] Running hailo compiler mapcoresegformer_small_optimized.har
[info] Compiling network
[info] To achieve optimal performance, set the compiler_optimization_level to “max” by adding performance_param(compiler_optimization_level=max) to the model script. Note that this may increase compilation time.
[info] Loading network parameters
[info] Starting Hailo allocation and compilation flow
[warning] Failed to add spatial reshapes
[info] Adding collapsed format conversion after conv_feature_splitter1_2
[info] Adding collapsed format conversion after conv_feature_splitter2_2
[error] Mapping Failed (allocation time: 30s)
No successful assignment for: format_conversion11

[error] Failed to produce compiled graph
[error] BackendAllocatorException: Compilation failed: No successful assignment for: format_conversion11

How to overcome it? why does it failed with the mapping?

Thanks
David