Error Converting ConvNeXT Model From ONNX to HEF Using DFC

General
, ,
1

Hello! I’m trying to take a ConvNeXT model that’s I’ve exported into ONNX format from PyTorch and converting it to HEF using the DFC.

Here is how I download and export the model. I’ve done the same with PyTorch’s ResNet18, and am able to convert successfully.

import torch
import torch.onnx
from torchvision.models import convnext_tiny

model = convnext_tiny(pretrained=True)
model.eval()

dummy_input = torch.randn(1, 3, 224, 224)

onnx_filename = "convnext_tiny.onnx"

torch.onnx.export(
    model,
    dummy_input,
    onnx_filename,
    export_params=True,
    opset_version=15,
    do_constant_folding=False,
    input_names=['input'],
    output_names=['output'],
)

print(f"Model exported to {onnx_filename}")

Here’s the model in Netron:

convnext_tiny.onnx
convnext_tiny.onnx174×24000 862 KB

I understand that not all layers are supported. However, if my model is not supported, I would like help in understand why that is. What part of the architecture is not supported?

2

Hi @blawyer,
What’s the error that you’re getting? In many cases, the DFC can suggest the right start or end nodes to get the model compiling.

3

My apologies for not including this in the original post!

hailo parser onnx --hw-arch hailo8l convnext_tiny.onnx
[info] Current Time: 10:42:58, 10/28/24
[info] CPU: Architecture: , Model: AMD Ryzen 7 5700G with Radeon Graphics, Number Of Cores: 16, Utilization: 0.1%
[info] Memory: Total: 27GB, Available: 26GB
[info] System info: OS: Linux, Kernel: 6.7.5-arch1-1
[info] Hailo DFC Version: 3.29.0
[info] HailoRT Version: Not Installed
[info] PCIe: No Hailo PCIe device was found
[info] Running `hailo parser onnx --hw-arch hailo8l convnext_tiny.onnx`
[info] Translation started on ONNX model convnext_tiny
[info] Restored ONNX model convnext_tiny (completion time: 00:00:00.26)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.63)
[info] Simplified ONNX model for a parsing retry attempt (completion time: 00:00:04.57)
Traceback (most recent call last):
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 235, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 316, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 367, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/model_translator/translator.py", line 83, in convert_model
    self._create_layers()
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 39, in _create_layers
    self._update_vertices_info()
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 217, in _update_vertices_info
    node.update_output_format()
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 485, in update_output_format
    or self.is_null_transpose()
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 6134, in is_null_transpose
    if self.is_torch_tile_resize_nearest() or self.is_null_transpose_near_torch_tile():
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 4656, in is_null_transpose_near_torch_tile
    input_shape = self.get_input_shapes(convert_to_nhwc=False)[0]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/tools/parser_cli.py", line 198, in run
    runner = self._parse(net_name, args, tensor_shapes)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/tools/parser_cli.py", line 282, in _parse
    runner.translate_onnx_model(
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/runner/client_runner.py", line 1158, in translate_onnx_model
    parser.translate_onnx_model(
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 276, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 316, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 367, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/model_translator/translator.py", line 92, in convert_model
    self._calculate_shapes(validate_shapes=False)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 198, in _calculate_shapes
    self._layers_graph.calculate_shapes(meta_edges_graph=self._meta_graph, validate_shapes=validate_shapes)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_common/hailo_nn/hailo_nn.py", line 760, in calculate_shapes
    layer.update_output_shapes(hn_stage=self.net_params.stage, validate_shapes=validate_shapes)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_common/hailo_nn/hn_layers/layer.py", line 460, in update_output_shapes
    self.output_shapes = [output_shape[:] for _ in range(self.output_copies)]
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_common/hailo_nn/hn_layers/layer.py", line 489, in output_shapes
    self._append_output_shape(shape)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_common/hailo_nn/hn_layers/layer.py", line 463, in _append_output_shape
    self._check_valid_shape(output_shape)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_common/hailo_nn/hn_layers/layer.py", line 456, in _check_valid_shape
    raise UnsupportedModelError(f"Unexpected zero dimension in shape {shape} at {self.full_name_msg}")
hailo_sdk_common.hailo_nn.exceptions.UnsupportedModelError: Unexpected zero dimension in shape [-1, 0, 0, 192] at base conv layer base_conv2 (translated from /features/features.2/features.2.1/Conv)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/blawyer/hailo/venv/bin/hailo", line 8, in <module>
    sys.exit(main())
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/tools/cmd_utils/main.py", line 111, in main
    ret_val = client_command_runner.run()
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/tools/cmd_utils/base_utils.py", line 68, in run
    return self._run(argv)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/tools/cmd_utils/base_utils.py", line 89, in _run
    return args.func(args)
  File "/home/blawyer/hailo/venv/lib/python3.8/site-packages/hailo_sdk_client/tools/parser_cli.py", line 219, in run
    raise ParserCLIException(str(err).replace("net_input_format", "input_format")) from err
hailo_sdk_client.tools.parser_cli.ParserCLIException: Unexpected zero dimension in shape [-1, 0, 0, 192] at base conv layer base_conv2 (translated from /features/features.2/features.2.1/Conv)
4

It might be that you’ve reduced the input size of the model? This error that somewhere inside the model, after a few strides (spatial resolution reduction) the spatial dimension is now 0. This obvisouly not possible, and means that you need to either increase the input size, or reduce the amount of strides applied.