Xfeat onnx translate failed

General
1

the onnx use by the script: accelerated_features/export.py at main · acai66/accelerated_features · GitHub
and the batch size is modified to 1.

pytorch version is 2.1.0+cpu

the export command is :

python3 export.py --xfeat_only_model_dualscale --export_path xfeat_ds.onnx --opset 17

the error I got :

>>> hn, npz = runner.translate_onnx_model('xfeat_ds.onnx', 'xfeat', start_node_names=['images'], end_node_names=['mkpts', 'feats', 'sc'])
[info] Translation started on ONNX model xfeat
[info] Restored ONNX model xfeat (completion time: 00:00:00.03)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.11)
[info] Simplified ONNX model for a parsing retry attempt (completion time: 00:00:00.43)
Traceback (most recent call last):
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 179, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(onnx_model, valid_net_name, start_node_names,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 237, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(onnx_model, None, net_name, start_node_names, end_node_names,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 255, in parse_model_to_hn
    converter = ONNXConverter(model=model,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 77, in __init__
    super(ONNXConverter, self).__init__(
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 48, in __init__
    self._calculate_valid_subgraph_scope()
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 422, in _calculate_valid_subgraph_scope
    current_vertex.in_valid_subgraph = True
AttributeError: 'NoneType' object has no attribute 'in_valid_subgraph'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 876, in translate_onnx_model
    parser.translate_onnx_model(model=model, net_name=net_name, start_node_names=start_node_names,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 211, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(simplified_model, valid_net_name,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 237, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(onnx_model, None, net_name, start_node_names, end_node_names,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 255, in parse_model_to_hn
    converter = ONNXConverter(model=model,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 77, in __init__
    super(ONNXConverter, self).__init__(
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 48, in __init__
    self._calculate_valid_subgraph_scope()
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 422, in _calculate_valid_subgraph_scope
    current_vertex.in_valid_subgraph = True
AttributeError: 'NoneType' object has no attribute 'in_valid_subgraph'

when I use the basis model of xfeat:

python3 export.py --xfeat_only_model --export_path xfeat_ds.onnx --opset 17

I got

>>> hn, npz = runner.translate_onnx_model('xfeat_ds.onnx', 'xfeat', start_node_names=['images'], end_node_names=['feats', 'keypoints', 'heatmaps'])
[info] Translation started on ONNX model xfeat
[info] Restored ONNX model xfeat (completion time: 00:00:00.02)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.07)
[info] Simplified ONNX model for a parsing retry attempt (completion time: 00:00:00.21)
Traceback (most recent call last):
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 179, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(onnx_model, valid_net_name, start_node_names,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 237, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(onnx_model, None, net_name, start_node_names, end_node_names,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 263, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 76, in convert_model
    self._create_layers()
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 28, in _create_layers
    self._add_direct_layers()
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 110, in _add_direct_layers
    self._layer_callback_from_vertex(vertex)
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 192, in _layer_callback_from_vertex
    if vertex.op in OPTIONAL_NULL_OPS and vertex.is_null_operation() and not is_flattened_global_maxpool:
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 3852, in is_null_operation
    if self.op == 'Slice' and self.is_null_slice():
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 3868, in is_null_slice
    if slices_args['steps'] != 1:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 876, in translate_onnx_model
    parser.translate_onnx_model(model=model, net_name=net_name, start_node_names=start_node_names,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 211, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(simplified_model, valid_net_name,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 237, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(onnx_model, None, net_name, start_node_names, end_node_names,
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 263, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 76, in convert_model
    self._create_layers()
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 28, in _create_layers
    self._add_direct_layers()
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 110, in _add_direct_layers
    self._layer_callback_from_vertex(vertex)
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 192, in _layer_callback_from_vertex
    if vertex.op in OPTIONAL_NULL_OPS and vertex.is_null_operation() and not is_flattened_global_maxpool:
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 3852, in is_null_operation
    if self.op == 'Slice' and self.is_null_slice():
  File "/home/dx/Downloads/hailo_ai_sw_suite/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 3868, in is_null_slice
    if slices_args['steps'] != 1:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

same with opset to 13

1 Like
2

in the xfeat function

	def extractDense(self, x, top_k = 8_000):
		if top_k < 1:
			top_k = 100_000_000

		x, rh1, rw1 = self.preprocess_tensor(x)

		M1, K1, H1 = self.net(x)
		
		B, C, _H1, _W1 = M1.shape
		
		xy1 = (self.create_xy(_H1, _W1, M1.device) * 8).expand(B,-1,-1)

		M1 = M1.permute(0,2,3,1).flatten(1, 2) # B, H*W, C
		H1 = H1.permute(0,2,3,1).flatten(1) # B, H*W

		_, top_k = torch.topk(H1, k = min(H1.shape[1], top_k), dim=-1)

		feats = torch.gather( M1, 1, top_k[...,None].expand(-1, -1, 64))
		mkpts = torch.gather(xy1, 1, top_k[...,None].expand(-1, -1, 2))
		if torch.onnx.is_in_onnx_export():
			# Avoid warning of torch.tensor being treated as a constant when exporting to ONNX
			mkpts[..., 0] = mkpts[..., 0] * rw1
			mkpts[..., 1] = mkpts[..., 1] * rh1
		else:
			mkpts = mkpts * torch.tensor([rw1, rh1], device=mkpts.device).view(1,-1)

		return mkpts, feats

torch.topK and torch.gather is support by the dataflow-compile now?

4

Hi,
The error is due to unsupported operations in our SW, specifically in the Parser. Our R&D are currently working on adding support for it.

Regards,

5

ok, I read the documentation later and found out that operators such as topk nonezero are not currently supported.:grin:

1 Like