Parse Whisper Model ONNX to HAR

Hi Community,

I was trying to parse the whisper base-encoder.onnx to create a har file, but i was encountering the error AttributeError: ‘NoneType’ object has no attribute ‘in_valid_subgraph’.

below is the code that i run, and the runtime logs, can someone help me or point me to the right direction to resolve this issue. TIA

from hailo_sdk_client import ClientRunner

chosen_hw_arch = “hailo8l”

onnx_model_name = “base-encoder”
onnx_path = “…/…/…/…/…/…/models-ai/sherpa-onnx-whisper-base/base-encoder.onnx”

runner = ClientRunner(hw_arch=chosen_hw_arch)
hn, npz = runner.translate_onnx_model(
onnx_path,
onnx_model_name,
start_node_names=[“mel”],
end_node_names=[“1500”],
net_input_shapes={“mel”: [1, 80, 512]},
)

[info] Translation started on ONNX model base-encoder
[info] Restored ONNX model base-encoder (completion time: 00:00:00.31)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.76)
[info] Simplified ONNX model for a parsing retry attempt (completion time: 00:00:09.74)


AttributeError Traceback (most recent call last)
File ~/hailodfc/cenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py:220, in Parser.translate_onnx_model(self, model, net_name, start_node_names, end_node_names, net_input_shapes, augmented_path, disable_shape_inference, disable_rt_metadata_extraction, net_input_format, **kwargs)
219 try:
→ 220 parsing_results = self._parse_onnx_model_to_hn(
221 onnx_model=onnx_model,
222 net_name=valid_net_name,
223 start_node_names=start_node_names,
224 end_node_names=end_node_names,
225 net_input_shapes=net_input_shapes,
226 disable_shape_inference=disable_shape_inference,
227 net_input_format=net_input_format,
228 )
230 except Exception as e:

File ~/hailodfc/cenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py:300, in Parser._parse_onnx_model_to_hn(self, onnx_model, net_name, start_node_names, end_node_names, net_input_shapes, disable_shape_inference, net_input_format, **kwargs)
298 self._logger.warning(f"ONNX shape inference failed: {e!s}")
→ 300 return self.parse_model_to_hn(
301 onnx_model,
302 None,
303 net_name,
304 start_node_names,
305 end_node_names,
306 nn_framework=NNFramework.ONNX,
307 output_shapes=output_shapes,
308 net_input_format=net_input_format,
309 **kwargs,
310 )

File ~/hailodfc/cenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py:340, in Parser.parse_model_to_hn(self, model, values, net_name, start_node_names, end_node_names, nn_framework, output_shapes, net_input_format, rename_layers_by_blocks)
339 elif nn_framework == NNFramework.ONNX:
→ 340 converter = ONNXConverter(
341 model=model,
342 values=values,
343 output_shapes=output_shapes,
344 start_node_names=start_node_names,
345 end_node_names=end_node_names,
346 net_input_format=net_input_format,
347 )
348 else:

File ~/hailodfc/cenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py:170, in ONNXConverter.init(self, model, values, output_shapes, start_node_names, end_node_names, net_input_format)
168 end_node_names = self._get_real_end_node_names(onnx_graph, net_output, end_node_names)
→ 170 super().init(
171 graph=ONNXGraph(
172 graph=onnx_graph,
173 values=values,
174 net_input=net_input,
175 tensor_shapes=tensor_shapes,
176 output_shapes=output_shapes,
177 opset_version=model.opset_import[0].version,
178 net_input_format=net_input_format,
179 ),
180 start_node_names=start_node_names,
181 end_node_names=end_node_names,
182 )
184 self._resize_layers_meta_vertices = {}

File ~/hailodfc/cenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py:27, in EdgeNNConverter.init(self, graph, start_node_names, end_node_names)
26 def init(self, graph, start_node_names=None, end_node_names=None):
—> 27 super().init(graph, start_node_names, end_node_names)
28 self._input_vertices = self._get_input_vertices()

File ~/hailodfc/cenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py:51, in HailoNNConverter.init(self, graph, start_node_names, end_node_names)
50 self._errors_dict = {}
—> 51 self._calculate_valid_subgraph_scope()
52 self._successful_end_nodes =

File ~/hailodfc/cenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py:388, in HailoNNConverter._calculate_valid_subgraph_scope(self)
387 current_vertex = preds_queue.pop()
→ 388 current_vertex.in_valid_subgraph = True
389 if self._start_node_names and current_vertex.name in self._start_node_names:

AttributeError: ‘NoneType’ object has no attribute ‘in_valid_subgraph’

Hey @markryancruz02,

Our ML team is currently working on creating a HEF and running Whisper on Hailo. This is a challenging task because Whisper is a large model, making it difficult to compile for the Hailo platform.

Regarding the error you’re encountering, the message AttributeError: 'NoneType' object has no attribute 'in_valid_subgraph' suggests that you’re attempting to access the in_valid_subgraph attribute of an object that is None. This usually happens when a method or function is expected to return an object but instead returns None, and then you try to access an attribute on that None value.

Hi omria,

Thank you for your response, may i also inquire if you already have a target date for the Whisper Hef models? or are you going to release a new version of DFC that will be able to parse ASR models? TIA

We don’t have a specific date set for the release, but we’ll make it available as soon as the ML team completes the conversion. We usually roll out new DFC versions quarterly. As for the question about support for parsing ASR models, I’ll look into it and get back to you with more information.

Thank you for reaching out!

Hi @omria,

appreciate the update, i’ll be looking forward for the completion of models of whisper.

Thank you very much for entertaining my inquiries.

1 Like