ONNX to HAR - lightglue model

General
, , ,
1

Hi Hailo Community,

I’m encountering an error while trying to convert the LightGlue ONNX model (Releases · fabio-sim/LightGlue-ONNX · GitHub) to a HAR file for the Hailo-8.

Goal: Convert lightglue.onnx to hef.

Environment:

• Hailo SDK Version: 4.20, DFC 3.31
• Target Hardware: Hailo-8
• Host OS: Linux BaseOP 5.15.167.4-microsoft-standard-WSL2
• Python Version: 3.10
• ONNX Model: LightGlue, tried these:

  1. https://github.com/fabio-sim/LightGlue-ONNX/releases/download/v0.1.0/superpoint_lightglue.onnx
  2. https://github.com/fabio-sim/LightGlue-ONNX/releases/download/v2.0/disk_lightglue_pipeline.trt.onnx
    • ONNX Opset Version: 16&17
    Conversion Script:

import traceback
import os # Import os module for path checking
from hailo_sdk_client import ClientRunner
 
hw_arch = 'hailo8'
 
onnx_model_path = "lightglue.onnx"
 
model_name = "lightglue"
 
har_path = f'{model_name}_{hw_arch}_model.har'
 
start_nodes = ["kpts0", "kpts1", "desc0", "desc1"]
 
end_nodes = ["matches0", "mscores0","matches1", "mscores1"]
representative_num_keypoints = 500 # 
net_input_shapes = {
    "kpts0": [1, representative_num_keypoints, 2],
    "kpts1": [1, representative_num_keypoints, 2],
    "desc0": [1, representative_num_keypoints, 256],
    "desc1": [1, representative_num_keypoints, 256]
}
print(f"--- Starting Hailo Conversion for {model_name} ---")
print(f"Target Arch:     {hw_arch}")
print(f"ONNX Model Path: {onnx_model_path}")
print(f"Output HAR Path: {har_path}")
print(f"Start Nodes:     {start_nodes}")
print(f"End Nodes:       {end_nodes}")
print(f"Input Shapes:    {net_input_shapes}")
print("-" * 30)
if not os.path.exists(onnx_model_path):
    print(f"\n--- ERROR ---")
    print(f"Input ONNX file not found at: {os.path.abspath(onnx_model_path)}")
    print("Please ensure the file exists and the path is correct.")
    print("-------------")
    exit() # Exit if file doesn't exist
 
try:
    print("Initializing ClientRunner...")
    runner = ClientRunner(hw_arch=hw_arch)
    print("ClientRunner initialized.")
 
    # Translate the ONNX model
    print("\nStarting ONNX model translation...")
    print("(This step can take several minutes depending on model complexity)")
    hn, npz = runner.translate_onnx_model(
        onnx_model_path,                  
        model_name
        start_node_names=start_nodes,     
        end_node_names=end_nodes,
        net_input_shapes=net_input_shapes
    )
    print("ONNX model translation successful.")
    print(f"  Network Group Handle (hn): {hn}")
    print(f"  Calibration Data (npz): {npz}")
 
    # Save the translated model to a HAR file
    print(f"\nSaving parsed model to HAR file: {har_path}...")
    runner.save_har(har_path)
    print(f"Model successfully saved to {har_path}")
 
    print("\n--- Conversion Process Completed Successfully ---")
 
except ImportError as e:
    print(f"\n--- ERROR ---")
    print(f"Hailo SDK client library or a dependency not found: {e}")
    print("Please ensure 'hailo_sdk_client' and its requirements are installed correctly in your Python environment.")
    print("-------------")
    traceback.print_exc()
 
except Exception as e:
    # Catch any other exceptions
    print(f"\n--- ERROR during conversion ---")
    print(f"An unexpected error occurred: {e}")
    print("\nFull Traceback:")
    traceback.print_exc()
    print("---------------------------------")

Problem:

The translate_onnx_model function fails during the ONNX parsing stage with the following error:
IndexError: list index (0) out of range
Full Traceback:

Traceback (most recent call last):
  File "/home// lg/LightGlue-ONNX-2.0/weights/convert_2_har.py", line 64, in <module>
    hn, npz = runner.translate_onnx_model(
  File "/home/ /lg/LightGlue-ONNX-2.0/weights/myenv_py310/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  # ... (Include the rest of the traceback as you provided) ...
  File "/home/ /lg/LightGlue-ONNX-2.0/weights/myenv_py310/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 868, in get_kernel
    const_shape = pred._info.attribute[0].t.dims
IndexError: list index (0) out of range

Observations:

• The error occurs within the get_kernel function, called from is_matmul_layer, suggesting an issue processing the weights/kernel input for a MatMul operation.
• I inspected the lightglue.onnx model using Netron. The model has the specified inputs/outputs. The outputs have dynamic dimensions (e.g., int64[Wherematches0_dim_0,num_matches0]), indicating dynamic operations (Where, NonZero, etc.) internally.
• It seems the parser expects a constant weight tensor with specific attribute information (pred._info.attribute[0]), but this information is missing or structured differently for at least one MatMul node in this model.
Question:

Is this a known limitation, or is there a potential workaround ?
Any insights or suggestions would be greatly appreciated.

Thanks you!

2

Hey @bublinimuzini,

Is this a known limitation?

Yeah, it is.
The Hailo SDK mainly supports static models (think regular CNNs or typical classification/vision nets).
If your model uses dynamic shapes or if MatMul nodes use weights that aren’t constants (i.e., not initializers), you’ll usually hit issues during parsing.
This happens especially if:

  • Inputs aren’t set as constant initializers,
  • MatMul is used for something other than basic Dense layers,
  • The model has dynamic shapes or advanced indexing (like in LightGlue).

How to debug this

  1. Check the MatMul nodes in Netron:
    • Open your ONNX in Netron, look at all the MatMul nodes.
    • See if their inputs are constant (initializers) or coming from another op.
  2. Print the ONNX graph in Python:
    import onnx
model = onnx.load("lightglue.onnx")
for node in model.graph.node:
    print(node.op_type, node.input)
    • If MatMul’s second input isn’t an initializer, that’s your problem node.
  3. Try the ONNX simplifier:
    onnxsim lightglue.onnx lightglue_sim.onnx
    • Sometimes this will help convert some dynamic subgraphs to static ones.

Bottom line

  • This is a known parser limitation.
  • For Hailo , MatMul and Conv weights have to be constant initializers in ONNX.
  • If you need help patching the model or figuring out which nodes are causing trouble, let me know!