Parsing error on Hailo8 with newest Python Wheels

General Discussion
, , ,
1

Hi everyone, fore some months I’ve been trying to port some models to my Raspberry PI 5, but unfortunately the latest combination of python wheels does not seem to work at all.

I have no clue on what could be going wrong, so I will try to detail step by step what I’ve been doing.

This problem seems to be model invariant, so I have the same result no matter the ONNX file I use

Setup

Hardware

  • Raspberry PI 5
  • RPI AI Hat+ with Hailo8 accelerator

Software

  • Trixie PI OS on the RPI
  • Python 3.10
  • HailoRT 4.23.0
  • Dataflow Compiler 3.33.1

Example of network

As said above, this problem seems to happen with all network, but I will use a known one as an example.

  • I start with the Torchvision version of Efficientnet B0, with fixed input size of 1 x 3 x 224 x 224
  • Convert the model to ONNX with Torch Dynamo
  • Create a calibration dataset in .npy format of 1024 images with that size
  • Convert the ONNX into .har using the ClientRunner and specifying the architecture as Hailo8
    runner = ClientRunner(hw_arch='hailo8')
    hn, calibration_data = runner.translate_onnx_model(
    model=model_path, # path to your ONNX model
    net_name=new_name, # a name for the model (optional)
    start_node_names=["x"], # name of the input node in the ONNX graph (optional)
    #end_node_names=enn, # name of the output node (optional)
    net_input_shapes={"x": [1, 3, res[0], res[1]]}, # input shape [N,C,H,W],
    disable_shape_inference = True,
    disable_rt_metadata_extraction = False
    )
  • Convert from .har to .hef with quantization and optimization
    • Using this model script
      model_script = """
      # Calibration and optimization settings
      model_optimization_config(
      calibration,
      batch_size=1,
      calibset_size=1024
      )
      # Let each channel have its own scale/zero-point
      #model_optimization_config(globals, output_encoding_vector=enabled)
      # Need to disable muxing or it won't work
      #allocator_param(enable_muxer=False)
      model_optimization_flavor(optimization_level=2)
      # try to solve quantization problems pre_quantization_optimization(global_avgpool_reduction, layers=[avgpool1, avgpool2], division_factors=[2, 2])
      """
    • Instantiating the client runner
      runner = ClientRunner(har=model_path, hw_arch='hailo8')
    • loading the script and optimizing
      runner.load_model_script(model_script_modded) runner.optimize(cal_data)
    • Compiling the model
      hef_binary = runner.compile()
    • Finally, saving the .hef file
      with open(save_path, 'wb') as f: pickle.dump(hef_binary, f)

Up this step, everything executes successfully.

The error

When I try to execute or parse the model info, i get the following error:

[HailoRT] [error] Unsupported hef version 362283009
[HailoRT] [error] Failed parsing HEF file
[HailoRT] [error] Failed creating HEF
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_HEF_NOT_SUPPORTED(92)
[HailoRT CLI] [error] CHECK_SUCCESS failed with status=HAILO_HEF_NOT_SUPPORTED(92)

This error occurs both if I try to run it on the pc used to perform the steps above or if I try to run it on the RPI 5.

To reproduce the error above, I simply run:
hailortcli parse-hef ./hailo_efficientnetb0_224_224.hef, where the .hef is the output of the steps above.

I had the same exact problem with the 3.33 version and commented on here, however much time has passed and I was hoping the new version would solve this specific problem.

I have no clue if it’s a problem with version compatibility (which it should not, as they are the latest ones available for Hailo8) or if I’m doing something wrong during the process.

If anyone has some idea on how to solve this, any help is gladly accepted

2

Please try

hef = runner.compile()

with open(file_name, "wb") as f:
    f.write(hef)
3

That did the trick, thanks!

1 Like