From Har fir to Quantized

General
1

Hello,
I am trying to compile custom b0_segformer.
I manage to create the har file but I am failing in the next step which is the creation of the quantized model:

I am ruuning this commad:
$hailo optimize mapcoresegformer_b0480_848_hailo_model_8.har --use-random-calib-set
…
…
…
File “/home/nuchailo1/anaconda3/envs/hcompiler2/lib/python3.10/site-packages/hailo_model_optimization/acceleras/atomic_ops/matmul_op.py”, line 141, in create_hw_params
raise AccelerasUnsupportedError(
hailo_model_optimization.acceleras.utils.acceleras_exceptions.AccelerasUnsupportedError: layer mapcoresegformer_b0480_848/matmul4 does not support shift delta. To overcome this issue you should force larger range at the inputs of the layer using command quantization_param([layer_name], force_range_in=[range_min, range_max]) current range of input 0 is (0.0005596954, 0.009956909) and input 1 is (-0.5742479, 0.79142684). You should increase the multiplication of these ranges by a factor of 6.4331885529243795

My question - how can I enforce larger range at the inputs of the layer using command quantization_pram as indicated in the error log???

Thanks in advanced
David

2

Hey @dudibs

I understand you’re encountering an error related to input ranges for the matmul4 layer during quantization. Don’t worry, this is a common issue that we can resolve by adjusting the input ranges. Let me walk you through the process:

First, the error message suggests increasing the multiplication factor for the input ranges. We’ll use the quantization_param() function to force a larger range at the input. Here’s how:

  1. Let’s look at the current input ranges from the error message:

    • Input 0: (0.0005596954, 0.009956909)
    • Input 1: (-0.5742479, 0.79142684)
    • Suggested multiplication factor: 6.4331885529243795

  2. We’ll apply this multiplication factor to expand the ranges. After calculation:

    • Input 0 becomes: (0.0036, 0.064)
    • Input 1 becomes: (-3.692, 5.09)

  3. Now, let’s use these new ranges in the quantization_param() function:

    quantization_param(['matmul4'], force_range_in=[[0.0036, 0.064], [-3.692, 5.09]])

Include this line in your optimization process before running the quantization again. This should resolve the error by providing sufficient input ranges for the matmul4 layer.

This should help resolve the error by expanding the input range as required for successful quantization. Let me know if you need further clarification!

Regards,
Omri

3

Thanks Omri.

I am using the CLI like that:

$hailo optimize <HAR_PATH> --use-random-calib-set

how I can include the line you mentioned in my optimization process before running the quantization again?

i can change it to some python file and call the ruuner but i cant understand how to introduce the line you mentioned in the process before running the optimize again.

4

Hi @dudibs,

If you’re using the CLI and want to include the quantization_param line, unfortunately, the CLI doesn’t provide an option to directly inject that command during optimization. To address this, you’ll need to switch to using a Python script to run the optimization process, as that will give you the flexibility to include the quantization_param function.

Here’s how you can do it:

Modify the Process in Python: You can create a simple Python script to load your HAR model, apply the quantization_param adjustment, and then proceed with optimization. Here’s an example:

from hailo_sdk_client import Client

# Load the model from the HAR file
har_path = '<HAR_PATH>'
client = Client()
net = client.load_model(har_path)

# Apply the quantization parameters
net.quantization_param(['matmul4'], force_range_in=[[0.0036, 0.064], [-3.692, 5.09]])

# Proceed with optimization
optimized_net = net.optimize(use_random_calib_set=True)

# Save the optimized model
optimized_net.save('<OUTPUT_PATH>')

This method allows you to insert the quantization_param modification, which isn’t possible directly in the CLI.

Let me know if you need further clarification on this approach!

Best Regards,
Omri

5

thanks a lot Omri.
My version of the DFC (3.28 and now 3.29) is not following the syntax you provided.

I manage to provide alls file with our custom params.

My script looks as follows and I am using the new 3.29 version of the DFC. also 4.19 pci driver and 4.19 hailort.

from hailo_sdk_client import ClientRunner

onnx_path = ‘b0_best_mIoU_iter_212000.opset15.sim.480_848.onnx’
onnx_model_name=onnx_path.replace(“.onnx”,“”)
start_node=[‘/backbone/layers.0.0/projection/Conv’]
end_node=[‘/backbone/layers.1.0/Transpose’]
runner = ClientRunner()
hn, npz = runner.translate_onnx_model(onnx_path, onnx_model_name,start_node_names=start_node,end_node_names=end_node)
hailo_model_har_name = onnx_model_name+“_.har”
runner.save_har(hailo_model_har_name)

#optimized
runner = ClientRunner(har=hailo_model_har_name)
runner.load_model_script(“mapcoresegformer_b0.alls”)
runner.optimize(calib_data=None)
runner.save_har(har_path=hailo_model_har_name+“_optimized.har”)

The DFC failing in the optimize step. There is unhand-led exception there. I can workaround it but then the mapping is failing in the compile step.

here is the error log from the optimize step:
…
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/optimal_zp_finder.py”, line 246, in find_the_best_zp
result = minimize(objective, x0, method=“SLSQP”, constraints=cons)
File “/home/nuchailo1/.local/lib/python3.10/site-packages/scipy/optimize/_minimize.py”, line 722, in minimize
res = _minimize_slsqp(fun, x0, args, jac, bounds,
File “/home/nuchailo1/.local/lib/python3.10/site-packages/scipy/optimize/_slsqp_py.py”, line 431, in _minimize_slsqp
slsqp(m, meq, x, xl, xu, fx, c, g, a, acc, majiter, mode, w, jw,
ValueError: failed to initialize intent(inout) array – expected elsize=8 but got 4

can we make some zoom meeting where I will share my screen?