DFC 3.29 - Failing on optimize

dudibs · October 8, 2024, 7:05am

Hello,

I installed the DFC 3.29 along with the pci driver 4.19 and the hailort 4.19.
I am trying to compile onnx model of b0-segformer.
I am creating the har file but failing on the optimize step:

here is the error log:

File “/home/nuchailo1/.local/lib/python3.10/site-packages/scipy/optimize/_minimize.py”, line 722, in minimize
res = _minimize_slsqp(fun, x0, args, jac, bounds,
File “/home/nuchailo1/.local/lib/python3.10/site-packages/scipy/optimize/_slsqp_py.py”, line 431, in _minimize_slsqp
slsqp(m, meq, x, xl, xu, fx, c, g, a, acc, majiter, mode, w, jw,
ValueError: failed to initialize intent(inout) array – expected elsize=8 but got 4

I can workaround it but then i am failing on the compile step.

any suggestions?

Thanks in advanced.

nina-vilela · October 8, 2024, 10:44am

Hi @dudibs,

what model script are you using?

dudibs · October 9, 2024, 9:32am

Do you mean which alls file? I am not using the one published in your website under the segformer compiled model.

nina-vilela · October 9, 2024, 10:48am

Yes, what we call model script is the file with the .alls suffix. Could you please share the commands in your model script?

nina-vilela · October 9, 2024, 11:31am

It’s important to add the normalization to the model script:

normalization199 = normalization([123.675, 116.28, 103.53], [58.395, 57.12, 57.375])

And which end-node names did you select? The one used with the pre-trained onnx should be “1438”

dudibs · October 9, 2024, 11:37am

putting these two lines cause

quantization_param([matmul4], force_range_in=[-3.692, 5.09])
normalization199 = normalization([123.675, 116.28, 103.53], [58.395, 57.12, 57.375])

raise NormalizationLayersAdditionException(
hailo_sdk_client.tools.normalization_layers_addition.NormalizationLayersAdditionException: b0_best_mIoU_iter_212000_opset15_sim_480_848/input_layer1 has 1 features and given 3 std values. They must be equal

putting the first line only works but i am failing in the minimize function …

regarding start node and end node I am not giving it anything…I can take a look on the model and choose the first and last node names and give it to the first har creation step.

nina-vilela · October 9, 2024, 11:44am

Could you open the model with netron.app and share a screenshot of the model’s input layer?

dudibs · October 9, 2024, 11:51am

nina-vilela · October 9, 2024, 11:57am

Did you re-train the model? What’s the reason for only one channel?

dudibs · October 9, 2024, 12:29pm

of course this is not your model. this is a model trained on our data. it accepts only one channel. assume inputs are infra red images.

nina-vilela · October 9, 2024, 12:39pm

So please update the normalization values in the command as it was used in your training. The first array is mean and the second std.

dudibs · October 9, 2024, 12:54pm

i understands. i will do so.

dudibs · October 13, 2024, 7:04am

Hello Nina,
I got the norm values of the channel.

Now, the DFC fails on Minimizes the norm of vector S subject to given constraints - specifically this line is in module optimal_zp_finder.py:

result = minimize(objective, x0, method=“SLSQP”, constraints=cons)

the result return None and then result.success fails alter on fails.

any idea?

Thanks
David

nina-vilela · October 13, 2024, 8:22am

Could you please share the new nms_postprocess command and the full error message?

dudibs · October 13, 2024, 8:49am

i don’t know what is the nms_postprocess command but here is the error message(note that it is including the har creation at the beginning):

/home/nuchailo1/anaconda3/envs/hcompiler29/bin/python /home/nuchailo1/PycharmProjects/hcompiler2/harcreation.py
[info] Found a ‘.’ character in net_name, which isn’t supported. New net_name is b0_best_mIoU_iter_212000_opset15_sim_480_848
[info] Translation started on ONNX model b0_best_mIoU_iter_212000_opset15_sim_480_848
[info] Restored ONNX model b0_best_mIoU_iter_212000_opset15_sim_480_848 (completion time: 00:00:00.06)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.21)
[info] Start nodes mapped from original model: ‘input’: ‘b0_best_mIoU_iter_212000_opset15_sim_480_848/input_layer1’.
[info] End nodes mapped from original model: ‘/ArgMax’.
[info] Translation completed on ONNX model b0_best_mIoU_iter_212000_opset15_sim_480_848 (completion time: 00:00:01.65)
[warning] hw_arch parameter not given, using the default hw_arch hailo8.
If another device is the target, please run again using one of hailo8, hailo8r, hailo8l, hailo15h, hailo15m, hailo15l, hailo10h
[info] Saved HAR to: /home/nuchailo1/PycharmProjects/hcompiler2/b0_best_mIoU_iter_212000.opset15.sim.480_8482_.har
[info] Loading model script commands to b0_best_mIoU_iter_212000_opset15_sim_480_848 from mapcoresegformer_b0.alls
[info] Starting Model Optimization
[warning] Reducing optimization level to 0 (the accuracy won’t be optimized and compression won’t be used) because there’s no available GPU
[warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results
[info] Model received quantization params from the hn
[info] Starting Mixed Precision
[info] Mixed Precision is done (completion time is 00:00:01.01)
[info] Starting LayerNorm Decomposition
[info] Using dataset with 64 entries for calibration
Calibration: 100%|██████████| 64/64 [01:56<00:00, 1.83s/entries]
[info] LayerNorm Decomposition is done (completion time is 00:02:10.00)
[info] Starting Statistics Collector
[info] Using dataset with 64 entries for calibration
Calibration: 100%|██████████| 64/64 [04:23<00:00, 4.11s/entries]
[info] Statistics Collector is done (completion time is 00:04:29.39)
[info] Starting Fix zp_comp Encoding
[info] Fix zp_comp Encoding is done (completion time is 00:00:00.00)
[info] Starting Matmul Equalization
Traceback (most recent call last):
File “/home/nuchailo1/PycharmProjects/hcompiler2/harcreation.py”, line 19, in
runner.optimize(calib_data=None)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2093, in optimize
self._optimize(calib_data, data_type=data_type, work_dir=work_dir)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 1935, in _optimize
self._sdk_backend.full_quantization(calib_data, data_type=data_type, work_dir=work_dir)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1045, in full_quantization
self._full_acceleras_run(self.calibration_data, data_type)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1229, in _full_acceleras_run
optimization_flow.run()
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 306, in wrapper
return func(self, *args, **kwargs)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 326, in run
step_func()
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 250, in wrapped
result = method(*args, **kwargs)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py”, line 124, in parent_wrapper
func(self, *args, **kwargs)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 345, in step1
self.core_quantization()
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 250, in wrapped
result = method(*args, **kwargs)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 402, in core_quantization
self._matmul_equalization()
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 250, in wrapped
result = method(*args, **kwargs)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 582, in _matmul_equalization
algo.run()
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/algorithms/optimization_algorithm.py”, line 50, in run
return super().run()
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/algorithms/algorithm_base.py”, line 150, in run
self._run_int()
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/matmul_equalization.py”, line 123, in _run_int
self.equalize_input_paths(normal_subflow, transpose_subflow, layer)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/matmul_equalization.py”, line 154, in equalize_input_paths
set_scales_forward(self._model, normal_flow, layer.groups, factors)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/matmul_equalization.py”, line 332, in set_scales_forward
scales, zp_points = get_scales_output(in_layer, groups, factors, transpose)
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/matmul_equalization.py”, line 359, in get_scales_output
scales, zp = find_the_best_zp(
File “/home/nuchailo1/anaconda3/envs/hcompiler29/lib/python3.10/site-packages/hailo_model_optimization/algorithms/matmul_equalization/optimal_zp_finder.py”, line 246, in find_the_best_zp
result = minimize(objective, x0, method=“SLSQP”, constraints=cons)
File “/home/nuchailo1/.local/lib/python3.10/site-packages/scipy/optimize/_minimize.py”, line 722, in minimize
res = _minimize_slsqp(fun, x0, args, jac, bounds,
File “/home/nuchailo1/.local/lib/python3.10/site-packages/scipy/optimize/_slsqp_py.py”, line 431, in _minimize_slsqp
slsqp(m, meq, x, xl, xu, fx, c, g, a, acc, majiter, mode, w, jw,
ValueError: failed to initialize intent(inout) array – expected elsize=8 but got 4

Process finished with exit code 1

nina-vilela · October 13, 2024, 2:24pm

My bad, I meant the normalization command.
Can you try again without the force range commands?

Omer · October 31, 2024, 9:22am

Hi @dudibs,
To solve the “ValueError: failed to initialize intent(inout) array – expected elsize=8 but got 4”, please downgrade the scipy version to 1.9.3 (pip install scipy==1.9.3)

Regards,

dudibs · November 3, 2024, 7:23am

thanks. i overcome it.

Topic		Replies	Views
Try to complie YOLOv11 with DFC 3.30 got ValueError: failed to initialize intent(inout) array -- expected elsize=8 but got 4 General dfc , error	6	68	January 29, 2025
Custom ONNX model optimization fails with Exception General dfc , hailo8 , error	12	425	September 10, 2024
Error converting segormer b0 onnx model using DFC General dfc	4	90	August 8, 2024
Error creating hef file after succeseful quantization- mapping fail General dfc , hailo8 , error	2	199	September 29, 2024
Compiling a .onnx to .hef with DFC v3.28.0 following official documentation General dfc , raspberry-pi	2	291	August 11, 2024

DFC 3.29 - Failing on optimize

Related topics