Hello,
Thanks for your answer !
I reinstalled hailo_model_zoo and, this time, I manage to generate a .hef file consistently by using hailomz compile --hw-arch hailo8 damoyolo_tinynasL25_S and hailomz compile --hw-arch hailo8 damoyolo_tinynasL35_M.
I assume my issue was coming from my calib_dataset, somehow causing all the crashes ? (Not sure).
However, after generating the .hef files, they consistently stay at 0 performance everywhere on eval and I don’t really understand why. I’m using all the base params from the .yaml config files, .alls for optimization, .onnx in the pretrained folders and the tfrecord files from the .hailomz models_files/coco/2023-08-03/coco_val2017.tfrecord & models_files/coco/2023-08-03/coco_calib2017.tfrecord for quantization and eval.
Here is my compilation for damoyolo_tinynasL25_S (same results on L35_M) :
(.venv_hailo) p20c02@p20c02:~/rendu/hailo_model_zoo$ hailomz compile --hw-arch hailo8 damoyolo_tinynasL25_S
<Hailo Model Zoo INFO> Start run for network damoyolo_tinynasL25_S ...
<Hailo Model Zoo INFO> Initializing the hailo8 runner...
------NetParser.run()--------
------NetParser.run()----1----
------NetParser.run()---2-----
------NetParser.run()---3-----
Translating onnx model
model path : /home/p20c02/.hailomz/data/models_files/ObjectDetection/Detection-COCO/yolo/damoyolo_tinynasL25_S/pretrained/2022-12-19/damoyolo_tinynasL25_S.onnx
self._start_nodes : None
self._end_nodes : ['Mul_239', 'Sigmoid_259', 'Mul_279', 'Sigmoid_299', 'Mul_319', 'Sigmoid_339']
tensor_shapes : None
[info] Translation started on ONNX model damoyolo_tinynasL25_S
[info] Restored ONNX model damoyolo_tinynasL25_S (completion time: 00:00:00.19)
/home/p20c02/.venv_hailo/lib/python3.10/site-packages/onnxsim/onnx_simplifier.py:73: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
sizes = (None, np.float32, np.uint8, np.int8, np.uint16, np.int16, np.int32, np.int64, str, np.bool,
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.68)
[info] Start nodes mapped from original model: 'images': 'damoyolo_tinynasL25_S/input_layer1'.
[info] End nodes mapped from original model: 'Mul_239', 'Sigmoid_259', 'Mul_279', 'Sigmoid_299', 'Mul_319', 'Sigmoid_339'.
[info] Translation completed on ONNX model damoyolo_tinynasL25_S (completion time: 00:00:02.03)
Done translating model
------NetParser.run()---4-----
------NetParser.run()---5-----
[info] Saved HAR to: /home/p20c02/rendu/hailo_model_zoo/damoyolo_tinynasL25_S.har
<Hailo Model Zoo INFO> Preparing calibration data...
[info] Loading model script commands to damoyolo_tinynasL25_S from /home/p20c02/rendu/hailo_model_zoo/hailo_model_zoo/cfg/alls/hailo8/base/damoyolo_tinynasL25_S.alls
[info] Starting Model Optimization
[warning] Reducing optimization level to 0 (the accuracy won't be optimized and compression won't be used) because there's no available GPU
[warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results
[warning] Reducing compression ratio to 0 because the number of parameters in the network is not large enough (16M and need at least 20M). Can be enforced using model_optimization_config(compression_params, auto_4bit_weights_ratio=0.200)
[info] Model received quantization params from the hn
[info] MatmulDecompose skipped
[info] Starting Mixed Precision
[info] Model Optimization Algorithm Mixed Precision is done (completion time is 00:00:00.36)
[info] LayerNorm Decomposition skipped
[info] Starting Statistics Collector
[info] Using dataset with 64 entries for calibration
Calibration: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [00:20<00:00, 3.11entries/s]
[info] Model Optimization Algorithm Statistics Collector is done (completion time is 00:00:21.51)
[info] Output layer damoyolo_tinynasL25_S/conv60 with sigmoid activation was detected. Forcing its output range to be [0, 1] (original range was [2.7490600587043446e-06, 0.9074400067329407]).
[info] Output layer damoyolo_tinynasL25_S/conv72 with sigmoid activation was detected. Forcing its output range to be [0, 1] (original range was [7.37098616809817e-07, 0.898878276348114]).
[info] Output layer damoyolo_tinynasL25_S/conv83 with sigmoid activation was detected. Forcing its output range to be [0, 1] (original range was [6.07195772772684e-07, 0.9397953152656555]).
[info] Starting Fix zp_comp Encoding
[info] Model Optimization Algorithm Fix zp_comp Encoding is done (completion time is 00:00:00.00)
[info] Matmul Equalization skipped
[info] Finetune encoding skipped
[info] Bias Correction skipped
[info] Adaround skipped
[info] Quantization-Aware Fine-Tuning skipped
[info] Layer Noise Analysis skipped
[info] Model Optimization is done
[info] Saved HAR to: /home/p20c02/rendu/hailo_model_zoo/damoyolo_tinynasL25_S.har
[info] Loading model script commands to damoyolo_tinynasL25_S from /home/p20c02/rendu/hailo_model_zoo/hailo_model_zoo/cfg/alls/hailo8/base/damoyolo_tinynasL25_S.alls
[info] To achieve optimal performance, set the compiler_optimization_level to "max" by adding performance_param(compiler_optimization_level=max) to the model script. Note that this may increase compilation time.
[info] Loading network parameters
[info] Starting Hailo allocation and compilation flow
[info] Running Auto-Merger
[info] Auto-Merger is done
[info] Running Auto-Merger
[info] Auto-Merger is done
[info] Using Single-context flow
[info] Resources optimization guidelines: Strategy -> GREEDY Objective -> MAX_FPS
[info] Resources optimization params: max_control_utilization=97,5%, max_compute_utilization=97,5%, max_compute_16bit_utilization=97,5%, max_memory_utilization (weights)=90%, max_input_aligner_utilization=97,5%, max_apu_utilization=97,5%
[info] Using Single-context flow
[info] Resources optimization guidelines: Strategy -> GREEDY Objective -> MAX_FPS
[info] Resources optimization params: max_control_utilization=97,5%, max_compute_utilization=97,5%, max_compute_16bit_utilization=97,5%, max_memory_utilization (weights)=90%, max_input_aligner_utilization=97,5%, max_apu_utilization=97,5%
Validating context_0 layer by layer (100%)
+ + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + + +
● Finished
[info] Solving the allocation (Mapping), time per context: 59m 59s
Context:0/0 Iteration 28: Trying parallel splits...
cluster_0 cluster_1 cluster_2 cluster_3 cluster_4 cluster_5 cluster_6 cluster_7 prepost
worker0
worker1
Context:0/0 Iteration 32: Trying parallel mapping...
cluster_0 cluster_1 cluster_2 cluster_3 cluster_4 cluster_5 cluster_6 cluster_7 prepost
worker0 X X * * * * * * V
worker1 X V * * * * * * V
worker2 X V * * * * * * V
worker3 V V V V V V V V V
Reverts on pre-mapping validation: 22
01:40 on split failed: 0
Reverts on cluster mapping: 1
Reverts on inter-cluster connectivity: 2
Reverts on pre-mapping validation: 25
Reverts on split failed: 0
[info] Iterations: 32
Reverts on cluster mapping: 4
Reverts on inter-cluster connectivity: 2
Reverts on pre-mapping validation: 25
Reverts on split failed: 0
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Cluster | Control Utilization | Compute Utilization | Memory Utilization |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | cluster_0 | 75% | 57,8% | 85,9% |
[info] | cluster_1 | 75% | 95,3% | 80,5% |
[info] | cluster_2 | 87,5% | 100% | 85,2% |
[info] | cluster_3 | 93,8% | 100% | 56,3% |
[info] | cluster_4 | 93,8% | 82,8% | 83,6% |
[info] | cluster_5 | 100% | 79,7% | 75% |
[info] | cluster_6 | 87,5% | 78,1% | 93% |
[info] | cluster_7 | 68,8% | 62,5% | 89,8% |
[info] +-----------+---------------------+---------------------+--------------------+
[info] | Total | 85,2% | 82% | 81,2% |
[info] +-----------+---------------------+---------------------+--------------------+
[info] Successful Mapping (allocation time: 3m 30s)
[info] Compiling context_0...
[info] Bandwidth of model inputs: 9.375 Mbps, outputs: 9.54895 Mbps (for a single frame)
[info] Bandwidth of DDR buffers: 0.0 Mbps (for a single frame)
[info] Bandwidth of inter context tensors: 0.0 Mbps (for a single frame)
[info] Building HEF...
[info] Successful Compilation (compilation time: 9s)
[info] Saved HAR to: /home/p20c02/rendu/hailo_model_zoo/damoyolo_tinynasL25_S.har
<Hailo Model Zoo INFO> HEF file written to damoyolo_tinynasL25_S.hef
Eval :
(.venv_hailo) p20c02@p20c02:~/rendu/hailo_model_zoo$ hailomz eval --hef damoyolo_tinynasL25_S.hef --target hardware damoyolo_tinynasL25_S
<Hailo Model Zoo INFO> Start run for network damoyolo_tinynasL25_S ...
<Hailo Model Zoo INFO> Initializing the runner...
<Hailo Model Zoo INFO> Chosen target is hardware
------NetParser.run()--------
------NetParser.run()----1----
------NetParser.run()---2-----
------NetParser.run()---3-----
Translating onnx model
model path : /home/p20c02/.hailomz/data/models_files/ObjectDetection/Detection-COCO/yolo/damoyolo_tinynasL25_S/pretrained/2022-12-19/damoyolo_tinynasL25_S.onnx
self._start_nodes : None
self._end_nodes : ['Mul_239', 'Sigmoid_259', 'Mul_279', 'Sigmoid_299', 'Mul_319', 'Sigmoid_339']
tensor_shapes : None
[info] Translation started on ONNX model damoyolo_tinynasL25_S
[info] Restored ONNX model damoyolo_tinynasL25_S (completion time: 00:00:00.18)
/home/p20c02/.venv_hailo/lib/python3.10/site-packages/onnxsim/onnx_simplifier.py:73: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
sizes = (None, np.float32, np.uint8, np.int8, np.uint16, np.int16, np.int32, np.int64, str, np.bool,
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.51)
[info] Start nodes mapped from original model: 'images': 'damoyolo_tinynasL25_S/input_layer1'.
[info] End nodes mapped from original model: 'Mul_239', 'Sigmoid_259', 'Mul_279', 'Sigmoid_299', 'Mul_319', 'Sigmoid_339'.
[info] Translation completed on ONNX model damoyolo_tinynasL25_S (completion time: 00:00:01.85)
Done translating model
------NetParser.run()---4-----
------NetParser.run()---5-----
[info] Saved HAR to: /home/p20c02/rendu/hailo_model_zoo/damoyolo_tinynasL25_S.har
<Hailo Model Zoo INFO> Preparing calibration data...
[info] Loading model script commands to damoyolo_tinynasL25_S from /home/p20c02/rendu/hailo_model_zoo/hailo_model_zoo/cfg/alls/hailo8/base/damoyolo_tinynasL25_S.alls
<Hailo Model Zoo INFO> Initializing the dataset ...
<Hailo Model Zoo INFO> Running inference...
Processed: 5000images [00:42, 118.08images/s]
creating index...
index created!
Loading and preparing results...
Converting ndarray to lists...
(407311, 7)
0/407311
DONE (t=1.56s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=4.33s).
Accumulating evaluation results...
DONE (t=0.81s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
<Hailo Model Zoo INFO> Done 5000 images AP=0.000 AP50=0.000
Do you have an idea on what I’m doing wrong ?
Cordially,