I have a Solidrun Hummingboard 8P Edge AI with a Hailo 8 M.2.
I have been trying to follow the documentation for the Hailo Software Suite and in particular the Hailo Dataflow Compiler. I downloaded the Yolo8.pt file from Ultralytics and fine tunned it. I converted it to ONNX and then to the HAR format. I then quantized the weights in Python using the Hailo docker. I have also ran ran inference with the model and received results however I am not sure they are correct without post processing but I am not sure how to correctly post processes them. Can anyone point me in the right direction?
1 Like
Hey @steven.nix
Welcome to the Hailo Community!
You can take a look at the way we do post-process in these examples :
If you have any more question , Feel free to ask.
Best Regards
1 Like
@steven.nix The recommended way is to add the post process as part of the compiled model (HEF). This can be done by adding the nms_postprocess command to the model script, as it is done here.
Then you just need to correctly access the results, for which you can use as reference the object_detection examples in the links shared by @omria
1 Like
I think I finally found what you are talking about page 34 of the Hailo Dataflow Compiler Guide:
Adding Non-Maximum Suppression (NMS) Layer Through Model Script Commands
This block will add an NMS layer at the end of the network through the model script command:
nms_postprocess. The following arguments can be used to:
• Config json: an external json file that allows the changing of the NMS parameters (can be skipped for the default
configuration).
• Meta architecture: which meta architecture to use (for example, yolov5, ssd, etc). In this example, yolov5
will be used.
• Engine: defines the inference device for running the nms: nn_core, cpu or auto (this example shows
cpu
I’d like build yolov8m.
Than, I download yolov8m.onnx from hailo model zoo.
And I have no ideal to set the end_node_names.
The DFC suggest me set the Reshape node by “model.22/dfl/Reshape”.
However, when I do the quantization, it got error message as follows:
[info] Loading model script commands to yolov8m from string
[info] Starting Model Optimization
[warning] Reducing optimization level to 0 (the accuracy won’t be optimized and compression won’t be used) because there’s no available GPU
[warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results
[info] Model received quantization params from the hn
[info] Starting Mixed Precision
[info] Mixed Precision is done (completion time is 00:00:00.65)
[info] Layer Norm Decomposition skipped
[info] Starting Stats Collector
[info] Using dataset with 64 entries for calibration
Calibration: 100%|█████████████████████████████████████████████| 64/64 [00:49<00:00, 1.30entries/s]
[info] Stats Collector is done (completion time is 00:00:50.92)
[info] Starting Fix zp_comp Encoding
[info] Fix zp_comp Encoding is done (completion time is 00:00:00.00)
[info] matmul_equalization skipped
[info] Finetune encoding skipped
[info] Bias Correction skipped
[info] Adaround skipped
[info] Fine Tune skipped
[info] Layer Noise Analysis skipped
[warning] The expected calibration set should be [0, 255] when using an in-net normalization layer, but the range received is [(-30.255793, 290.42224), (-30.255793, 290.42224), (-30.255793, 290.42224)].
UnsupportedModelError Traceback (most recent call last)
Cell In[11], line 7
4 runner.load_model_script(alls)
6 # Call Optimize to perform the optimization process
----> 7 runner.optimize(calib_dataset)
9 # Save the result state to a Quantized HAR file
10 quantized_model_har_path = f"{model_name}_quantized_model.har"
File ~/test001/lib/python3.8/site-packages/hailo_sdk_common/states/states.py:16, in allowed_states..wrap..wrapped_func(self, *args, **kwargs)
12 if self._state not in states:
13 raise InvalidStateException(
14 f"The execution of {func.name} is not available under the state: {self._state.value}",
15 )
—> 16 return func(self, *args, **kwargs)
File ~/test001/lib/python3.8/site-packages/hailo_sdk_client/runner/client_runner.py:2093, in ClientRunner.optimize(self, calib_data, data_type, work_dir)
2061 @allowed_states(States.HAILO_MODEL, States.FP_OPTIMIZED_MODEL)
2062 def optimize(self, calib_data, data_type=CalibrationDataType.auto, work_dir=None):
2063 “”"
2064 Apply optimizations to the model:
2065
(…)
2091
2092 “”"
→ 2093 self._optimize(calib_data, data_type=data_type, work_dir=work_dir)
File ~/test001/lib/python3.8/site-packages/hailo_sdk_common/states/states.py:16, in allowed_states..wrap..wrapped_func(self, *args, **kwargs)
12 if self._state not in states:
13 raise InvalidStateException(
14 f"The execution of {func.name} is not available under the state: {self._state.value}",
15 )
—> 16 return func(self, *args, **kwargs)
File ~/test001/lib/python3.8/site-packages/hailo_sdk_client/runner/client_runner.py:1935, in ClientRunner._optimize(self, calib_data, data_type, work_dir)
1933 if self._state == States.HAILO_MODEL:
1934 self._optimize_full_precision(calib_data=calib_data, data_type=data_type)
→ 1935 self._sdk_backend.full_quantization(calib_data, data_type=data_type, work_dir=work_dir)
1936 self._state = States.QUANTIZED_MODEL
File ~/test001/lib/python3.8/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py:1045, in SDKBackendQuantization.full_quantization(self, data, data_type, work_dir, force_results_by_layer)
1043 self.pre_quantization_structural()
1044 params_pre_quantization = copy.deepcopy(self.get_params_pre_quantization())
→ 1045 self._full_acceleras_run(self.calibration_data, data_type)
1046 self._logger.verbose(“Core and post Quantization is done with Acceleras”)
1047 self._finalize_quantization()
File ~/test001/lib/python3.8/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py:1239, in SDKBackendQuantization._full_acceleras_run(self, data, data_type)
1237 hn = optimization_flow.get_hn()
1238 jsonschema.validate(hn, HNImporter()._load_schema())
→ 1239 self._model = HailoNN.from_hn(json.dumps(hn))
1240 return params_exported
File ~/test001/lib/python3.8/site-packages/hailo_sdk_common/hailo_nn/hailo_nn.py:1594, in HailoNN.from_hn(hn_json)
1591 @staticmethod
1592 def from_hn(hn_json):
1593 “”“Get Hailo model from HN raw JSON data.”“”
→ 1594 return HNImporter().from_hn(hn_json)
File ~/test001/lib/python3.8/site-packages/hailo_sdk_common/hailo_nn/hailo_nn.py:1847, in HNImporter.from_hn(self, hn_json)
1846 def from_hn(self, hn_json):
→ 1847 return self.from_parsed_hn(json.loads(ensure_str(hn_json)))
File ~/test001/lib/python3.8/site-packages/hailo_sdk_common/hailo_nn/hailo_nn.py:1835, in HNImporter.from_parsed_hn(self, hn_json, validate)
1833 self._update_input_indices()
1834 self._hailo_nn.update_output_indices()
→ 1835 self._hailo_nn.calculate_shapes()
1836 self._hailo_nn.validate_shapes()
1837 self._hailo_nn.update_output_layers_order()
File ~/test001/lib/python3.8/site-packages/hailo_sdk_common/hailo_nn/hailo_nn.py:755, in HailoNN.calculate_shapes(self, meta_edges_graph, validate_shapes)
752 else:
753 layer.output_copies = self.get_number_of_successors(layer)
→ 755 layer.update_output_shapes(hn_stage=self.net_params.stage, validate_shapes=validate_shapes)
File ~/test001/lib/python3.8/site-packages/hailo_sdk_common/hailo_nn/hn_layers/feature_splitter.py:117, in FeatureSplitterLayer.update_output_shapes(self, validate_shapes, **kwargs)
115 def update_output_shapes(self, validate_shapes=True, **kwargs):
116 if validate_shapes and not self._validate_output_shapes():
→ 117 raise UnsupportedModelError(
118 f"Unexpected split shapes at {self.full_name_msg}, "
119 f"output_shapes={self.output_shapes}, input_shapes={self.input_shapes})",
120 )
121 # Overrided because len(output_shapes)>1 but output_copies == 1
122 self.output_shapes = self._calc_output_shape()
UnsupportedModelError: Unexpected split shapes at feature splitter layer yolov8m/feature_splitter9 (translated from /model.22/dfl/Reshape), output_shapes=[[-1, 1, 8400, 64]], input_shapes=[[-1, 1, 8400, 144]])
If you want to use the pre-trained yolov8m for inference on a Hailo device, you can download the already compiled version under the “compiled” column available in the public models section of our ModelZoo.
Otherwise, if you wish to learn about the compilation process, we recommend that you go through the tutorials available with the command hailo tutorials.
Dear sir,
I want to learn about the compilation process.
Hence, I am trying to compile yolov8m.onnx according to hailo tutorials.
As my understanding, the nms should be separated on yolo model. I have tried yolov5m.onnx, and the compilation is fine. However, i have no ideal about yolov8m. I can’t find which node is correct to separate nms process. I tried different nodes to separate nms, but still got error at the quantization step. Do you have any documentation to teach us to set the end node for yolov8?
Hi @brad.pai,
Usually, the parser recommends the correct end nodes to be selected.
For models already part of our ModelZoo collection, the compilation process can use default configurations when using the hailomz CLI tool. This way the start and end nodes are automatically selected. You can read more about the ModelZoo installation here.
Specifically for yolov8m, it’s possible to see in the ModelZoo config file that the end-nodes to be selected are:
- /model.22/cv2.0/cv2.0.2/Conv
- /model.22/cv3.0/cv3.0.2/Conv
- /model.22/cv2.1/cv2.1.2/Conv
- /model.22/cv3.1/cv3.1.2/Conv
- /model.22/cv2.2/cv2.2.2/Conv
- /model.22/cv3.2/cv3.2.2/Conv
@brad.pai if you want to compile a model yourself similar to what I did, you will need to follow the Hailo Dataflow Compiler guide. You can actually grab their docker environment and when you run that it will have everything installed and ready to go including the python virtual environment you will need to use. After that just follow the necessary instruction to translate the onnx model over to the HAR file. When you do that conversion, it will give you an info message that should indicate which end nodes to use. That being said being familiar with the model architecture will help as the most likely end nodes will be the final nodes of the Head which are the nodes mentioned by @nina-vilela . The naming of the end nodes could vary depending on your model.