As I mentioned previously, I have a python app that I’m porting to the Hailo platform (using a Pi5 and the Hailo8l AI Hat). I’m using the HailoRT Async inference api. I know there is a yolo person/face detector available on the Hailo8l, because it’s part of one of the demo picam apps. It looks like that’s a gstreamer app though, and I don’t see the actual model file. Is that model file available anywhere for use without using the tappas/gstreamer pipeline model? Are examples of the pre-post processing available as well?
Alternatively I thought I would try converting a ready made yolo based face detector. I found one here and then set up an Ubuntu 20.04 box and installed the hailo dataflow compiler docker container. I ran an instance of the container, converted the pt file to an onnx with:
yolo export model=yolov8_facedetector.pt format=onnx opset=12
From there I’ve tried converting it to a hef. I’m confused on how to go about that. I use an AI assistant to help me walk through it but I’m pretty sure the AI wasn’t that familiar with Hailo stuff. It tried to get me to convert the onnx to a HAR file first. I think that was successful, as the output said:
hailomz parse yolov8n \ –ckpt yolov8_facedetector.onnx \ –hw-arch hailo8l
[info] No GPU chosen and no suitable GPU found, falling back to CPU.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1756343776.685614 69 cuda_dnn.cc:8310]
Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1756343776.863816 69 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
In file included from /local/workspace/hailo_virtualenv/lib/python3.10/site-packages/numpy/core/include/numpy/ndarraytypes.h:1929,
from /local/workspace/hailo_virtualenv/lib/python3.10/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /local/workspace/hailo_virtualenv/lib/python3.10/site-packages/numpy/core/include/numpy/arrayobject.h:5,
from /root/.pyxbld/temp.linux-x86_64-3.10/local/workspace/hailo_model_zoo/hailo_model_zoo/core/postprocessing/cython_utils/cython_nms.c:1142:
/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning:
#warning "Using deprecated NumPy API, disable it with " “#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION” [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with "
| ^~~~~~~
Start run for network yolov8n …
Initializing the runner…
[info] Translation started on ONNX model yolov8n
[info] Restored ONNX model yolov8n (completion time: 00:00:00.34)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.71)
[info] Simplified ONNX model for a parsing retry attempt (completion time: 00:00:01.46)
[info] According to recommendations, retrying parsing with end node names: [‘/model.22/Concat_3’].
[info] Translation started on ONNX model yolov8n
[info] Restored ONNX model yolov8n (completion time: 00:00:00.04)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.23)
[info] NMS structure of yolov8 (or equivalent architecture) was detected.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.22/cv2.0/cv2.0.2/Conv /model.22/cv3.0/cv3.0.2/Conv /model.22/cv2.1/cv2.1.2/Conv /model.22/cv3.1/cv3.1.2/Conv /model.22/cv2.2/cv2.2.2/Conv /model.22/cv3.2/cv3.2.2/Conv.
[info] Start nodes mapped from original model: ‘images’: ‘yolov8n/input_layer1’.
[info] End nodes mapped from original model: ‘/model.22/Concat_3’.
[info] Translation completed on ONNX model yolov8n (completion time: 00:00:01.25)
[info] Translation started on ONNX model yolov8n
[info] Restored ONNX model yolov8n (completion time: 00:00:00.04)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.30)
[info] NMS structure of yolov8 (or equivalent architecture) was detected.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.22/cv2.0/cv2.0.2/Conv /model.22/cv3.0/cv3.0.2/Conv /model.22/cv2.1/cv2.1.2/Conv /model.22/cv3.1/cv3.1.2/Conv /model.22/cv2.2/cv2.2.2/Conv /model.22/cv3.2/cv3.2.2/Conv.
[info] Start nodes mapped from original model: ‘images’: ‘yolov8n/input_layer1’.
[info] End nodes mapped from original model: ‘/model.22/cv2.0/cv2.0.2/Conv’, ‘/model.22/cv3.0/cv3.0.2/Conv’, ‘/model.22/cv2.1/cv2.1.2/Conv’, ‘/model.22/cv3.1/cv3.1.2/Conv’, ‘/model.22/cv2.2/cv2.2.2/Conv’, ‘/model.22/cv3.2/cv3.2.2/Conv’.
[info] Translation completed on ONNX model yolov8n (completion time: 00:00:01.53)
[info] Appending model script commands to yolov8n from string
[info] Added nms postprocess command to model script.
[info] Saved HAR to: /workspace/yolov8n.har
I then generated a calibration set for the conversion from my device and tries to compile the .har into a .hef and this is the output:
hailomz compile yolov8n --hw-arch hailo8l --har yolov8n.har --calib-path /workspace/calib_resized/calib_list.txt
\[info\] No GPU chosen and no suitable GPU found, falling back to CPU.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1756344732.162390 276 cuda_dnn.cc:8310\] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1756344732.168548 276 cuda_blas.cc:1418\] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
Start run for network yolov8n …
Initializing the hailo8l runner…
Preparing calibration data…
\[info\] Loading model script commands to yolov8n from /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov8n.alls
\[info\] Found model with 3 input channels, using real RGB images for calibration instead of sampling random data.
\[info\] Starting Model Optimization
\[warning\] Reducing optimization level to 0 (the accuracy won’t be optimized and compression won’t be used) because there’s less data than the recommended amount (1024), and there’s no available GPU
\[warning\] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results
\[info\] Model received quantization params from the hn
\[info\] MatmulDecompose skipped
\[info\] Starting Mixed Precision
\[info\] Model Optimization Algorithm Mixed Precision is done (completion time is 00:00:00.78)
\[info\] LayerNorm Decomposition skipped
\[info\] Starting Statistics Collector
\[info\] Using dataset with 64 entries for calibration Calibration: 0%| | 0/64 \[00:00<?, ?entries/s\]
Traceback (most recent call last):
File “/local/workspace/hailo_virtualenv/bin/hailomz”, line 33, in
sys.exit(load_entry_point(‘hailo-model-zoo’, ‘console_scripts’, ‘hailomz’)())
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main.py”, line 122, in main
run(args)
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main.py”, line 111, in run
return handlers[args.command](args)
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main_driver.py”, line 248, in compile
\_ensure_optimized(runner, logger, args, network_info)
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/main_driver.py”, line 91, in \_ensure_optimized
optimize_model(
File “/local/workspace/hailo_model_zoo/hailo_model_zoo/core/main_utils.py”, line 353, in optimize_model
runner.optimize(calib_feed_callback)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, \*args, \*\*kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2206, in optimize
result = self.\_optimize(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, \*args, \*\*kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 2025, in \_optimize
checkpoint_info = self.\_sdk_backend.full_quantization(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1196, in full_quantization
new_checkpoint_info = self.\_full_acceleras_run(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 1434, in \_full_acceleras_run
new_checkpoint_info = self.\_optimization_flow_runner(optimization_flow, checkpoint_info)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py”, line 2088, in \_optimization_flow_runner
optimization_flow.run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py”, line 239, in wrapper
return func(self, \*args, \*\*kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 357, in run
step_func()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py”, line 153, in parent_wrapper
func(self, \*args, \*\*kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 375, in step1
self.pre_quantization_optimization()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 426, in pre_quantization_optimization
self.\_collect_stats()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py”, line 503, in \_collect_stats
algo.run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/optimization_algorithm.py”, line 55, in run
return super().run()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/algorithm_base.py”, line 159, in run
self.\_run_int()
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/algorithms/stats_collection/stats_collection.py”, line 41, in \_run_int
stats_collector.collect_stats(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/acceleras/statistics/stats_collector.py”, line 122, in collect_stats
acceleras_model.predict_on_batch(preprocessed_data)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/keras/src/backend/tensorflow/trainer.py”, line 581, in predict_on_batch
batch_outputs = self.predict_function(\[(x,)\])
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/tensorflow/python/util/traceback_utils.py”, line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/keras/src/backend/tensorflow/trainer.py”, line 208, in one_step_on_data_distributed
outputs = self.distribute_strategy.run(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/keras/src/backend/tensorflow/trainer.py”, line 198, in one_step_on_data
return self.predict_step(data)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/keras/src/backend/tensorflow/trainer.py”, line 96, in predict_step
y_pred = self(x, training=False)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py”, line 122, in error_handler
raise e.with_traceback(filtered_tb) from None
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/acceleras/model/hailo_model/hailo_model.py”, line 1303, in call
output = self.\_call_layer(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/acceleras/model/hailo_model/hailo_model.py”, line 1431, in \_call_layer
outputs = acceleras_layer(inputs, training=training, encoding_tensors=encoding_tensors, \*\*kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/acceleras/hailo_layers/base_hailo_none_nn_core_layer.py”, line 45, in call
outputs = self.call_core(inputs, training, \*\*kwargs)
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/acceleras/hailo_layers/hailo_postprocess.py”, line 121, in call_core
return self.bbox_decoding_and_nms_call(
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/acceleras/hailo_layers/hailo_postprocess.py”, line 157, in bbox_decoding_and_nms_call
decoded_bboxes, detection_score = self.yolov8_decoding_call(inputs, offsets=\[0.5, 0.5\])
File “/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_model_optimization/acceleras/hailo_layers/hailo_postprocess.py”, line 375, in yolov8_decoding_call
decoded_bboxes = tf.expand_dims(decoded_bboxes, axis=2)
ValueError: Exception encountered when calling HailoPostprocess.call().
Tried to convert ‘input’ to a tensor and failed. Error: None values not supported.
Arguments received by HailoPostprocess.call():
• inputs=\[‘tf.Tensor(shape=(1, 80, 80, 64), dtype=float32)’, ‘tf.Tensor(shape=(1, 80, 80, 1), dtype=float32)’, ‘tf.Tensor(shape=(1, 40, 40, 64), dtype=float32)’, ‘tf.Tensor(shape=(1, 40, 40, 1), dtype=float32)’, ‘tf.Tensor(shape=(1, 20, 20, 64), dtype=float32)’, ‘tf.Tensor(shape=(1, 20, 20, 1), dtype=float32)’\]
• training=False
• kwargs={‘encoding_tensors’: ‘None’, ‘cache_config’: ‘None’}
Calibration: 0%| | 0/64 \[00:20<?, ?entries/s\]
This is where I’m stuck. I’ve tried some different conversions but I always end up with this error.
My calibration dataset has 736 images, but it looks like the hailo compiler is hardcoded to limit the dataset to 64 items without a GPU, and regardless of the dataset size I always end up with the input to the tensor being ‘None’.
Can someone please provide assistance either getting the yolo personface hef file for the 8l or converting the yolo face detector I found to hef format?
