Hi,
as the title says, I am looking for a working example of a head-pose-estimation pipeline. It has to be a controlled environment with all parameters known, which should work on a Hailo 8.
What I’ve tried so far:
Since we are limited of what the DFC can compile. I chose this repo because it provides weights for MobileNetV2 and V3.
I’ve converted the model from .pt to .onnx and then compiled it with the DFC the .onnx to a .hef with: hailomz compile mobilenet_v2_1.4 --ckpt=mobilenetv2.onnx --hw-arch hailo8 --calib-path 300W_LP-Calibration_images
The corresponding .alls (mobilenet_v2_1.4) applies a normalization, which differs from the original and which I tried to compensate.
Within the code, I infer first with an Oak-D/ depthai pipeline. Because I haven’t figured out how to do inference with two models on the Hailo 8.
Whatever I try, I can’t make the head-pose model work (there is an output, but it could also be just quantization jitter).
Kind of solved, after trying with an Oak-D/depthai converted FP16 Model:
I found out that the headpose-NN worked better with a less shaky input. And from a newbie-in-CV-perspective, it somehow makes sense. The NN seems to fixate on features it recognizes in the Input image, and when those features wiggle around, it is difficult to keep track of the angles.
If that is so, then the use of quantized models doesn’t make sense in this kind of scenario.
I want to mark this thread as solved. @omria, you are as always very, very helpful. Especially the explanation of how to use two models at the same time. I will definitely try this out in other contexts. Regarding Head pose estimation, I have given up this topic. I’ve tested a few models (there is actually not much publicly available) and I am not convinced about their functionality yet. They kind of work, but honestly very unreliable. I hope a few more models will appear on the surface. I have actually no skills to plan and build a NN myself and therefore depend on other smart people. It is kind of strange how a few years ago those models were made available and in the last 1-2 years everyone is cooking their own proprietary soup.