How would one do face recognition on Hailo8L?
I have been able to get Face Recognition work with retinaface model detecting bbox, confidence etc. However as I add another “hailonet” element in the gstreamer pipeline it fails. The reason been two of the models possibly cant coexist in the same pipeline on a single hailo8L chip. Any clue how to get a pipeline
You can run multiple models in a pipeline. HailoRT contains a scheduler that will switch the models.
There are a few applications in Hailo Tappas that use multiple models.
This is the one that comes closest to what you want to do.
GitHub - Tappas - Face Recognition
Note that the app uses some custom models HEF that have been compiled to Hailo-8 and therefore will not run on the Hailo-8L. They would need to be converted for Hailo-8L.
If you first want to test/benchmark two or more models you can use the HailoRT run2 command line tool. It will run the models without an app.
For instance this command will run one model at 30 fps and the other will run as fast as possible.
hailortcli run2 set-net model1.hef --framerate 30 set-net model2.hef
You can also use the HailoRT monitor in a second terminal to watch the models run.
hailortcli monitor
You need to set the environment variable HAILO_MONITOR in the first terminal for this to work.
export HAILO_MONITOR=1
hailortcli run2 set-net model1.hef --framerate 30 set-net model2.hef
With this you can then play with the different parameters (batch-size, framerate, scheduler-threshold …) to optimize the network switching.
Thank you… it works. One more question the Hailo8L arcface_mobilenet model that I downloaded has the output tensor as:
arcface_mobilefacenet_2022-09/fc1
while expected is
arcface_mobilefacenet/fc1"
Causing probably the following error in the pipeline
terminate called after throwing an instance of ‘std::invalid_argument’
what(): No tensor with name arcface_mobilenet_v1/fc1
Hi,
I’m working on this topic too.
Looking into face_recognition.sh inside the repository I just found all the networks compiled for Hailo 8l here but I missing also the libvms_face_align.so and libvms_croppers.so.
Can you put me on the right way to get them?
Thank you,
Mattia
They are part of Tappas. They can be in different places depending your system and how you installed Tappas.
Are you using Raspberry Pi OS on the Pi 5? Then you can find them under /usr/lib/aarch64-linux-gnu/…
Can you tell me what steps you took to get face recognition working please?
I wrote this up in here, Face Detection on RPI5 using Hailo8L | by Sanjoy | Aug, 2024 | Medium
Also got face match working by changing the sample code will publish soon as I find time.
You article is amazing, the details are outstanding. I will test it out shortly. Thanks for doing god’s work.
To make face recognition (matching) working, use the code here GitHub - sanjoyg/rpi_hailo8l_face_recognition: Face Recognition using RPI5 Hailo8L AI Accelerator KIT
can we run it on the rpi5 with hailo8l kit
Yes it is for rpi5 with ai kit.
@sanjoyg you know how to crop frame by using the bbox in detection.py of hailo-rpi5-examples
Hi,
starting from you shell script I created a python script…so thank you so much for your work and to sharing it with us!
Just a question, do you how to get back the name of the recognized
person from hailogallery to use it in a user callback?
Thank you
I don’t think this article is “face recognition” it is face detection.
It does both detection and recognition
I’m struggling to get this to work. If possible when you have the time can you check if your code is working and maybe improve the instructions? I was really excited about trying this out but keep getting errors along the way. Also are all the steps necessary if Firmware Version: 4.18.0 seems to be already there on new install? I’m trying once again from a clean raspberry pi os flash and will followup if successful.
What kind of errors are u getting? Can you paste the log?
well I had several errors in different places but now I’ve started over in case some other installation or configuration was causing the problems. loaded up a fresh image of pi os bookworm and then did regular installation and my firmware version was at 4.18.0 and was unsure then if it was necessary to do step 3? for step 4 it’s unclear of the directory structure since when i mkdir tappas && cd tappas and then git clone GitHub - hailo-ai/tappas: High-performance, optimized pre-trained template AI application pipelines for systems using Hailo devices am i not creating now a tappas/tappas/ folder? anyhow at the end of step 4 we run hailortcli fw-control identify and are checkin if we have 4.18.0? well I’ve got that from the start so is step 4 also unnecessary? so I’ve not done any of those steps and went right to running the pipeline and referenced the retinaface_mobilnet_v1 and a test video and i see this:
pi@raspberrypi3:~/rpicam-apps $ gst-launch-1.0
hailomuxer name=hmux filesrc location=input.mkv name=src_0 !
queue name=hailo_preprocess_q_0 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 !
decodebin !
videoconvert !
videoscale qos=false !
queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 !
hailonet hef-path=/home/pi/hailo-ai/models/retinaface_mobilenet_v1.hef !
queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 !
hailofilter so-path=/usr/lib/aarch64-linux-gnu/post_processes/libface_detection_post.so name=face_detection_hailofilter qos=false function_name=retinaface !
queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 !
hailooverlay name=hailo_overlay qos=false show-confidence=false line-thickness=5 font-thickness=2 !
videoconvert !
autovideosink
Setting pipeline to PAUSED …
Could not load lib /usr/lib/aarch64-linux-gnu/post_processes/libface_detection_post.so: cannot open shared object file: No such file or directory
Cannot load symbol: /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailotools.so: undefined symbol: retinaface
Caught SIGSEGV
#0 0x00007ffef34882a4 in __GI___wait4 (pid=11871, stat_loc=0x7ffffd80753c, options=0, usage=0x0) at …/sysdeps/unix/sysv/linux/wait4.c:30
#1 0x00007ffef36347d0 in g_on_error_stack_trace () at /lib/aarch64-linux-gnu/libglib-2.0.so.0
#2 0x00005555d914626c in ()
#3 0x00007ffef393c7b0 in ()
#4 _dl_close (_map=0x0) at ./elf/dl-close.c:755
#5 0x00007ffef34fe8bc in __GI__dl_catch_exception (exception=exception@entry=0x7ffffd808aa0, operate=0x7ffef3905d80 <_dl_close>, args=0x0) at ./elf/dl-error-skeleton.c:208
#6 0x00007ffef34fe980 in __GI__dl_catch_error (objname=0x7ffffd808b18, errstring=0x7ffffd808b20, mallocedp=0x7ffffd808b17, operate=, args=) at ./elf/dl-error-skeleton.c:227
#7 0x00007ffef344a880 in _dlerror_run (operate=, args=) at ./dlfcn/dlerror.c:138
#8 0x00007ffef344a5b0 in __dlclose (handle=) at ./dlfcn/dlclose.c:31
#9 0x00007ffef2c71f1c in gst_hailofilter_start(_GstBaseTransform*) () at /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailotools.so
#10 0x00007ffef2b4b97c in () at /lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#11 0x00007ffef2b4bbe4 in () at /lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#12 0x00007ffef38091f8 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#13 0x00007ffef3809918 in gst_pad_set_active () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#14 0x00007ffef37dc978 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#15 0x00007ffef37f50f8 in gst_iterator_fold () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#16 0x00007ffef37dd06c in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#17 0x00007ffef37df708 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#18 0x00007ffef37dfa94 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#19 0x00007ffef37e1fec in gst_element_change_state () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#20 0x00007ffef37e2688 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#21 0x00007ffef37ba0e4 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#22 0x00007ffef3811d14 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#23 0x00007ffef37e1fec in gst_element_change_state () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#24 0x00007ffef37e2028 in gst_element_change_state () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#25 0x00007ffef37e2688 in () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#26 0x00005555d9145d6c in ()
#27 0x00007ffef33f7740 in __libc_start_call_main (main=main@entry=0x5555d91438c0, argc=argc@entry=58, argv=argv@entry=0x7ffffd8097b8) at …/sysdeps/nptl/libc_start_call_main.h:58
#28 0x00007ffef33f7818 in __libc_start_main_impl (main=0x5555d91438c0, argc=58, argv=0x7ffffd8097b8, init=, fini=, rtld_fini=, stack_end=) at …/csu/libc-start.c:360
#29 0x00005555d9143930 in ()
Spinning. Please run ‘gdb gst-launch-1.0 11860’ to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.
it’s also unclear to me are we manually downloading from the developper zone or are we cloning from github the same thing?