Hailo-rpi5 scrfd_10g model error

I am trying to use scrfd_10g to do face detection test on rpi5+hailo8. The python file I created is based on haio-rpi5-examples.

I have changed the post process file and function name correspondingly as the following:

        self.default_post_process_so = os.path.join(self.postprocess_dir, 'libscrfd_post.so')
        self.post_function_name = "scrfd_10g"



    def get_pipeline_string(self):
        source_pipeline = SOURCE_PIPELINE(self.video_source)
        # detection_pipeline = DETECTION_PIPELINE(hef_path=self.hef_path, batch_size=self.batch_size, labels_json=self.labels_json, additional_params=self.thresholds_str)
        infer_pipeline = INFERENCE_PIPELINE(
            hef_path=self.hef_path,
            post_process_so=self.default_post_process_so,
            post_function_name=self.post_function_name
        )
        user_callback_pipeline = USER_CALLBACK_PIPELINE()
        display_pipeline = DISPLAY_PIPELINE(video_sink=self.video_sink, sync=self.sync, show_fps=self.show_fps)
        pipeline_string = (
            f'{source_pipeline} '
            f'{infer_pipeline} ! '
            f'{user_callback_pipeline} ! '
            f'{display_pipeline}'
        )
 
        print(pipeline_string)
        return pipeline_string

But I get a weird error:

(venv_hailo_rpi5_examples) pi2@pi2:~/desui/hailo-rpi5-examples $ python basic_pipelines/face_detection.py --network face_detect --input rpi -f
libcamerasrc name=source ! video/x-raw, format=RGB, width=1536, height=864 !  queue name=source_scale_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0  ! videoscale name=source_videoscale n-threads=2 ! queue name=source_convert_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0  ! videoconvert n-threads=3 name=source_convert qos=false ! video/x-raw, format=RGB, pixel-aspect-ratio=1/1 !  queue name=inference_scale_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0  ! videoscale name=inference_videoscale n-threads=2 qos=false ! queue name=inference_convert_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0  ! video/x-raw, pixel-aspect-ratio=1/1 ! videoconvert name=inference_videoconvert n-threads=2 ! queue name=inference_hailonet_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0  ! hailonet name=inference_hailonet hef-path=/home/pi2/desui/hailo-rpi5-examples/basic_pipelines/../resources/scrfd_10g.hef batch-size=1  force-writable=true ! queue name=inference_hailofilter_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0  ! hailofilter name=inference_hailofilter so-path=/usr/lib/aarch64-linux-gnu/hailo/tappas/post_processes/libscrfd_post.so   function-name=scrfd_10g  qos=false  ! queue name=identity_callback_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0  ! identity name=identity_callback  ! queue name=hailo_display_hailooverlay_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0  ! hailooverlay name=hailo_display_hailooverlay ! queue name=hailo_display_videoconvert_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0  ! videoconvert name=hailo_display_videoconvert n-threads=2 qos=false ! queue name=hailo_display_q leaky=no max-size-buffers=3 max-size-bytes=0 max-size-time=0  ! fpsdisplaysink name=hailo_display video-sink=xvimagesink sync=false text-overlay=true signal-fps-measurements=true 
Showing FPS
[5:59:09.361815433] [443608]  INFO Camera camera_manager.cpp:325 libcamera v0.3.2+27-7330f29b
[5:59:09.370223494] [443623]  INFO RPI pisp.cpp:695 libpisp version v1.0.7 28196ed6edcf 29-08-2024 (16:33:32)
[5:59:09.380432402] [443623]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/ov5647@36 to CFE device /dev/media2 and ISP device /dev/media0 using PiSP variant BCM2712_C0
Config file doesn't exist, using default parameters
terminate called after throwing an instance of 'std::runtime_error'
  what():  Function name is not valid, should be lightface or retinaface
Aborted

I have checked the libscrfd_post.so, and confirm that this so file contains scrfd_10g function.

Why the script still suggest lightface or retinaface while I have choosed libscrfd_post.so as the postprocess file. Did I miss any steps?

regards.
Desuikong

Hello Desuikong,

Welcome to the Hailo Community!

Running a face detection pipeline on the Raspberry Pi5 + Hailo-8/8L is possible, but the postprocessing requires some modifications.

1. Solve the function name error:

This error is due to the function name not being recognized in the scrfd.cpp initialization method. You can either modify this check, or pass a configuration file to the hailofilter element, as it i done in the face_recognition pipeline in the Hailo SW Suite (suggested).

Using the config file should solve the error. However, depending on the Model Zoo version of the scrfd_10g model, you may experience a SEGFAULT. In that case, see solution below.

2. Adapt the postprocessing to the Hailo-8/8L scrfd_10g model:

When using the scrfd_10g model compiled for the Hailo-8L, the output node names may be different. Therefore, the postprocessing file may be adapted accordingly.

  1. Use hailortcli to identify the model’s outputs

    hailortcli parse scrfd_10g.hef
    
  2. Copy scrfd.cpp and scrfd.hpp to the hailo-rpi5-examples/cpp/ folder

  3. Modify the BOXES_10g, CLASSES10g and LANDMARKS_10g in scrfd.cpp, according to the output of hailortcli parse scrfd_10g.hef.
    For the precompiled Hailo-8L model downloaded from the Hailo Model Zoo, the outputs are:

    std::vector<std::string> BOXES_10g {"scrfd_10g/conv42",
                                      "scrfd_10g/conv50",
                                      "scrfd_10g/conv57"};
    std::vector<std::string> CLASSES_10g {"scrfd_10g/conv41",
                                        "scrfd_10g/conv49",
                                        "scrfd_10g/conv56"};
    std::vector<std::string> LANDMARKS_10g {"scrfd_10g/conv43",
                                          "scrfd_10g/conv51",
                                          "scrfd_10g/conv58"};
    
  4. Modify the meson.build to set up the compilation of the postprocessing. For example:

    scrfd_sources = [
    'scrfd.cpp'
    ]
    shared_library('scrfd_post_h8l',
        scrfd_sources,
        include_directories : rapidjson_inc,  
        dependencies : postprocess_dep,
        gnu_symbol_visibility : 'default',
        install: true,
        install_dir: join_paths(meson.project_source_root(), 'resources'),
    )
    
  5. Recompile the postprocessing files using the compile_postprocess.sh script.

  6. In hailofilter, adjust the path to the compiled .so library

2 Likes

Hi @pierrem ,

Thanks for your reply, it helps me a lot!

I have solved my problems. But I found some problems during this process.

Following your step one, I met SEGFAULT.

I download scrfd_10g hef model from your official web:
https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ModelZoo/Compiled/v2.13.0/hailo8/scrfd_10g.hef

but after parse the model, it shows:

(hailo_tappas_venv) pi2@pi2:~/desui/hailo-rpi5-examples/resources $ hailortcli parse-hef scrfd_10g.hef
Architecture HEF was compiled for: HAILO8
Network group name: scrfd_10g, Single Context
    Network name: scrfd_10g/scrfd_10g
        VStream infos:
            Input  scrfd_10g/input_layer1 UINT8, NHWC(640x640x3)
            Output scrfd_10g/conv42 UINT8, NHWC(80x80x8)
            Output scrfd_10g/conv41 UINT8, NHWC(80x80x2)
            Output scrfd_10g/conv43 UINT8, FCR(80x80x20)
            Output scrfd_10g/conv50 UINT8, NHWC(40x40x8)
            Output scrfd_10g/conv49 UINT8, NHWC(40x40x2)
            Output scrfd_10g/conv51 UINT8, FCR(40x40x20)
            Output scrfd_10g/conv57 UINT8, NHWC(20x20x8)
            Output scrfd_10g/conv56 UINT8, NHWC(20x20x2)
            Output scrfd_10g/conv58 UINT8, FCR(20x20x20)

The output nodes differ from the nodes in scrdf.cpp, and seems like that it’s of hailo 8l model.

By going your step 2, it works.

Model scrfd2.5g for hailo8 on your official web also has this problem.

this is my device detail:

(hailo_tappas_venv) pi2@pi2:~/desui/hailo-rpi5-examples/resources $ hailortcli fw-control identify
Executing on device: 0000:01:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 4.18.0 (release,app,extended context switch buffer)
Logger Version: 0
Board Name: Hailo-8
Device Architecture: HAILO8
Serial Number: HLLWM2B225100557
Part Number: HM218B1C2FAE
Product Name: HAILO-8 AI ACC M.2 M KEY MODULE EXT TEMP

Plz check it, thanks.

regards,
Desui

Hi @desuikong,

Thanks for confirming that the suggested solution works for you.
Regarding the output nodes, I figured out that the TAPPAS postprocessing are indeed based on the models used in the TAPPAS applications, which have different output names w.r.t. the models in the Model Zoo. For example, this is the output of the scrfd_10g model used in the TAPPAS demo.

Architecture HEF was compiled for: HAILO8
Network group name: scrfd_10g, Single Context
    Network name: scrfd_10g/scrfd_10g
        VStream infos:
            Input  scrfd_10g/input_layer1 UINT8, NHWC(640x640x3)
            Output scrfd_10g/conv48 UINT8, FCR(80x80x8)
            Output scrfd_10g/conv47 UINT8, FCR(80x80x2)
            Output scrfd_10g/conv49 UINT8, FCR(80x80x20)
            Output scrfd_10g/conv54 UINT8, FCR(40x40x8)
            Output scrfd_10g/conv53 UINT8, FCR(40x40x2)
            Output scrfd_10g/conv55 UINT8, FCR(40x40x20)
            Output scrfd_10g/conv57 UINT8, NHWC(20x20x8)
            Output scrfd_10g/conv56 UINT8, NHWC(20x20x2)
            Output scrfd_10g/conv58 UINT8, FCR(20x20x20)

Instead, you confirmed that you have the following output for both Hailo-8 and Hailo-8L models (downloaded from the Model Zoo):

Network group name: scrfd_10g, Single Context
    Network name: scrfd_10g/scrfd_10g
        VStream infos:
            Input  scrfd_10g/input_layer1 UINT8, NHWC(640x640x3)
            Output scrfd_10g/conv42 UINT8, NHWC(80x80x8)
            Output scrfd_10g/conv41 UINT8, NHWC(80x80x2)
            Output scrfd_10g/conv43 UINT8, FCR(80x80x20)
            Output scrfd_10g/conv50 UINT8, NHWC(40x40x8)
            Output scrfd_10g/conv49 UINT8, NHWC(40x40x2)
            Output scrfd_10g/conv51 UINT8, FCR(40x40x20)
            Output scrfd_10g/conv57 UINT8, NHWC(20x20x8)
            Output scrfd_10g/conv56 UINT8, NHWC(20x20x2)
            Output scrfd_10g/conv58 UINT8, FCR(20x20x20)

Therefore, the correct procedure is to modify and recompile the post-processing using the new output nodes - as you did - which is was I suggested in step 2).
I adjusted my previous post so that other users can benefit from this experience.