Custom post-process errors: invalid ELF header and undefined symbol

General
1

Hi,

I’m trying to modify the visual output of the instance segmentation template. I am also doing it directly in the pipeline as to now have to use the --use-frame arg, as this seems to cause great performance issues.

The goal is to have an output akin to an alpha mask (black for all un segmented parts of the image (the background), and white for all segmented parts). All other coloured masks and bounding boxes should be removed, hence why I am trying to alter the post-process. See below for an example:

people_mask_2
people_mask_23508×2480 90.9 KB

This would then be processed further for different visual outputs, but I am having trouble even getting to the alpha mask stage.

Am I correct in believing that to achieve this I would have to create my own post-processing file to replace the one used by default in the ‘instance_segmentation_pipeline.py’ file in the inference pipeline when the ‘get_pipeline_string()’ function is called.

I tried following the guide here` but have encountered the errors:

‘Cannot load lib (path to file)/postprocess_custom.py: invalid ELF header’

and

‘Cannot load symbol: /lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailotools.so: undefined symbol: run’

For context, the custom py post process file is stored in the same directory as the default one. The undefined symbol being flagged is the name as the function. The code that directs to this custom post process file is as follows:

post_process_so= os.path.join(self.current_path, ‘…/resources/postprocess_custom.py’),
post_function_name=‘run’,

Am I mistaken in where to attach this post processing file to the pipeline? Am I wrong to define the post_function_name as the function included in the file?

I am trying to avoid the --use-frame argument as mentioned earlier to prevent a second window being opened, and frame rate issues. Can I add post processing to another part of the pipeline? I’m open, and appreciative, to any suggestions at all!

Thanks in advance :slight_smile:

2

I’ve managed to get around the ELF header issue- I was misunderstanding the difference between a pythonfilter post process, and the c++ post processes.

I was wondering if anyone had any thoughts on processing the model segmentation outputs to create the black and white mask? can either be using a python filter post process or the c++ alternative, depends on the performance of the python one.

what do you think @omria ?

3

Hey @eep ,

Welcome to the Hailo Community!

Optimizing Hailo Instance Segmentation Post-Processing: C++ vs Python

When working with GStreamer in Hailo, it’s essential to understand the performance implications of implementing post-processing in C++ as a shared object (.so) versus using a Python script. Here’s a comparison:

C++ Post-Processing (Recommended)

  1. GStreamer is written in C and optimized for low-latency video processing, expecting post-processing elements to be compiled shared objects.
  2. C++ post-processing runs inline within GStreamer’s processing chain, allowing for near-zero overhead execution.
  3. Using a .so file ensures that GStreamer correctly recognizes and loads the post-processing function.
  4. C++ can directly access Hailo’s GStreamer buffers, eliminating the need for CPU-Hailo8 data transfer overhead.

To modify the instance segmentation pipeline in C++ and generate an alpha mask instead of bounding boxes and colored masks, follow these steps:

  1. Locate the post-processing code in cpp/yolov5seg.cpp.
  2. Modify the function that processes segmentation masks to generate a binary mask:
    void process_instance_segmentation(HailoROIPtr roi)
{
    // ...
    cv::Mat alpha_mask(mask_tensor->height(), mask_tensor->width(), CV_8UC1, cv::Scalar(0));
    for (int i = 0; i < mask_tensor->height(); ++i) {
        for (int j = 0; j < mask_tensor->width(); ++j) {
            float mask_value = mask_data[i * mask_tensor->width() + j];
            if (mask_value > 0.5) {
                alpha_mask.at<uchar>(i, j) = 255;
            }
        }
    }
    roi->add_object(std::make_shared<HailoConfClassMask>(alpha_mask.data, alpha_mask.rows, alpha_mask.cols, 0.3, 0));
}
  3. Disable bounding box rendering by commenting out the relevant line:
    // HailoDetection detected_instance(bbox, class_index, label, confidence);
  4. Recompile the modified post-processing using ./compile_postprocess.sh.

Python Post-Processing (Not Recommended)

  1. Python introduces an execution bottleneck due to its interpreted nature, causing FPS drops and increased latency.
  2. Python post-processing requires extracting each frame from the GStreamer pipeline, passing it to Python, and re-inserting it back into GStreamer, causing additional buffering and performance issues.
  3. If using Python-based post-processing, you often need to use the --use-frame flag, which spawns a separate processing window, causing a performance hit due to redundant image copies.
4

Thank you!

I was trying to avoid the Python Post Processing as much as possible due to the performance hit making it unfeasible for real-time. I had been slowly working through modifying the same C++ yolov5 segmentation post process mentioned but I am inexperienced.

I will try your solution later.

Again, Thank you!

1 Like