I’m currently working with my Hailo device using a Raspberry Pi 5 AI Kit. I tried to run the Scrfd_10g model for the WiderFace task, which I found on the Hailo official website. I followed the instructions and used the code provided on the Hailo GitHub repository (detection.py) to test the model. However, I wasn’t able to get the model to work as expected.
The error message I received during execution is as follows:
NMS score threshold is set, but there is no NMS output in this model.
CHECK_SUCCESS failed with status=6
This is my first time working on an AI-related project, and I would greatly appreciate it if someone could share any solutions or code examples to help resolve this issue. Any assistance would be highly appreciated!
Resolving NMS Error in Scrfd_10g Model on Raspberry Pi 5 AI Kit
Error Description
“NMS score threshold is set, but there is no NMS output in this model”
This error indicates that the model doesn’t output data in a format directly supporting Non-Maximum Suppression (NMS), or the post-processing step applying NMS isn’t configured correctly.
Resolution Steps
1. Check Post-Processing Logic
Scrfd_10g may not include NMS in its default post-processing pipeline.
Verify if detection.py or libyolo_hailortpp_post.so supports NMS for this model.
2. Modify Post-Processing Code
If NMS isn’t handled, implement it manually:
def nms(boxes, scores, iou_threshold):
# Function to perform Non-Maximum Suppression
...
return selected_boxes
Apply this function to detection results before display or storage.
3. Consider Alternative Models
Switch to a model with built-in post-processing (e.g., SCRFD_500M) from the Hailo Model Zoo.
4. Adjust NMS Parameters
If supported, try adjusting nms-score-threshold and nms-iou-threshold values.
Conclusion
The issue likely stems from the post-processing step, particularly NMS. Modify the post-processing pipeline, use a different model with built-in NMS, or manually implement NMS as needed.
Thank you for the detailed response! I have reviewed your suggestions and tried to understand the implementation of Non-Maximum Suppression (NMS) as you mentioned. However, I’m still encountering issues, and I’m not sure if I’m using the correct implementation. Could you please provide a more detailed explanation or example of how to modify the detection.py file?
Additionally, I found that the detection.py file provided on the Hailo GitHub repository appears to be incorrect. I have not made any modifications to the original file, so it should match the version currently on GitHub.
Could you help point out the exact section in the detection.py file where the corrections should be made or where the NMS logic needs to be added? I want to ensure that I’m implementing the necessary changes in the right place to avoid further errors.
Lastly, I’m uncertain about how the iou_threshold parameter is defined in the scrfd_10g.hef file. Could you explain what the iou_threshold represents and how it should be set? Any information regarding its effect on the detection results would be very helpful.
Thanks for sharing the details! Based on your description, I’ve added Non-Maximum Suppression (NMS) to the detection.py file. This will help you eliminate overlapping detections, ensuring that only the best bounding box for each object is kept.
Key Modifications:
NMS Logic: I added a new function apply_nms() to handle the filtering of overlapping bounding boxes based on confidence scores and the Intersection over Union (IoU) threshold.
Incorporation into the Callback: The detections from roi.get_objects_typed() are now passed through the apply_nms() function to remove duplicates before being processed.
Code Snippet with Modifications:
Here is the part of the code that was changed or added:
# -----------------------------------------------------------------------------------------------
# Non-Maximum Suppression (NMS) function
# -----------------------------------------------------------------------------------------------
def apply_nms(detections, iou_threshold=0.4, score_threshold=0.5):
boxes = []
confidences = []
final_detections = []
# Extract bounding boxes and confidence scores
for detection in detections:
bbox = detection.get_bbox() # (x, y, w, h)
confidence = detection.get_confidence()
if confidence >= score_threshold:
boxes.append([int(bbox[0]), int(bbox[1]), int(bbox[2]), int(bbox[3])])
confidences.append(float(confidence))
# Apply OpenCV's NMS function
indices = cv2.dnn.NMSBoxes(boxes, confidences, score_threshold, iou_threshold)
# Filter detections based on NMS
for i in indices:
i = i[0]
final_detections.append(detections[i])
return final_detections
# -----------------------------------------------------------------------------------------------
# User-defined callback function - Modified Part
# -----------------------------------------------------------------------------------------------
def app_callback(pad, info, user_data):
# ... existing code ...
# Get the detections from the buffer
roi = hailo.get_roi_from_buffer(buffer)
detections = roi.get_objects_typed(hailo.HAILO_DETECTION)
# Apply NMS to the detections
filtered_detections = apply_nms(detections)
# Parse the filtered detections
detection_count = 0
for detection in filtered_detections:
label = detection.get_label()
bbox = detection.get_bbox()
confidence = detection.get_confidence()
if label == "person":
string_to_print += f"Detection: {label} {confidence:.2f}\n"
detection_count += 1
# ... existing code ...
Explanation:
apply_nms(): This function uses OpenCV’s NMS to suppress redundant bounding boxes. You can adjust the iou_threshold and score_threshold to fine-tune the filtering based on your requirements.
After filtering with NMS, only the best detections are processed and displayed on the frame.
Notes:
iou_threshold: This threshold controls how much overlap is allowed between boxes before they are considered duplicates. Lowering it will make the NMS more aggressive.
score_threshold: This sets a minimum confidence score for detections to be considered. Increase or decrease this based on how confident the model needs to be in its detections.
Let me know if this resolves the issue or if you need further clarification!
Thanks. Finally a useful documentation on how to add the NMS. Your colleagues even provided the hailo8l modified hef file, but how can it run with the raspi 5 examples?
Detection.py requires NMS post processing. It would be ideal if you guys could share a working solution which does not require insane coding skills. It should just work out of the box for raspberry pi Ai module using a custom model such as the ones from your model zoo which do not include NMS.
Sorry, I’ve been a bit busy lately, but I’ve already updated all the data on my Raspberry Pi and also updated the GitHub library. However, there is still no NMS (Non-Maximum Suppression) update.