Problem Description
While the Hailo Raspberry Pi 5 Examples repository contains example code for object detection in videos, adjusting that code to work with images turns out to be rather challenging as it is based on the GStreamer framework and would require to rewrite the pipeline.
Hailo Application Code Examples
An alternative is the Hailo Application Code Examples repository. It contains example code for object detection which takes an image as input, uses the Yolov7 model (by default) to detect objects in that image, annotates the image with bounding boxes for the detected objects, and writes the annotated image into an output directory. For each detected object, the model is expected to return a bounding box as coordinates of the lower left and upper right corner, a label (class), and a score how sure the model is that the object belongs to this class.
The problem is: the code is written for a Hailo8 device and doesn’t work out of the box with the Hailo8L accelerator for Raspberry Pi 5. Luckily, we can make it work with minimal changes.
Disclaimer
There is a more elegant solution than the one described below. If you adjust the code in object_detection.py (in process_output function) to run with the HailoRT version 4.18.0, you might not need to change anything in objection_detection_utils.py. In my case, I started debugging object_detection_utils.py and worked my way backwards leading to the changes described next.
Changes to object_detection.py
Select the right model
If you try to run object_detection.py with the Hailo AI Kit for Raspberry Pi 5, it will throw an error because the Yolov7, the default model used in that script, is built for Hailo8 architecture while the AI accelerator for Raspberry Pi has a Hailo8L architecture. You can see for yourself in the hailort.log log file which is automatically produced for every invocation of the AI accelerator.
So, to label the images using the Hailo8L accelerator for Raspberry Pi 5, you’ll need the yolov8s_h8l model. (the other model in the source code). You can download the model by checking out the Hailo Raspberry Pi 5 Examples repository and running the download_resources.sh script which will download the model into the resources directory.
You can then copy the yolov8s_h8l.hef file to the directory where your objeect_detection.py script is stored and adjust the parse_args function in object_detection.py like this:
def parse_args() -> argparse.Namespace:
-- snip --
parser.add_argument(
"-n", "--net",
help="Path for the network in HEF format.",
default="yolov8s_h8l.hef"
)
-- snip --
Remove the duplicate call to utils.extract_detections
The process_output function in object_detection.py apparently tries to differentiate between the version of the Hailo runtime (hailort). It looks like the runtime versions before 4.19.0 return the inference results (i.e., the object detection results) as an array of arrays. So, the outer array is stripped on lines 123-124:
-- snip --
# Deals with the expanded results from hailort versions < 4.19.0
if len(infer_results) == 1:
infer_results = infer_results[0]
-- snip --
However, the utils.extract_detections function is called twice: the first call is on line 120 before infer_results is adjusted, and the second call is on line 126 after infer_results is adjusted (if the Hailo runtime’s version is lower than 4.19.0). Because the Raspberry Pi 5 AI Kit uses Hailo runtime 4.18.0, the second call to utils.extract_detections makes the program crash since infer_results is passed in two different forms.
We can fix this by simply commenting the lines 123, 124 and 126:
def process_output(
-- snip --
processed_image, infer_results = result
detections = utils.extract_detections(infer_results)
# Deals with the expanded results from hailort versions < 4.19.0
#if len(infer_results) == 1:
# infer_results = infer_results[0]
#detections = utils.extract_detections(infer_results)
utils.visualize(
-- snip --
Changes to object_detection_utils.py
Because the raw detections from the model are passed to utils.extract_detections function defined in object_detect_utils.py as an array of arrays, we need to strip the outer array before entering the for det in detection loop. To do this, we can adjust extract_detections to look like this:
def extract_detections(self, input_data: list, threshold: float = 0.5) -> dict:
-- snip --
for detections in input_data:
if len(detections) == 0:
continue
for i, detection in enumerate(detections):
for det in detection:
bbox, score = det[:4], det[4]
if score >= threshold:
boxes.append(bbox)
scores.append(score)
classes.append(i)
num_detections += 1
-- snip --
With these changes, you can run the object_detection.py to detect objects in images using the Hailo AI Kit for Raspberry Pi 5.
