In my setup I have a camera that publishes images to a topic, e.g. /camera/raw. Now I want to perform object detection on this image every time a new frame comes in. So to do this, in the callback of the Subscriber (to /camera/raw) I perform inference using my custom yolov8m model. However my current implementation only reaches ~30ms inference time per frame (640x640x3), so I suspect that my implementation is inefficient. I have done the following (based on this page):
In the init of my DetectionNode:
# Setup for Hailo
self.input_queue = queue.Queue()
self.output_queue = queue.Queue()
self.hailo_inference = HailoAsyncInference(HEF_MODEL_PATH, self.input_queue, self.output_queue, 1, send_original_frame=False)
And then in my callback:
cv_image = self.bridge.imgmsg_to_cv2(image_msg, desired_encoding='rgb8')
output_image = cv_image.copy()
preprocessed_image = preprocess(output_image, 640, 640)
preprocessed_image = np.expand_dims(preprocessed_image, 0)
self.input_queue.put(preprocessed_image)
self.input_queue.put(None) # Add sentinel value to signal end of input
self.hailo_inference.run()
results = self.output_queue.get()
Can someone help me speed this up? I want to keep it simple but efficient.