Hailo.run input format

General
1

Hello, I have the following question. I’m performing inference on Arducam frames. The format of the frames is XBGR8888 in Picamera2, which corresponds to a 32-bit RGB format (although this may seem counterintuitive, Picamera2 uses this convention).

The issue is that I need to convert the frames to RGB before passing them to hailo.run(). If I instead set the format to RGB directly and pass the frames as-is, I encounter an error.

I’m working on a Raspberry Pi Compute Module 4 (CM4). Could you please help me understand why this is happening? I’d also like to know if it’s possible to avoid the conversion step in order to save memory.

What is the expected input format for the hailo.run() function?

Thank you


        picam2 = Picamera2()

        # Configure and start Picamera2.
        picam2.video_configuration.main.size = (video_w, video_h)
        picam2.video_configuration.main.format = "XBGR8888"  # this is RGB in Picamera2
        picam2.configure("video")

        # Generate control dictionary from yaml file
        camera_control_dict = generate_controls_from_yaml(
            args.config_file_path)
        picam2.set_controls(camera_control_dict)

        picam2.start()

        while True:
            frame = picam2.capture_array()

            # PREPROCESSING
            rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            resized_frame = cv2.resize(rgb, (model_w, model_h))

            # INFERENCE
            results = hailo.run(resized_frame)

            # Extract detections from the inference results
            detections = extract_detections(
                results, video_w, video_h, class_names, score_thresh
            )
2

Hey @dgarrido,

Just a heads up - hailo.run() is pretty picky about the input format. It wants a contiguous H×W×3 uint8 array in RGB order (that’s what matches the video/x-raw,format=RGB caps). If you throw anything else at it like XBGR8888 or BGR, it’ll fail during caps negotiation or the shape check.

The easiest fix is just doing that cv2.cvtColor on the host side like you mentioned. But if you’re really trying to avoid that extra copy, you could either bake the color swap directly into your HEF or use a minimal NumPy view on a BGR24 capture.

Worth checking the RPi docs & forum on this too, This is the Raspberry Pi API layer on top of ours, so there might be some quirks there.