Hi,
I installed the Hailo drivers on my Raspberry Pi AI HAT+ running Raspberry Pi OS Lite using:
sudo apt-get install hailo-all
The device initially worked as expected. However, after some time, my AI pipeline stopped functioning. In the logs, I observed:
[HailoRT] [error] CHECK_GRPC_STATUS failed with error code: 4.
[HailoRT] [warning] Make sure HailoRT service is enabled and active!
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_RPC_FAILED(77)
CHECK_EXPECTED failed with status=77
Running hailortcli fw-control identify produced:
[HailoRT] [error] CHECK failed - Failed to open device file /dev/hailo with error 6
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_OPERATION_FAILED(36)
[HailoRT CLI] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_OPERATION_FAILED(36)
At this point, the system details were:
hailortcli scan
Hailo Devices:
[-] Device: 0001:01:00.0
uname -r
6.12.34+rpt-rpi-2712
hailortcli --version
HailoRT-CLI version 4.20.0
cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
Kernel logs from dmesg | grep -i hailo:
hailo_vdma_buffer_map+0x144/0x618 [hailo_pci]
hailo_vdma_buffer_map_ioctl+0xdc/0x358 [hailo_pci]
hailo_vdma_ioctl+0x1d4/0x268 [hailo_pci]
hailo_pcie_fops_unlockedioctl+0x178/0x7d8 [hailo_pci]
hailo 0001:01:00.0: Failed launch transfer -14
hailo 0001:01:00.0: Failed writing fw control to pcie
After reboot, everything worked, but the same issue reoccurred after a few seconds. Upgrading the system resolved the issue, and the device has been stable since.
System details after upgrade:
uname -r
6.12.47+rpt-rpi-2712
cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
I observed similar issues on my other devices too.(Raspberry Pi AI HAT+ and Seeed R2000) - where driver reinstallation was required to restore functionality.
What could be causing these intermittent Hailo driver failures, and are there recommended steps to prevent them from recurring?
Thank you in advance for any guidance!