Hailo-10H SKU-ID 1 timeout at stage 2 firmware boot - u-boot never requests fitImage

General Discussion
1

Hi,

I have a Hailo-10H PCIe accelerator that fails to initialize on every boot. The driver loads and writes the first three firmware files successfully, but then times out waiting for the chip to request the next stage firmware file. The chip never requests fitImage after u-boot loads.

Hardware:

  • Device: Hailo-10H, PCI ID 1e60:45c4 rev 01, SKU-ID 1

  • Host: Dell Vostro 3590 laptop

  • PCIe link: Speed 8GT/s, Width x4 (confirmed healthy)

  • SlotPowerLimit: 25W

Software tested:

  • HailoRT driver: 5.1.1 and 5.3.0 (both fail identically)

  • Kernel: Ubuntu 24.04, tested on 6.14.0-27 and 6.14.0-37 (both fail identically)

dmesg output:

 hailo1x 0000:02:00.0: Writing file hailo/hailo10h/customer_certificate.bin
hailo1x 0000:02:00.0: File hailo/hailo10h/customer_certificate.bin written successfully
hailo1x 0000:02:00.0: Writing file hailo/hailo10h/scu_fw.bin
hailo1x 0000:02:00.0: File hailo/hailo10h/scu_fw.bin written successfully
hailo1x 0000:02:00.0: Board SKU-ID is: 1
hailo1x 0000:02:00.0: Writing file hailo/hailo10h/u-boot-1.dtb.signed
hailo1x 0000:02:00.0: File hailo/hailo10h/u-boot-1.dtb.signed written successfully
hailo1x 0000:02:00.0: Timeout waiting for firmware file
hailo1x 0000:02:00.0: Failed writing SOC firmware on stage 2
hailo1x 0000:02:00.0: SCU log could not be read from device
hailo1x 0000:02:00.0: Firmware load failed
hailo1x 0000:02:00.0: Failed activating board -110
hailo1x 0000:02:00.0: probe with driver hailo1x failed with error -110

What I have ruled out:

  • Not a kernel version issue (tested two kernels)

  • Not a driver version issue (tested 5.1.1 and 5.3.0)

  • Not missing firmware files (all files present in /lib/firmware/hailo/hailo10h/)

  • Not a PCIe link issue (8GT/s x4 confirmed)

  • Not a permissions or package installation issue (clean reinstall done multiple times)

  • pci_set_master() is present in the driver source

Additional observations:

  • The chip locks up permanently after the first failed probe and requires a full power cycle (not just reboot) to attempt again

  • SCU log is unreadable after failure

  • hailortcli scan returns Hailo devices not found

  • The exact same failure occurs whether probed at boot or manually via modprobe