Hailo10h PCIe link down

General Discussion
1

I am currently trying to test an LLM model by connecting a Hailo-10h module to the Novatek NT98336 board.
Previously, I had successfully used a Hailo-8 module on the same NT98336 board without any issues, but when I connected the Hailo-10 module to the same board, I observed that the PCI link disconnects after firmware loading.

Below is the kernel log after executing insmod hailo1x_pci.ko on the NT98336 board:

[   22.772418] hailo1x: Init module. driver version 5.1.1
[   22.777964] hailo1x 0000:01:00.0: Probing on: 1e60:45c4...
[   22.783491] hailo1x 0000:01:00.0: Probing: Allocate memory for device extension, 11240
[   22.791422] hailo1x 0000:01:00.0: enabling device (0000 -> 0002)
[   22.797434] hailo1x 0000:01:00.0: Probing: Device enabled
[   22.803027] hailo1x 0000:01:00.0: Probing: mapped bar 0 - 000000007b2b7bde 16384
[   22.810465] hailo1x 0000:01:00.0: Probing: mapped bar 2 - 00000000242ab86d 4096
[   22.817806] hailo1x 0000:01:00.0: Probing: mapped bar 4 - 000000000db228cf 16384
[   22.825198] hailo1x 0000:01:00.0: Probing: Setting max_desc_page_size to 4096, (PAGE_SIZE=4096)
[   22.833893] hailo1x 0000:01:00.0: Probing: Enabled 64 bit dma
[   22.839635] hailo1x 0000:01:00.0: Probing: Using specialized dma_ops=arm64_swiotlb_dma_ops
[   22.847892] hailo1x 0000:01:00.0: Disabling ASPM L0s
[   22.852943] hailo1x 0000:01:00.0: Manually disabled ASPM L0s
[   22.859056] hailo1x 0000:01:00.0: Writing file hailo/hailo10h/customer_certificate.bin
[   22.870300] hailo1x 0000:01:00.0: File hailo/hailo10h/customer_certificate.bin written successfully
[   22.879347] hailo1x 0000:01:00.0: Writing file hailo/hailo10h/scu_fw.bin
[   22.952096] hailo1x 0000:01:00.0: File hailo/hailo10h/scu_fw.bin written successfully
[   23.021771] hailo1x 0000:01:00.0: Board SKU-ID is: 1
[   23.026736] hailo1x 0000:01:00.0: Writing file hailo/hailo10h/u-boot-1.dtb.signed
[   23.043340] hailo1x 0000:01:00.0: File hailo/hailo10h/u-boot-1.dtb.signed written successfully
[   23.145430] hailo1x 0000:01:00.0: Reading firmware file hailo/hailo10h/u-boot-spl.bin
[   23.157839] hailo1x 0000:01:00.0: Reading firmware file hailo/hailo10h/u-boot-tfa.itb
[   23.165803] hailo1x 0000:01:00.0: Reading firmware file hailo/hailo10h/fitImage
[   23.665176] hailo1x 0000:01:00.0: Reading firmware file hailo/hailo10h/image-fs
[   26.741025] hailo1x 0000:01:00.0: Firmware file programmed successfully
[   26.747675] hailo1x 0000:01:00.0: Firmware file index 0 programmed successfully
[   26.764240] hailo1x 0000:01:00.0: Firmware file programmed successfully
[   26.770843] hailo1x 0000:01:00.0: Firmware file index 2 programmed successfully
[   26.834330] hailo1x 0000:01:00.0: Firmware file programmed successfully
[   26.840935] hailo1x 0000:01:00.0: Firmware file index 3 programmed successfully
[   26.848245] hailo1x 0000:01:00.0: Firmware batch programming completed for stage 2
[   26.972129] hailo1x 0000:01:00.0: vDMA transfer completed, triggering boot
[   27.527638] hailo1x 0000:01:00.0: SOC Firmware Batch loaded successfully
[   27.534341] hailo1x 0000:01:00.0: Firmware loaded in 4675 ms
[   27.578187] PCIE TOP ISR: link down
[   27.578572] hailo1x 0000:01:00.0: Probing: Added board 1e60-45c4, /dev/hailo0
[   27.700622] wait_speed_change: elps 27946167 ns, resolution 83 ns (start 7617553424160, end 7617581370327)
[   27.710344] wait_speed_change: link sts 0xa0130000

I don’t know what differences between Hailo-8 and Hailo-10 could cause this issue. Are there any specific things I should check?

2

Ah, when I enabled the no_power_mode option as Y while loading the hailo driver, the issue was resolved.

3

Hey @jspark,

Good to see you back!

You’re right – I saw you already used the no_power_mode flag, and I was going to suggest exactly that. What’s happening is the device drops into a low-power PCIe state after firmware loads, which causes the link to drop. The no_power_mode parameter prevents that transition and keeps everything stable.

Hope that clears things up! Let me know if you need anything else.