Hailo-8 farm :-)

General
1

We are developing a carrier board with a NVIDIA jetson orin nano and 12 M2 pcie slots.
It’s normal use is to control 12 JNX400 boards with up to 48 NVIDIA jetsons in a 19" rack. :slight_smile:

We want to test 12 hailo-8 M.2 modules on this board.
4 key M slots are connected with 2 pcie lanes.
8 key M slots are connected with 1 pcie lane.

Is it possible to drive a hailo-8 M.2 module with only 1 lane ?
Where can i find pcie drivers running on jetson orin nano ( arm 64 , linux for tegra, near ubuntu 18.04 ).

Regards andi

2

Hey @auvidea.as,

Yes, you can run the Hailo 8 using only a single lane. The performance will be on par with Raspberry Pi connections, as they also use a single lane.

You can find all of our software versions at https://hailo.ai/developer-zone/software-downloads/. For older versions, please check the archive section on that page.

Feel free to reach out if you have any other questions!

Best Regards,
Omria

3

Thank you,
the hailo-pcie driver is running on this board:

38601-3_X410_Hailo_setup_iso
38601-3_X410_Hailo_setup_iso3844×3844 832 KB

The compute modul is a jetson xavier nx with ubuntu 18.04 and today i will try to run hailort on docker with ubuntu 22.04.
Hope it works.

Best regards, andi

4

Hi, hailortcli ist running but lspci -v tells:
0005:06:00.0 Co-processor: Device 1e60:2864 (rev 01)

  • Subsystem: Device 1e60:2864*
  • Flags: bus master, fast devsel, latency 0, IRQ 35*
  • Memory at 1c00000000 (64-bit, prefetchable) [size=16K]*
  • Memory at 1c00008000 (64-bit, prefetchable) [size=4K]*
  • Memory at 1c00004000 (64-bit, prefetchable) [size=16K]*
  • Capabilities: [80] Express Endpoint, MSI 00*
  • Capabilities: [e0] MSI: Enable- Count=1/1 Maskable- 64bit+*
  • Capabilities: [f8] Power Management version 3*
  • Capabilities: [100] Vendor Specific Information: ID=1556 Rev=1 Len=008 <?>*
  • Capabilities: [108] Latency Tolerance Reporting*
  • Capabilities: [110] L1 PM Substates*
  • Capabilities: [128] Alternative Routing-ID Interpretation (ARI)*
  • Capabilities: [200] Advanced Error Reporting*
  • Capabilities: [300] #19*
  • Kernel driver in use: hailo*
  • Kernel modules: hailo_pci*

and:
dmesg | grep hailo
[ 5.829245] hailo_pci: loading out-of-tree module taints kernel.
[ 5.831571] hailo: Init module. driver version 4.19.0
[ 5.831970] hailo 0005:06:00.0: Probing on: 1e60:2864…
[ 5.832083] hailo 0005:06:00.0: Probing: Allocate memory for device extension, 11632
[ 5.832399] hailo 0005:06:00.0: enabling device (0000 → 0002)
[ 5.832429] hailo 0005:06:00.0: Probing: Device enabled
[ 5.832683] hailo 0005:06:00.0: Probing: mapped bar 0 - 000000001eccc50d 16384
[ 5.832826] hailo 0005:06:00.0: Probing: mapped bar 2 - 0000000002333a39 4096
[ 5.832964] hailo 0005:06:00.0: Probing: mapped bar 4 - 000000003d955230 16384
[ 5.833124] hailo 0005:06:00.0: Probing: Setting max_desc_page_size to 4096, (page_size=4096)
[ 5.833277] hailo 0005:06:00.0: Probing: Enabled 64 bit dma
[ 5.833387] hailo 0005:06:00.0: Probing: Using specialized dma_ops=iommu_dma_ops
[ 5.834119] hailo 0005:06:00.0: Probing: Using userspace allocated vdma buffers
[ 5.834313] hailo 0005:06:00.0: Disabling ASPM L0s
[ 5.834630] hailo 0005:06:00.0: Successfully disabled ASPM L0s
[ 5.834959] hailo 0005:06:00.0: Writing file hailo/hailo8_fw.bin
[ 6.122420] hailo 0005:06:00.0: File hailo/hailo8_fw.bin written successfully
[ 6.122593] hailo 0005:06:00.0: Writing file hailo/hailo8_board_cfg.bin
[ 6.122763] Failed to write file hailo/hailo8_board_cfg.bin
[ 6.122869] hailo 0005:06:00.0: File hailo/hailo8_board_cfg.bin written successfully
[ 6.123003] hailo 0005:06:00.0: Writing file hailo/hailo8_fw_cfg.bin
[ 6.123143] Failed to write file hailo/hailo8_fw_cfg.bin
[ 6.123241] hailo 0005:06:00.0: File hailo/hailo8_fw_cfg.bin written successfully
[ 6.222090] hailo 0005:06:00.0: Firmware loaded successfully
[ 6.242764] hailo 0005:06:00.0: Probing: Added board 1e60-2864, /dev/hailo0

The Driver is not running. :frowning:

5

@auvidea.as The log file you sent seems ok.
The files hailo8_fw_cfg.bin and hailo8_board_cfg.bin are optional. The hailo8_fw.bin is mandatory and is has been written correctly.
How did you determine that the driver is not running?
Please try the following:

  • Scan for the device (same as lspci):

    hailortcli scan

  • Check if you can read the firmware configuration:

    hailortcli fw-control identify

  • Check if you can run inference:

    hailortcli run <HEF-PATH>

    You can download a precompiled HEF from the Hailo Model Zoo for this purpose.

If you do not have issues with these commands, it means you can successfully communicate with the device.

6

Hi, thanks for your post.
The hailo-pcie-driver is installed on l4t 18.04.
The hailort ist installed in docker image ( Ubuntu 22.04.5 LTS )
In the Dockerimage the device file /dev/hailo0 doesn’t exist.
I think that is the problem.
So your commands doesn’t work:
root@015c21ec22c1:/app# hailortcli scan
Hailo Devices:
[-] Device: 0005:06:00.0
root@015c21ec22c1:/app# hailortcli fw-control identify
[HailoRT] [error] CHECK failed - Failed to open device file /dev/hailo0 with error 2
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_FAIL(36)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_FAIL(36)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_FAIL(36)
[HailoRT CLI] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_FAIL(36)

Regards as