How can I connect a Hailo-8/8L to my PC or laptop?

[klausk]

How can I connect a Hailo-8/8L to my PC or laptop?

The Hailo-8 is connected to a host via the PCIe interface. There are several options available to connect a Hailo-8 to a PC, IPC or laptop.

Hailo-8 M.2 and mPCIe modules

Many modern motherboards have M.2 connectors. The Hailo-8 M.2 modules are available with different keys. A+E (2 PCIe lanes), B+M (2 PCIe lanes) and M (4 PCIe lanes). When available make use of the build in M.2 heatsink otherwise attach your own heat solution.

Note: For tests of the Hailo-8L you can use a Hailo-8 and compile the networks with the architecture set to hailo8l.

Older systems or some embedded platforms may have a mini PCIe slot. There is a Hailo-8 mPCIe module available. It provides 13 TOPS and has a single PCIe lane.

PCIe adapters

If no M.2 slot is available a PCIe to M.2 adapter can be used with standard PCIe slots available in many desktop and IPC systems. The adapters are widely available e.g. on Amazon and cost around USD/EURO 10-20.

There are also adapters available for mPCIe.

Thunderbolt NVME adapter

Some laptops and few desktop motherboards support Thunderbolt. Thunderbolt uses the USB-C connector but combines PCIe and Display Port protocol. The connector is usually marked with a lightning bolt.

When a Thunderbolt port is available a Thunderbolt NVME adapter can be used. These are available at Amazon from different companies like Orico and Other World Computing (OWC). It is important to study the information carefully because there are many similar adapters for USB (will not work). Thunderbolt adapters are more expensive, usually above USD/EURO 100.

These adapters can be used with the Hailo-8 key M (recommended) or B+M module.

When the adapter comes with multiple cables you must use the one suitable for Thunderbolt. They are short and often thick because of the shielding required, have two USB-C type connectors and often a lightning bolt or 40Gbps sign. Using a USB type A port or adapter will not work. Also normal USB-C cables e.g. shipped with mobile phones do not work.

Here are posts about a few Thunderbolt SSD enclosures I have tested:

Orico M2V01-C4 M.2 SSD Enclosure for Hailo-8

Orico M234C3-U4 M.2 SSD Enclosure for Hailo-8

UGREEN USB4 M.2 NVME SSD Enclosure for Hailo-8

Installation

It is recommended to test your system before installing the HailoRT PCIe driver. PCIe capabilities of all devices are negotiated at boot without the need of custom drivers. You should be able to see the Hailo-8 when running lspci on Linux or in the Windows Device Manager.

Linux lspci command

04:00.0 Co-processor: Hailo Technologies Ltd. Hailo-8 AI Processor (rev 01)

or

04:00.0 Co-processor: Device 1e60:2864 (rev 01)

Windows Device Manager

[tltangliang]

USB4 is good for laptop

[tltangliang]

[klausk]

Hi @tltangliang could you be so kind and edit your post and add the vendor and name of your adapter and a source? That could be useful to other users. Thank you.

[tltangliang]

USB4 M.2 expansion board

This expansion board from WisdPi is based on ASMedia’s ASM2464PD design and is mainly used to convert USB4 or Thunderbolt 3/4 to standard PCIe/NVMe.

Requirement

Make sure your laptop or desktop motherboard supports the USB4 or Thunderbolt 3/4 protocol, which is usually indicated on the interface.

Installation

Since the expansion board provides a direct M.2 M-Key interface and is compatible with the 2280 size, we simply plug in the Hailo8 module and connect it to the computer with the supplied cable without any additional operations.

Linux lspci command

04:00.0 Co-processor: Hailo Technologies Ltd. Hailo-8 AI Processor (rev 01)

Link

WisdPi ASM2464 USB4 M.2 expansion board

[gregoire]

Realtek has a very popular chip included in many PCIe to USB adapters: RTL9210B (for 10Gbps). There is also a 20Gbps version.

I have tried one of these adapters with a Hailo 8L B+M module and unfortunately, the Linux PC doesn’t show anything in dmesg. Even without any compiled driver, I would have expected to get a message. I guess that those adapters can’t work with Hailo module.

According to the post and picture above, the WisdPi adapter seems to be working but it’s indeed a little bit more expensive than all those Chinese-based Realtek-based adapters.

[smolin.y]

I got [UGreen USB4 m.2 enclosure] (Amazon.com: UGREEN 40Gbps M.2 NVMe Enclosure with Cooling Fan, Anti-Drop SSD Enclosure for MacBook, Aluminum M.2 Enclosure Support M/B+M Key Size 2280/2260/2242/2230 Compatible with Thunderbolt 3/4/USB4/3.2/3.1 : Electronics) and it’s not working for B+M Hailo 8L. It’s ASM2464PD based and has the same specs as previously mentioned adapters, but no success.

Does someone tested other brands except Orico, which is hard to find now on Amazon. What brands could you suggest that works with B+M Hailo 8?

[klausk]

@smolin.y Welcome to the Hailo Community!

I tested the UGREEN enclosure and it works on my systems.

Have a look at the guide I created.

UGREEN USB4 M.2 NVME SSD Enclosure for Hailo-8

Are you certain your port supports Thunderbolt?

[smolin.y]

@klausk

tested the UGREEN enclosure and it works on my systems.

Have a look at the guide I created.

Thanks a lot for your answer! I have this exact enclosure from Ugreen. Checked on my laptop that definitely has thunderbolt 4 and double checked on my desktop PC. However I used B+M key, maybe this is the issue? M-key is working for you, but did you check B+M key?

[klausk]

@smolin.y

but did you check B+M key?

Yes, I just did. I did run a complete Hailo integration tool test and run a couple of TAPPAS demos.

One thing I just noticed when I switched the module: I added an insulating pad on the backside of the Hailo-8 M.2 module because there were a couple of components on the adapter in the same area. I wasn’t sure if the clearance was sufficient.

What OS are you using? I checked it on Ubuntu 20 and Windows 10.

What do you get when you run lspci?

Can you also try whether there is a difference when you boot with the adapter connected vs connecting it when the system is already running?

What is the LED doing on your adapter? Mine is blinking a few times when I connect it, then stays static ON and starts blinking when I run a model.

[smolin.y]

@klausk Thanks a lot for your detailed explanation. I did everything, I use Ubuntu 22. When I plugged into my PC directly via PCIe on Ubuntu 22 and see Hailo in lspci. However on my laptop via thunderbolt nothing. Maybe, it’s just not working properly and some hardware issue on my thunderbolt side. I will try to find another laptop/PC with working thunderbolt and check it.
Anyway, this seems encouraging that it should work, and at least you help me to isolate the issue. Thunderbolt connection doesn’t require any additional drivers, correct? and I should see co-processor Hailo in my lspci without anything?

[klausk]

@smolin.y

Thunderbolt connection doesn’t require any additional drivers, correct? and I should see co-processor Hailo in my lspci without anything?

Yes, you should see the Hailo device without installing the HailoRT PCIe driver when you run the lspci command.