Hailo Emulation Failing

General
1

Has anyone been able to successfully run the emulator? I am getting a similar result to @user59 in this topic.

Basically, after build hailortcli with HAILO_BUILD_EMULATOR set to ON it fails with a HAILO_DRIVER_NOT_INSTALLED(64) error.

Steps to reproduce:

  1. On a clean ubuntu 22.04 system with no physical hailo card installed git clone the latest release: git clone --branch v4.21.0 https://github.com/hailo-ai/hailort.git (note the master branch seems to be broken - does not build)

  2. Run cmake with emulation set on: cmake -H. -Bbuild -DCMAKE_BUILD_TYPE=Release -DHAILO_BUILD_EMULATOR=ON -DHAILO_BUILD_EXAMPLES=ON

  3. Install hailortcli with: sudo cmake --build build --config release --target install

  4. Run hailortcli scan or execute any of the examples e.g. ./build/hailort/libhailort/examples/cpp/vstreams_example/cpp_vstreams_example

Result is:

[HailoRT] [error] Can't find hailort driver class. Can happen if the driver is not installed, if the kernel was updated or on some driver failure (then read driver dmesg log)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_NOT_INSTALLED(64) - Failed listing hailo devices
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_NOT_INSTALLED(64)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_NOT_INSTALLED(64)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_NOT_INSTALLED(64)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_DRIVER_NOT_INSTALLED(64)
Failed create vdevice, status = 64

Anyone manage to get either CPU or GPU emulation working?

We want to be able to run CI release tests for Hailo without our build systems needing to have a physical chip in them, which is not practical in some cases.

2

Also worth noting I tried to rebuild the hailo drivers from source with the “for internal use only” EMULATION flag set. This gets you a little bit further: instead of the driver error it now gets an HAILO_OUT_OF_PHYSICAL_DEVICES error …

3

I’m stuck on the same thing as you are. I didn’t try to enable the EMULATOR flag though. Looking at the source code it also doesn’t appear to do much besides changing some timeouts.

Are there any examples for how to emulate a device?

4

I thought it would be helpful to follow up on this post and described what I discovered regarding Hailo “emulation”.

What I was trying to achieve was to emulate the output of a Hailo processor running a HEF file using either a CPU or GPU on a system that does not have a physical Hailo processor. This is not possible.

What can be achieved through the Data Flow Compiler is to test a trained model at different stages of compilation using the Hailo Archive: HAR file. A HAR stores the model in 3 different states as it progresses from ONNX to the HEF: translation, optimization and then quantization. If you have the HAR then you can load it and run any one of the 3 models stages using the inference context and the three flags: SDK_NATIVE, SDK_FP_OPTIMIZED, SDK_QUANTIZED. There is a defined python interface to inference in those different modes from a HAR but there are no C++ samples. You cannot do this with a HEF as it only contains the final quantized format of the model.

Note that the HAILO_BUILD_EMULATOR flag in the HailoRT library is for internal use by the Hailo team: it allows them to build the runtime for testing with their internal FPGA. It is not related to HAR inference context described above.