Switching Between Two ROS 2 Nodes Using Hailo on RPi5 – Device Still Reported as Busy

General Discussion
1

I am working on a Raspberry Pi 5 with a Hailo accelerator and ROS 2.

I have two different ROS 2 nodes, each using a different neural network (different HEF files) running on Hailo.

My goal is to alternate between these two nodes, i.e.:

  1. Start node A (uses network A on Hailo)

  2. Stop node A

  3. Start node B (uses network B on Hailo)

However, even after stopping node A, when I start node B I get an error indicating that the Hailo device is already in use.

This happens even though node A is no longer active from the ROS point of view.

My questions are:

  • Is the Hailo device exclusive per process, and only released when the process fully exits?

  • Is it expected that stopping or deactivating a ROS 2 node does not release the Hailo device?

  • Is there a supported way to switch between multiple networks across different nodes, or is the recommended approach to use a single process/node that loads multiple HEFs and switches between them internally?

Any clarification on the expected behavior and best practices for this use case would be appreciated.

Thank you.

2

If you use Hailo-8 or Hailo-8L and your app uses multiple processes, the recommended approach is to use the Multi Process Service. This way, all your processes communicate with the service, and the service is the only process that communicates directly with the kernel driver. The code change in your app that is needed to work through the service is very small, and the app can stay mostly the same. Please see the HailoRT User Guide for more details.

In addition, there is no need to manually switch between models/HEFs. You can load all HEFs you need and use them concurrently. Under the hood, the scheduler, that is part of the HailoRT library, or part of the service in case of a multi process app, will take care of loading the right model every time.

3

Hi @Nir_Mualem ,

Following Eldad’s, maybe this will assist: hailo-apps/doc/user_guide/running_parallel.md at main · hailo-ai/hailo-apps · GitHub

Thanks,

4

I assume, you’re using a HAILO-8* device with your Pi5? The docs for the Multi-Process Service would be here: https://hailo.ai/developer-zone/documentation/hailort-v4-23-0/?sp_referrer=inference/inference.html#multi-process-service

I have not tried this myself. But I already found, that my packages have a mismatch:-( The systemd unit file expects the daemon in /usr/local/bin while it’s now installed in /usr/bin/:frowning: So, take all the docs with a grain of salt!

Good luck;-)

5

@marco This guide should work: hailo-apps/doc/user_guide/running_parallel.md at main · hailo-ai/hailo-apps · GitHub

6

Yes, I modified the hailo-rpi5-examples ./venv_hailo_rpi_examples/lib/python3.12/site-packages/hailo_apps/hailo_app_python/apps/pose_estimation/pose_estimation_pipeline.py ./venv_hailo_rpi_examples/lib/python3.12/site-packages/hailo_apps/hailo_app_python/apps/detection/detection_pipeline.py to include the
multi_process_service=‘true’ and two system processes can share one device. But they are both running way slower than alone! So, my guess is, that’s a workable solution when you only need ‘switchable’ models (one at a time as needed).

7

Hi @marco Please kindly note that our new recommended repo is GitHub - hailo-ai/hailo-apps - and it’s replacing the hailo-rpi5-examples.

8

OK, but it does not seem to make a big difference n this case.

9

Hi @marco , The benchmark is executing in parallel detection from our new repo, without ROS nodes etc. - Just the Python app from the repo on Pi 5: hailo-apps/hailo_apps/python/pipeline_apps/detection at main · hailo-ai/hailo-apps · GitHub
It suppose to run smoothly even if there are 2 apps running in parallel.

10

Running the same network on different video streams might be ok, but I tried to run different networks (detection example and pose estimation example) and at that point, the framerate is << 5 fps.

Two hailo-detect -f 10 --show-fps processes seem to be able to keep up at 10 fps.

11

Hi @marco ,

Below is from Pi with 8L detection (Yolo8m) and detection simple (Yolo6n) both running in parallel and FPS above > 30.

2
2871×661 130 KB

Can you please confirm if running in parallel from our new hailo-apps repo + following the running parallel guide there, resulting in your case low FPS?

Thanks,

12

OK, with

$ cp -p  /usr/local/hailo/resources/videos/example.mp4 /run/user/1000/
$ hailo-detect-simple -i /run/user/1000/example.mp4 -f 30 --show-fps --disable-sync >/
dev/null 2>&1 &
$ hailo-detect --input rpi -f 30 --show-fps --disable-sync >/dev/null 2>&1 &

can I get to about 24 fps for both.
That’s on a Pi5 with 8GB RAM, a HAILO-8 (AI+ hat) running Ubuntu 24.04 (6.8.0-1047-raspi).
The RT kernel alone does not improve things.

Thanks,
– Marco

13

Hi @marco ,

  1. Can you please try without -f and without --disable-sync?
  2. What FPS, resolution, and scene complexity (e.g., crowded/fast vs. single/static) do you need - and how many parallel apps?

Thanks,