we developed a software in which different networks are loaded and used by different processes. Each process has its own call to inference.
Now we updated Hailo drivers from 4.13 to 4.19. The software did not work as it was and the drivers asked to enable the hailort service. The software runs but a new process shows up on htop using 50% CPU.
We developed a hybrid solution keeping the new changes but:
hailort service is not installed
we disabled all the multiprocessing related options in the code
we re-enabled network groups
The performances are better but we do not understand what is happening.
Hi @luciccone
Are you able to run multiple models across multiple processes without enabling Hailort multi-process service? What is this hybrid solution?
Hello @shashi ,
yes it works. In reference to the 4.13 version, we instantiated all the needed networks in the father process that distributed the resources to the children.
The 4.19 version required to instantiate directly in the subprocesses but we disabled all the multiprocessing stuff related to hailo.
Hi @luciccone
Thanks for the explanation. In our PySDK, we are able to get multi-process to work without Hailort multi process service. From our benchmarking, the overhead of multi-process service is noticeable on weaker hosts like Raspberry Pi but is not noticeable on stronger hosts (say Intel i3/i5 etc).