Custom Multi Networks Compilation

General
, ,
1

I have two models yolov8n, pidnet(not included in model zoo) and I want to try combine two models into one .hef file while compiling.
Each model can be compiled with HEF and work well with Hailo-8.

I found multi-network sample script(fast_depth_ssd, face_detection_landmarks) in Hailo Model Zoo, and I want to apply it, but I don’t know how.

# ../cfg/multi-networks/default_multi.yaml
results_dir: "./"
models: [ ]
alls_script: null

If Hailo Model Zoo supports multi-network compilation using custom models,
is there any example?

2

Hey @mmmsk

Here’s a shorter version of the explanation:

The Hailo Dataflow Compiler (DFC) supports multi-network compilation using the join() API. This allows combining multiple models for joint compilation. Available join types include NONE, AUTO_JOIN_INPUTS, AUTO_CHAIN_NETWORKS, and CUSTOM.

To join two models:

runner1.join(runner2, scope1_name=net1_scope_names, scope2_name=net2_scope_names)

After joining, use the compile() method to create a single HEF file. If the combined model is too large, it’s automatically partitioned into multiple contexts.

For your yolov8n and pidnet scenario, refer to the multi-network sample scripts in the Hailo Model Zoo (e.g., fast_depth_ssd, face_detection_landmarks). Define your models in a configuration file like default_multi.yaml and specify how they should be joined.

3

Thank you for the answer @omria
I successfully optimized and joined my custom models(yolov8n, pidnet) to one .har file by using multi-main.py code in the Hailo Model Zoo.
But I got yolov8_nms_postprocesserrors while compiling the combined har file.
This is error log. Do you know how to resolve this error?

image
image1380×292 61.4 KB

# > my_multi_networks.alls

allocator_param(automatic_ddr=disabled)
#performance_param(fps=100)
performance_param(compiler_optimization_level=max)
4

Hey @mmmsk,

Could you please provide the full log or HAR file for us to review?

Also, I should have mentioned this earlier: you can run two models with separate HEFs on the same chip using the scheduler method, which is more efficient than the join method. You can find an example of this here:

Scheduler Method Example

5

Thanks @omria ,
My Hailo8 is working on low-bandwidth(pcie 3.0 x1). It takes quite a while to transfer two images to both models. So I want to try transfer one Image to one HEF(two models joined).
This is my joined HAR structure and Error Log of yolov8n postprocess part from compiling.

my_hailo_har
my_hailo_har1058×595 54.6 KB

2024-09-02 17:55:46,488 - INFO - client_runner.py:386 - Loading model script commands to yolov8n from /home/mobilus/workspace/adm/hailo/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/custom_yolov8n.alls
2024-09-02 17:55:51,250 - INFO - client_runner.py:386 - Loading model script commands to pidnet_m_mobilus_640_opt3 from /home/mobilus/workspace/adm/hailo/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/custom_pspnet.alls
2024-09-02 17:55:51,873 - INFO - network_join.py:144 - Order outputs by original network outputs are first and joined network outputs are second
2024-09-02 17:55:54,351 - INFO - client_runner.py:1443 - Saved HAR to: /home/mobilus/workspace/adm/hailo/my_hailo/multi-networks/yolo-pidnet/results_test/joined_yolov8n_pidnet_m_mobilus_640_opt3.har
2024-09-02 17:55:54,352 - INFO - client_runner.py:386 - Loading model script commands to joined_yolov8n_pidnet_m_mobilus_640_opt3 from ./my_multi_networks.alls
2024-09-02 17:55:55,696 - ERROR - hailo_tools_runner.py:431 - Failed to produce compiled graph
2024-09-02 17:55:55,984 - ERROR - hailo_tools_runner.py:219 - BackendAllocatorException: Compilation failed: NMSPostProcessNetworkNode yolov8n/yolov8_nms_postprocess protobuf has an unknown postprocess_type
2024-09-02 17:55:55,985 - COMMAND - hailo_tools_runner.py:220 - Traceback (most recent call last):
  File "/home/mobilus/workspace/adm/hailo/my_hailo/multi-networks/yolo-pidnet/my_multi_main.py", line 132, in <module>
    main(args.cfg, har_path=args.har)
  File "/home/mobilus/workspace/adm/hailo/my_hailo/multi-networks/yolo-pidnet/my_multi_main.py", line 107, in main
    hef = runner.compile()
  File "/home/mobilus/workspace/adm/hailo/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 715, in compile
    return self._compile()
  File "/home/mobilus/workspace/adm/hailo/hailo_venv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/home/mobilus/workspace/adm/hailo/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 833, in _compile
    serialized_hef = self._sdk_backend.compile(fps, self.model_script, mapping_timeout)
  File "/home/mobilus/workspace/adm/hailo/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py", line 1448, in compile
    hef, mapped_graph_file = self._compile(fps, allocator_script, mapping_timeout)
  File "/home/mobilus/workspace/adm/hailo/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py", line 1439, in _compile
    hef, mapped_graph_file, auto_alls = self.hef_full_build(fps, mapping_timeout, model_params, allocator_script)
  File "/home/mobilus/workspace/adm/hailo/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py", line 1410, in hef_full_build
    auto_alls, self._mapped_graph, self._integrated_graph = allocator.create_mapping_and_full_build_hef(
  File "/home/mobilus/workspace/adm/hailo/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/allocator/hailo_tools_runner.py", line 598, in create_mapping_and_full_build_hef
    self.call_builder(network_graph_path, output_path, compilation_output_proto=compilation_output_proto,
  File "/home/mobilus/workspace/adm/hailo/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/allocator/hailo_tools_runner.py", line 556, in call_builder
    self.run_builder(network_graph_path, output_path, **kwargs)
  File "/home/mobilus/workspace/adm/hailo/hailo_venv/lib/python3.10/site-packages/hailo_sdk_client/allocator/hailo_tools_runner.py", line 419, in run_builder
    raise e.internal_exception("Compilation failed:", hailo_tools_error=compiler_msg) from None
hailo_sdk_client.sdk_backend.sdk_backend_exceptions.BackendAllocatorException: Compilation failed: NMSPostProcessNetworkNode yolov8n/yolov8_nms_postprocess protobuf has an unknown postprocess_type

+) unknown postprocess_type error occurs not only with my custom models, but also when joining the default YOLO model with other default models.