Conversion from ONNX to HAR changed Inputs dimensions

pedrosantos · August 14, 2024, 8:57am

Hi,

I have a simple regressor model that I’m trying to use with Hailo.
For information, it’s definition in PyTorch is this:

        self.regressor = nn.Sequential(
            nn.Conv2d(input_channels, 196, 7, padding=3),
            nn.ReLU(),
            nn.UpsamplingBilinear2d(scale_factor=2),
            nn.Conv2d(196, 128, 5, padding=2),
            nn.ReLU(),
            nn.UpsamplingBilinear2d(scale_factor=2),
            nn.Conv2d(128, 64, 3, padding=1),
            nn.ReLU(),
            nn.UpsamplingBilinear2d(scale_factor=2),
            nn.Conv2d(64, 32, 1),
            nn.ReLU(),
            nn.Conv2d(32, 1, 1),
            nn.ReLU(),
        )

The inputs to this model come from a ResNet50 features map, so the input tensor is a 4D shape (actually it’s 5D but I remove the batch to simplify).

After the conversion from PyTorch to ONNX, I use ONNXSIM to simplify and the result is this:

The input shape is : float32 [19, 6, 57, 75]
The output shape is: float32 [19, 1, 456, 600]

I start the process of conversion to HAR, inside a new docker environment of AI Suite, using the “hailo parser” to convert to HAR:

hailo parser onnx 
--hw-arch hailo8 
--net-name famnet 
--har-path ./hailo_models/famnet_converted_model.har 
--start-node-names features 
--end-node-names result  
--tensor-shapes "features=[19, 6, 57, 75]" 
--parsing-report-path ./hailo_models/parser_report.txt  
models/FamNet_600_sim.onnx

The conversion seems to run successfully:

[info] Current Time: 18:02:47, 08/13/24
[info] CPU: Architecture: x86_64, Model: Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz, Number Of Cores: 6, Utilization: 17.6%
[info] Memory: Total: 15GB, Available: 10GB
[info] System info: OS: Linux, Kernel: 6.1.0-21-amd64
[info] Hailo DFC Version: 3.27.0
[info] HailoRT Version: 4.17.0
[info] PCIe: No Hailo PCIe device was found
[info] Running `hailo parser onnx --hw-arch hailo8 --net-name famnet --har-path ./hailo_models/famnet_converted_model.har --start-node-names features --end-node-names result --tensor-shapes features=[19, 6, 57, 75] --parsing-report-path ./hailo_models/parser_report.txt models/FamNet_vasos_600_sim.onnx`
[info] Translation started on ONNX model famnet
[info] Restored ONNX model famnet (completion time: 00:00:00.03)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.06)
[info] Start nodes mapped from original model: 'features': 'famnet/input_layer1'.
[info] End nodes mapped from original model: '/regressor/regressor.12/Relu'.
[info] Translation completed on ONNX model famnet (completion time: 00:00:09.56)
[info] Saved HAR to: /local/shared_with_docker/FamNet/hailo_models/famnet_converted_model.har
[info] Saved parsing report to: ./hailo_models/parser_report.txt

However, when I check the HAR using “hailo profiler” I see this:

Which says that my input tensor is a [57 x 75 x 6] dimension and my output is a [456 x 600 x 1]. It seems during the conversion from ONNX to HAR the process squeezed the input and the output shape, removing 1 dimension.

This is a big problem as I need the 19 maps.

What can I do to preserve the model input shapes during the conversion to HAR ?

Thank you

Regards,

Nadav · August 14, 2024, 12:57pm

Hi @pedrosantos,
Thank you for the detailed question, the DFC removes the batch dimention since we treat it in runtime (only). So, it’s irrelevant from the compiler POV.
I hope that this makes sense.

pedrosantos · August 14, 2024, 1:54pm

Hi @Nadav

Thank you for your answer.

That means I can’t use (at all) any input tensors with more than 3 dimensions ? or that the DFC removes 1 “batch” dimension always ?

if it’s the later, does it work to try to use a 5D shape input (like [1, 19, 6, 57, 75]) in order to let the DFC remove the first dimension and end with [19, 6, 57, 75] ?

Thank you,

Regards

pedrosantos · August 16, 2024, 2:28pm

Hi,
I’m still trying to find a solution to my problem.

Now, I’m trying to convert the all model instead of only the Regressor component as it all starts with a regular image [3, 456, 600] input as this seems are what the Hailo is designed for the most.

For context, my Regressor network is part of a larger network that uses a ResNet50 for feature extraction, execute some calculations on the features and then it ends calling the Regressor to generate the output map (with shape [1, 1, 456, 600]).

With the feature_maps from the ResNet50 I do some complementar processing, but the final operations that I think matter most are the following:

 (...)
combined_features = torch.cat( combined_features_list, dim=0)
specific_features = combined_features.permute(1, 0, 2, 3)
output = regressor_model (specific_features)
output = torch.mean(output, dim=(0), keepdim=True)

For the context sake:

“combined_features_list” is a list of 6 tensors each with shape [1, 19, 57, 75]
The “concat” operation returns a tensor with [6, 19, 57, 75] which is permuted to: [19, 6, 57, 75] which is the shape my “Regressor” network needs for the input.
“regressor_model” is the network CountRegressor (definition on top of this thread);

This all work fine.
I then export this model to ONNX, use ONNXSimplifer and I get this:

┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃                 ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Add             │ 13             │ 13               │
│ Cast            │ 9              │ 0                │
│ Concat          │ 13             │ 1                │
│ Constant        │ 178            │ 104              │
│ ConstantOfShape │ 6              │ 0                │
│ Conv            │ 54             │ 52               │
│ Gather          │ 2              │ 0                │
│ MaxPool         │ 1              │ 1                │
│ Pad             │ 6              │ 0                │
│ ReduceMean      │ 1              │ 1                │
│ Relu            │ 45             │ 45               │
│ Reshape         │ 12             │ 0                │
│ Resize          │ 6              │ 5                │
│ Shape           │ 5              │ 0                │
│ Slice           │ 11             │ 2                │
│ Transpose       │ 7              │ 1                │
│ Unsqueeze       │ 6              │ 0                │
│ Model Size      │ 50.6MiB        │ 46.0MiB          │
└─────────────────┴────────────────┴──────────────────┘

I’m close to successfully converting the ONNX model to HAF format, but I’m encountering issues with these two operations: CONCAT and TRANSPOSE:

hailo_sdk_client.tools.parser_cli.ParsingWithRecommendationCLIException: Parsing failed. The errors found in the graph are:

UnsupportedConcatLayerError in op /feature_combiner/Concat_12: Unsupported concat over axis 0
UnsupportedShuffleLayerError in op /feature_combiner/Transpose_6: Failed to determine type of layer to create in node /feature_combiner/Transpose_6

I read the DFC documentation and I saw that these operations are supported but with some caveats.

I tried different things but I’m not able to remove the concat or the permutation operation from my model definition.

Can someone give some help on how I can successfully convert this model to HAR ?

Maybe some way on how to make these operations work for the conversion or how I can substitute these operations by other hailo friendly operations that do the same.

Any assistance is appreciated.

Thank you,
Regards.

pedrosantos · August 18, 2024, 6:06am

if it helps, this is the graph from the ONNX file with the 2 operations:

Nadav · August 18, 2024, 6:45am

That means that if your data is indeed 4D + 1D for batch, it would pose a problem for the compiler. if the 5D op is only on the margins and the bulk of hte model is in 4D, you can omit that node from being run on Hailo

Nadav · August 18, 2024, 6:48am

Transpose is a problematic op for the DFC, especially if it operates on the batch dimension as in the above example.
Some options to address:

If this is the last op, remove it from the HEF
Change the conv such that he transpose will not be required.
Divide to 2 HEFs, do the transpose on the host

pedrosantos · August 25, 2024, 4:45am

Hi @Nadav
Thank you for your answer.

I went with your suggestion of dividing the Model in two parts and generating 2 HEF files.

One of the parts I was able to successfully complete all the conversion process.
However, the first part of the model (and the larger) that contains a ResNet50 and some additional CONVs layers, I was not able to finish the process and it failed in the compilation phase due to the following error:

Iteration #66 - Contexts: 2 
[error] Mapping Failed (Timeout, allocation time: 3h 43m 45s)
Compiler could not find a valid partition to contexts. Most commom error is: Failed to resolve prepost with 42/66 failures.
Mapping Failed (Timeout, allocation time: 3h 43m 45s)

[error] Failed to produce compiled graph
[error] BackendAllocatorException: Compilation failed: Compiler could not find a valid partition to contexts. Most commom error is: Failed to resolve prepost with 42/66 failures.
Mapping Failed (Timeout, allocation time: 3h 43m 45s)

Complete hailo_sdk.log is here:

hailo_sdk.log

[2024-08-23 00:50:08.361] [default] [info] Ran from command: "convert_resnetCombinerToHailo.py"
[2024-08-23 00:50:08.361] [default] [info] Loading network parameters
[2024-08-23 00:50:08.529] [default] [info] Starting Hailo allocation and compilation flow
[2024-08-23 00:50:08.550] [default] [info] Model name: famnet_resnetCombiner
[2024-08-23 00:51:47.043] [default] [info] Trying to solve in single context
[2024-08-23 00:51:49.630] [default] [info] Model fits in single context
[2024-08-23 00:51:49.630] [default] [info] Model fits in single context, trying to increase utilization
[2024-08-23 00:51:49.653] [default] [info] Initial resources counter with: SCs: 268/512, 16bit SCs: 0/512, APUs: 78/160, IAs: 81/256, L3 weights: 339/922, L3 ios: 232/922, LCUs: 78/154
[2024-08-23 00:53:37.780] [default] [info] Converged after 3 iterations
[2024-08-23 00:53:37.780] [default] [info] Final resources: SCs: 270/512, 16bit SCs: 0/512, APUs: 78/160, IAs: 81/256, L3 weights: 339/922, L3 ios: 165/922, LCUs: 78/154
[2024-08-23 00:53:37.780] [default] [info] Trying to apply higher utilization solution
[2024-08-23 00:53:40.802] [default] [info] Running Auto-Merger
[2024-08-23 00:53:49.979] [default] [info] Auto-Merger is done
[2024-08-23 00:54:28.775] [default] [info] Higher utilization solution failed validation: Failed to resolve prepost, skipping to the next one
[2024-08-23 00:54:28.775] [default] [info] Trying to apply higher utilization solution
[2024-08-23 00:54:31.475] [default] [info] Running Auto-Merger
[2024-08-23 00:54:40.427] [default] [info] Auto-Merger is done
[2024-08-23 00:55:20.848] [default] [info] Higher utilization solution failed validation: Failed to resolve prepost, skipping to the next one
[2024-08-23 00:55:23.965] [default] [info] Running Auto-Merger
[2024-08-23 00:55:33.290] [default] [info] Auto-Merger is done
[2024-08-23 00:56:15.576] [default] [info] Single context solution failed, skipping it
[2024-08-23 00:56:15.576] [default] [info] Single context flow failed: Failed to resolve prepost, continuing in multi context
[2024-08-23 00:57:46.218] [default] [info] Finding the best partition to contexts...
[2024-08-23 01:03:31.459] [default] [info] Running Auto-Merger
[2024-08-23 01:04:31.458] [default] [info] Auto-Merger is done
[2024-08-23 01:08:31.459] [default] [info] Iteration #1 - Contexts: 1, Changed: context_0, Fast FPS: 12.0848 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:09:38.702] [default] [info] Iteration #2 - Contexts: 2, Changed: context_0, Fast FPS: 12.0403 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:11:38.808] [default] [info] Iteration #3 - Contexts: 2, Changed: context_0, Fast FPS: 12.0403 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:14:31.469] [default] [info] Iteration #4 - Contexts: 2, Changed: context_0, Fast FPS: 12.0402 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:16:31.467] [default] [info] Iteration #5 - Contexts: 2, Changed: context_0, Fast FPS: 12.0402 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:18:31.469] [default] [info] Iteration #6 - Contexts: 2, Changed: context_0, Fast FPS: 12.0402 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:20:31.470] [default] [info] Iteration #7 - Contexts: 2, Changed: context_0, Fast FPS: 12.0402 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:21:38.940] [default] [info] Iteration #8 - Contexts: 2, Changed: context_0, Fast FPS: 12.0403 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:23:38.891] [default] [info] Iteration #9 - Contexts: 2, Changed: context_0, Fast FPS: 12.0402 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:25:38.908] [default] [info] Iteration #10 - Contexts: 2, Changed: context_0, Fast FPS: 12.0403 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:27:38.901] [default] [info] Iteration #11 - Contexts: 2, Changed: context_0, Fast FPS: 12.0399 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:29:38.914] [default] [info] Iteration #12 - Contexts: 2, Changed: context_0, Fast FPS: 12.0403 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:31:38.913] [default] [info] Iteration #13 - Contexts: 2, Changed: context_0, Fast FPS: 12.0403 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:33:38.894] [default] [info] Iteration #14 - Contexts: 2, Changed: context_0, Fast FPS: 12.0403 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 01:50:39.012] [default] [info] Iteration #15 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 01:53:38.727] [default] [info] Iteration #16 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 01:57:33.888] [default] [info] Iteration #17 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:01:31.607] [default] [info] Iteration #18 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:03:38.758] [default] [info] Iteration #19 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:06:38.864] [default] [info] Iteration #20 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:09:39.043] [default] [info] Iteration #21 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:12:38.612] [default] [info] Iteration #22 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:15:38.815] [default] [info] Iteration #23 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:18:38.919] [default] [info] Iteration #24 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:21:38.558] [default] [info] Iteration #25 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:24:38.745] [default] [info] Iteration #26 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:27:38.929] [default] [info] Iteration #27 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:30:38.543] [default] [info] Iteration #28 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:33:38.803] [default] [info] Iteration #29 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:36:38.898] [default] [info] Iteration #30 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:39:38.499] [default] [info] Iteration #31 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:42:38.459] [default] [info] Iteration #32 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:45:36.076] [default] [info] Iteration #33 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:48:38.535] [default] [info] Iteration #34 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:51:38.496] [default] [info] Iteration #35 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:54:38.434] [default] [info] Iteration #36 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 02:57:38.433] [default] [info] Iteration #37 - Contexts: 2, Changed: context_0, Fast FPS: 10.2506 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d1 with Agent infeasible
Validator failed on node: conv28_defuse_1x1_d2 with Agent infeasible

[2024-08-23 03:15:33.102] [default] [info] Iteration #38 - Contexts: 2, Changed: context_0, Fast FPS: 7.04852 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible
Validator failed on node: conv27_defuse_1x1_d1 with Agent infeasible

[2024-08-23 03:32:40.734] [default] [info] Iteration #39 - Contexts: 2, Changed: context_0, Fast FPS: 12.2839 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 03:35:39.652] [default] [info] Iteration #40 - Contexts: 2, Changed: context_0, Fast FPS: 12.2884 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 03:38:39.706] [default] [info] Iteration #41 - Contexts: 2, Changed: context_0, Fast FPS: 12.1568 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 03:41:40.400] [default] [info] Iteration #42 - Contexts: 2, Changed: context_0, Fast FPS: 12.1043 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 03:44:39.934] [default] [info] Iteration #43 - Contexts: 2, Changed: context_0, Fast FPS: 12.1171 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 03:47:39.908] [default] [info] Iteration #44 - Contexts: 2, Changed: context_0, Fast FPS: 12.1117 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 03:50:37.170] [default] [info] Iteration #45 - Contexts: 2, Changed: context_0, Fast FPS: 12.1048 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 03:53:35.634] [default] [info] Iteration #46 - Contexts: 2, Changed: context_0, Fast FPS: 12.1095 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 03:56:35.637] [default] [info] Iteration #47 - Contexts: 2, Changed: context_0, Fast FPS: 12.1041 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 03:59:35.703] [default] [info] Iteration #48 - Contexts: 2, Changed: context_0, Fast FPS: 12.098 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:02:35.701] [default] [info] Iteration #49 - Contexts: 2, Changed: context_0, Fast FPS: 12.1026 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:05:35.712] [default] [info] Iteration #50 - Contexts: 2, Changed: context_0, Fast FPS: 12.0972 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:08:35.649] [default] [info] Iteration #51 - Contexts: 2, Changed: context_0, Fast FPS: 12.1037 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:11:35.640] [default] [info] Iteration #52 - Contexts: 2, Changed: context_0, Fast FPS: 11.936 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:14:31.525] [default] [info] Iteration #53 - Contexts: 2, Changed: context_0, Fast FPS: 11.9426 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:16:31.540] [default] [info] Iteration #54 - Contexts: 2, Changed: context_0, Fast FPS: 11.8955 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:17:39.891] [default] [info] Iteration #55 - Contexts: 2, Changed: context_0, Fast FPS: 11.894 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:19:39.835] [default] [info] Iteration #56 - Contexts: 2, Changed: context_0, Fast FPS: 11.898 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:21:39.829] [default] [info] Iteration #57 - Contexts: 2, Changed: context_0, Fast FPS: 11.9049 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:23:39.848] [default] [info] Iteration #58 - Contexts: 2, Changed: context_0, Fast FPS: 11.9034 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:25:39.756] [default] [info] Iteration #59 - Contexts: 2, Changed: context_0, Fast FPS: 11.9074 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:27:39.718] [default] [info] Iteration #60 - Contexts: 2, Changed: context_0, Fast FPS: 11.9142 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:29:39.723] [default] [info] Iteration #61 - Contexts: 2, Changed: context_0, Fast FPS: 11.9126 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:31:39.731] [default] [info] Iteration #62 - Contexts: 2, Changed: context_0, Fast FPS: 11.9164 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:33:39.716] [default] [info] Iteration #63 - Contexts: 2, Changed: context_0, Fast FPS: 11.9162 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:35:39.671] [default] [info] Iteration #64 - Contexts: 2, Changed: context_0, Fast FPS: 11.9164 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:37:39.614] [default] [info] Iteration #65 - Contexts: 2, Changed: context_0, Fast FPS: 11.9021 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:39:39.590] [default] [info] Iteration #66 - Contexts: 2, Changed: context_0, Fast FPS: 12.0103 (best: 0), Failed on: Failed to resolve prepost
[2024-08-23 04:40:31.731] [default] [info] Slowest context (context_0) has 1 layer (input_layer1)
[2024-08-23 04:41:31.854] [default] [info] e[1;41mMapping Failed (Timeout, allocation time: 3h 43m 45s)e[0m
[2024-08-23 04:41:31.857] [default] [info] Builder fail with status 94: Compiler could not find a valid partition to contexts. Most commom error is: Failed to resolve prepost with 42/66 failures.
Mapping Failed (Timeout, allocation time: 3h 43m 45s)
[2024-08-23 04:41:31.857] [default] [info] Compiler could not find a valid partition to contexts. Most commom error is: Failed to resolve prepost with 42/66 failures.
Mapping Failed (Timeout, allocation time: 3h 43m 45s)

The optimization phase was successful. For information the ALLS_Model_script I used was:

alls_model_script = [
    "normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])\n",
    "normalization2 = normalization([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n",
    "model_optimization_flavor(optimization_level=2, compression_level=0, batch_size=1)\n",
    "performance_param(compiler_optimization_level=max)\n",
]

I also attach here the model network structure, if it helps:

Model Structure

I’m using a new docker environment with Hailo AI suite 2024-07.1

Can you help me trying to find out what is the problem of the compilation phase and what can I do to fix this ?

Thank you,
Regards.

Nadav · August 25, 2024, 7:01am

Hi @pedrosantos, please give it a try without this line in the alls:
performance_param(compiler_optimization_level=max)

This puts more stress on the compiler to really push the allocation.

pedrosantos · August 25, 2024, 10:41am

Hi @Nadav

I already tried that and also with:
“model_optimization_flavor(optimization_level=0, compression_level=0, batch_size=1)\n”,

Tried also on Ai Suite 2024-04 (docker enviroment)
All with the same result…

My last log :

Tracelog

[2024-08-23 23:39:55.581] [default] [info] Ran from command: “convert_resnetCombinerToHailo.py”

[2024-08-23 23:39:55.582] [default] [info] Loading network parameters

[2024-08-23 23:39:55.972] [default] [info] Starting Hailo allocation and compilation flow

[2024-08-23 23:39:55.996] [default] [info] Model name: famnet_resnetCombiner

[2024-08-23 23:41:41.561] [default] [info] Trying to solve in single context

[2024-08-23 23:41:44.297] [default] [info] Model fits in single context

[2024-08-23 23:41:44.297] [default] [info] Model fits in single context, trying to increase utilization

[2024-08-23 23:41:44.323] [default] [info] Initial resources counter with: SCs: 268/512, 16bit SCs: 0/512, APUs: 78/160, IAs: 81/256, L3 weights: 339/922, L3 ios: 232/922, LCUs: 78/154

[2024-08-23 23:43:37.013] [default] [info] Converged after 3 iterations

[2024-08-23 23:43:37.013] [default] [info] Final resources: SCs: 270/512, 16bit SCs: 0/512, APUs: 78/160, IAs: 81/256, L3 weights: 339/922, L3 ios: 165/922, LCUs: 78/154

[2024-08-23 23:43:37.014] [default] [info] Trying to apply higher utilization solution

[2024-08-23 23:43:40.234] [default] [info] Running Auto-Merger

[2024-08-23 23:43:49.389] [default] [info] Auto-Merger is done

[2024-08-23 23:44:23.185] [default] [info] Higher utilization solution failed validation: Failed to resolve prepost, skipping to the next one

[2024-08-23 23:44:23.185] [default] [info] Trying to apply higher utilization solution

[2024-08-23 23:44:25.984] [default] [info] Running Auto-Merger

[2024-08-23 23:44:34.923] [default] [info] Auto-Merger is done

[2024-08-23 23:45:12.118] [default] [info] Higher utilization solution failed validation: Failed to resolve prepost, skipping to the next one

[2024-08-23 23:45:15.319] [default] [info] Running Auto-Merger

[2024-08-23 23:45:24.473] [default] [info] Auto-Merger is done

[2024-08-23 23:45:58.833] [default] [info] Single context solution failed, skipping it

[2024-08-23 23:45:58.834] [default] [info] Single context flow failed: Failed to resolve prepost, continuing in multi context

[2024-08-23 23:47:46.461] [default] [info] Finding the best partition to contexts…

[2024-08-23 23:56:31.911] [default] [info] Running Auto-Merger

[2024-08-23 23:57:31.917] [default] [info] Auto-Merger is done

[2024-08-24 00:00:37.393] [default] [info] Iteration #1 - Contexts: 1, Changed: context_0, Fast FPS: 12.0848 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:02:37.571] [default] [info] Iteration #2 - Contexts: 2, Changed: context_0, Fast FPS: 12.0407 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:04:37.710] [default] [info] Iteration #3 - Contexts: 2, Changed: context_0, Fast FPS: 12.0407 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:06:38.685] [default] [info] Iteration #4 - Contexts: 2, Changed: context_0, Fast FPS: 12.0406 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:08:38.462] [default] [info] Iteration #5 - Contexts: 2, Changed: context_0, Fast FPS: 12.0406 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:10:39.043] [default] [info] Iteration #6 - Contexts: 2, Changed: context_0, Fast FPS: 12.0406 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:12:39.691] [default] [info] Iteration #7 - Contexts: 2, Changed: context_0, Fast FPS: 12.0406 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:14:38.472] [default] [info] Iteration #8 - Contexts: 2, Changed: context_0, Fast FPS: 12.0407 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:16:38.476] [default] [info] Iteration #9 - Contexts: 2, Changed: context_0, Fast FPS: 12.0406 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:18:38.465] [default] [info] Iteration #10 - Contexts: 2, Changed: context_0, Fast FPS: 12.0407 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:20:38.452] [default] [info] Iteration #11 - Contexts: 2, Changed: context_0, Fast FPS: 12.0403 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:22:37.754] [default] [info] Iteration #12 - Contexts: 2, Changed: context_0, Fast FPS: 12.0407 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:24:37.789] [default] [info] Iteration #13 - Contexts: 2, Changed: context_0, Fast FPS: 12.0407 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:26:37.752] [default] [info] Iteration #14 - Contexts: 2, Changed: context_0, Fast FPS: 12.0407 (best: 0), Failed on: Failed to resolve prepost

[2024-08-24 00:33:29.028] [default] [info] Iteration #15 - Contexts: 2, Changed: context_0, Fast FPS: 10.2262 (best: 0), Failed on: Validator failed on node: conv27_defuse_1x1_d0 with Agent infeasible