Fast SAM Onnx Different

General
,
1

Wondering why the onnx file from FastSAM-s from model zoo have different model structure with the one i export from ultralytics.

I able to compile with the model from model zoo but not the one from ultralytics

2

Hey @junsen.yee

The structural differences between the FastSAM-s ONNX file from the Hailo Model Zoo and the one you exported from Ultralytics can be attributed to the following factors:

1. Preprocessing and Postprocessing Integration

  • Models in the Hailo Model Zoo, including FastSAM-s, are specifically optimized for Hailo hardware.
  • Preprocessing steps (e.g., resizing, normalization) and postprocessing tasks (e.g., output handling) are often integrated into the ONNX structure or added during compilation for Hailo inference.
  • This integration can modify the ONNX structure compared to standard exports from frameworks like Ultralytics.

2. Quantization and Optimization

  • The Hailo Model Zoo applies quantization techniques to convert floating-point models to lower-precision formats like INT8, which significantly alters the model structure but improves inference performance on Hailo hardware.
  • Models exported from Ultralytics without such optimizations may retain a more standard, unmodified structure.

3. Custom Node Names

  • Hailo allows users to define custom start and end node names to ensure efficient parsing of the ONNX model.
  • This customization can introduce additional structural differences between a Hailo-optimized model and one exported directly from Ultralytics.

These factors likely explain why the FastSAM-s model from the Hailo Model Zoo compiles successfully for Hailo hardware, while the Ultralytics-exported version does not.

3

Is there any guide on optimizing the model specifically for Hailo hardware? i.e., I would like to try FastSAM-x instead of FastSAM-s. Is there any way I can do that? or we can only wait until it is officialy supported in Hailo Zoo?

4

To help you with your model compilation, I suggest the following:

  1. Check out our documentation at https://hailo.ai/developer-zone/documentation/
  2. Specifically, review the DFC and Model Zoo documentation. These sections contain our guides on how to compile models and list the supported operations.
  3. Compare the operations in your model against the supported operations list. This will help you identify any potential unsupported operations.
  4. After reviewing, try compiling your model again.

You mentioned you’ve already attempted to compile before. What specific error are you encountering during the compilation process? If you can share the error message, it will help us provide more targeted assistance.

5

Please see this link for the error I encoutered. I try to parse the fastSAM-x with recommended end node, optimization went fine, but during the compiling it shows:

[error] Failed to produce compiled graph
[error] BackendAllocatorException: Compilation failed: No successful assignment for: format_conversion2, concat18, feature_splitter9, shortcut_softmax1, reduce_max_softmax1, ew_sub_softmax1, reduce_sum_softmax1, ew_mult_softmax1, conv76