Subject: matmul_decomposition fails with "LayerPrecisionConfig" object has no field "precision" on DFC [3.34]

General Discussion
, ,
1

Hi,

I’m hitting an internal SDK error when applying matmul_decomposition in a model script (ALLS) during quantization (runner.optimize()), on a custom SuperGlue-based model (attention-heavy, ONNX → HAR).

ALLS line used:

pre_quantization_optimization(matmul_decomposition, layers=[hailo_superglue_full/matmul9], policy=enabled, precision_mode=a16_w16)

Error:

ValueError: "LayerPrecisionConfig" object has no field "precision"

Full traceback:

File ".../hailo_model_optimization/flows/optimization_flow.py", line 396, in setup_optimization
    self._switch_decompose_matmul()
File ".../hailo_model_optimization/flows/optimization_flow.py", line 853, in _switch_decompose_matmul
    algo.run()
File ".../hailo_model_optimization/algorithms/matmul_decompose/matmul_decompose_algo.py", line 56, in _run_int
    decomp.update_mo_config(self._model_config)
File ".../hailo_model_optimization/algorithms/matmul_decompose/matmul_decompose_blocks.py", line 260, in update_mo_config
    ps.precision = presicion
File ".../pydantic/v1/main.py", line 357, in __setattr__
    raise ValueError(f'"{self.__class__.__name__}" object has no field "{name}"')
ValueError: "LayerPrecisionConfig" object has no field "precision"

This fails during setup_optimization(), before any model-specific quantization runs — so it seems to happen purely from having a matmul_decomposition directive present at all, regardless of the target layer or precision_mode.

Environment:

  • Hailo DFC / hailo_sdk_client version: 3.34

  • Hailo Dataflow Compiler v3.34.0

  • Target hw_arch: hailo8 / hailo8l

  • Python: 3.10

Question: Is this a known issue with matmul_decomposition in this DFC release? Is there a fixed version, or a different parameter name/schema expected for LayerPrecisionConfig in this build? Any guidance on the correct way to invoke precision_mode=a16_w16 decomposition would be appreciated.

Thanks!

2

Hi,
Did you manage to compile the SuperGlue model to HEF successfully?

3

Hi @jack_Gray :waving_hand:

That one is a bug in the SDK itself, not something wrong in your alls line. I went digging in my 3.33 install and the code is already broken there, so moving between 3.33 and 3.34 will not help you.

LayerPrecisionConfig declares these fields (mo_config_layer.py):

class LayerPrecisionConfig(LayerConfigBaseModel, validate_assignment=True):
    quantization_groups: int = Field(None)
    quantization_weight_groups: int = Field(None)
    precision_mode: PrecisionMode = Field(None)
    bias_mode: BiasMode = Field(None)
    signed_output: bool = Field(None)

There is no precision field, only precision_mode, and the class is built with validate_assignment=True, so pydantic refuses the assignment. Your traceback lands on matmul_decompose_blocks.py:260, which does exactly that:

ps.precision = presicion          # line 260, MatmulDecompose1616  -> breaks

Now the interesting part. The same file has a second copy of that loop for the other decomposition path, and that one is written correctly:

ps.precision_mode = presicion     # line 359, MatmulDecompose168   -> fine

So the bug only lives on the 16x16 path. MatmulDecompose1616 is what you get when you ask for precision_mode=a16_w16, and MatmulDecompose168 is the default a16_w8. Just dropping precision_mode=a16_w16 from your line should get you past it:

pre_quantization_optimization(matmul_decomposition, layers=[hailo_superglue_full/matmul9], policy=enabled)

If you really need 16x16, the local patch is one line, change ps.precision to ps.precision_mode on line 260.

One thing worth keeping in mind while you are there: on hailo8/hailo8l the matmul layer itself only takes three modes (hailo_matmul.py):

SUPPORTED_PRECISION_MODE = {
    PrecisionMode.a8_w8,
    PrecisionMode.a8_w8_a8,
    PrecisionMode.a8_w8_a16,
}

The matmul is an 8-bit engine on this generation, and matmul_decomposition is the trick that builds a 16-bit matmul out of 8-bit ones. So the precision you can reach there has a ceiling no matter how the config is written.

To be honest about what I checked: I read this from the 3.33 source and I could not reproduce your exact ValueError locally, my run dies a bit earlier in _validate_layer_config (which the SDK itself comments as “a bit too harsh”) while it builds the error message. Same area, different stop.

@Setare_Khosravi asked whether the SuperGlue model ever compiled — I have been working on that today from the other thread. It does compile on 3.33 without touching matmul_decomposition at all. What the attention actually needed was quantization_param(force_range_in=...) on the matmuls that consume the softmax, because with 500 keypoints the softmax sits around 1/N and the optimiser refuses it with “does not support shift delta”. Three GNN layers give me a 16.09 MB HEF in 6 contexts. I put the details in the other thread ..

Hope this helps!

1 Like
4

Thanks for your Detail Answer, that helped a lot!
looking forward to other detail for more information from you :slight_smile:

1 Like