DFC 3.34.0 / Hailo-8L — Compiler SIGABRT on fixed selection-matrix matmuls

General Discussion
, , ,
1

Environment:

  • Hailo DFC 3.34.0
  • Target: hailo8l
  • Ubuntu 24.04 / WSL2, Python 3.10

Model: A 6-block, 768-dim, 12-head RoPE-based Vision Transformer encoder (attention-only, standard pre-norm blocks, ~7M params/block). ONNX export verified numerically correct against the original PyTorch model (max relative error ~1e-6 across the pipeline). Translation and INT8 quantization succeed reliably on every graph variant described below — only allocation/compilation fails.

Failure 1 — Compiler SIGABRT (fatal signal 6) on fixed selection-matrix matmuls

Our graph uses fixed 0/1 selection-matrix matmuls (shapes like [1,512,256], [1,516,512]) to scatter/assemble tokens (query + visual + register tokens into one sequence) — this replaced Concat ops that were separately failing with Agent infeasible at the same boundary. With this matmul-scatter approach, the native compiler process aborts with SIGABRT during allocation, with no structured error message returned to the Python API (BackendAllocatorException: Compilation failed with unexpected crash). Kernel trace confirms fatal signal 6 in the compiler process. Reproduces even on a minimal isolated graph containing only the token-assembly step (no transformer blocks at all) — so it’s specific to this op pattern, not model size. Quantization of this same graph completes successfully; only runner.compile() crashes.

Failure 2 — auto_spatial_reshape / Agent infeasible on bare [seq, channels] sequence boundaries

Independently, when a [1, 516, 768] sequence activation (register+query+visual tokens, pre-head) sits at a graph input or output edge — e.g., when compiling a subset of transformer blocks as their own graph — allocation fails with Agent infeasible on auto_spatial_reshape_from_input_layer1_to_conv2... (and the mirrored error at the output boundary). We attempted the documented format_conversion(layer, hailo_rgb_to_tf_rgb / tf_rgb_to_hailo_rgb) workaround (as suggested by the compiler’s own hint text) on both input and output edges; this did not resolve it — the format_conversion layers themselves come back with Agent infeasible. Reproduces on a single isolated attention block and on a 6-block-only subgraph (no stem).

What does work: the full single-graph encoder (stem + 6 blocks + head as one ONNX graph, so all [seq,768] tensors are internal, never at a graph edge) translates and quantizes successfully, but allocation for the full graph separately times out (native 1-hour resolver watchdog, ~1484/1486 iterations reverting on cluster mapping in context_0) rather than crashing — suggesting the graph is right at or past the edge of what a single-context/limited-context mapping can place.

Questions:

  1. Is there a known limitation or workaround for large (>~100K element) fixed selection-matrix matmuls used for token scatter/gather — is there a supported alternative op pattern for this kind of static index-based token assembly?
  2. What is the correct way to expose a [seq, channels] transformer sequence activation (not an image-shaped tensor) at a Hailo graph edge, if format_conversion isn’t sufficient? Is there a required intermediate reshape/layout op we’re missing?
  3. Any guidance on why the SIGABRT occurs without a diagnostic message — is there a way to get more verbose native-compiler error output from hailo_sdk_client?

Happy to share the minimal repro ONNX files / HARs if useful.