Has anyone successfully deployed a neural single-object tracker (.hef) on Hailo-8L?

General Discussion
, , , ,
1

Hi everyone,

I’m working on a single-object tracking application on Hailo-8L and I’m looking for a neural tracker that can run almost entirely on the Hailo accelerator.

My setup

  • Hailo-8L
  • Hailo DFC 3.34.0
  • HailoRT
  • Single object (car tracking)
  • Linux / C++
  • Detector: YOLO (already running successfully on Hailo)

What I’ve already tried

I successfully exported and parsed the NanoTrack backbone.

However, the NanoTrack head (DepthwiseBAN) fails during parsing because of the dynamic depthwise cross-correlation layer.

I’ve also looked into SiamRPN/PySOT and understand they rely on similar Siamese correlation operations that are difficult for the Hailo compiler.

What I’m looking for

I’m not looking for ByteTrack, DeepSORT, or CPU-based tracking.

I’m specifically looking for a neural single-object tracker whose entire network (or nearly the entire network) can be compiled into a .hef and run on Hailo.

Examples I’m curious about:

  • LightTrack
  • SiamFC++
  • MixFormer-lite
  • OSTrack-lite
  • HiFT
  • ROMTrack
  • Any custom CNN-based tracker

Has anyone successfully deployed a neural tracker like this on Hailo?

If yes, could you share:

  • Tracker name
  • GitHub/project
  • Whether it compiles to .hef
  • Any modifications required

Thanks!

2

Yes, I got a full neural single-object tracker compiling to a single-context .hef on Hailo-8L. The trick was replacing the dynamic correlation, not dropping the Siamese idea.

Your diagnosis is right. DepthwiseBAN, SiamRPN, and SiamFC++ all do F.conv2d(search, template_features_as_weight), so the template becomes a conv kernel generated at runtime. A static-graph compiler can’t lower that because the weights depend on the input.

The fix is to express template-to-search matching with static ops. I built a small custom CNN plus a cross-attention tracker, where search locations are queries and template locations are keys/values (Q·Kᵀ → Softmax → ·V). The correlation comes out as Conv 1×1 + Reshape + Transpose + MatMul + Softmax, all standard and all DFC-supported, so the whole network parses and compiles. It’s also a stronger matcher than xcorr.

DFC-verified results (Hailo DFC 3.33.1, --hw-arch hailo8l):

  • Parser translates the entire graph to end nodes: cls, box, nothing left on CPU.

  • Single-context .hef, about 1.9 MB, 4 clusters, roughly 50% compute utilization.

  • hailo profiler estimate is about 2,964 FPS at 0.75 ms latency.

  • About 540K params. Template 1×3×128×128, search 1×3×256×256, out to targetness 1×1×32×32 plus ltrb box 1×4×32×32 (anchor-free). Profiler report attached.

Three things that actually mattered to get it through the DFC:

  1. Use single-head attention. Multi-head reshapes the head dim into the batch axis and does a batched MatMul, and the DFC silently cut the graph at the correlation. Single-head keeps everything at batch = 1 and the parser goes end to end. If your parse “succeeds” but the recommended end-nodes land mid-net, this is why.

  2. Export SiLU as Sigmoid plus Mul, not a single Silu/Swish node. The onnx.checker rejects the non-standard op (No Op registered for Silu).

  3. Export Reshape with the shape as an input tensor, not an attribute (opset 13+). Otherwise the checker throws input size 1 not in range [2,2].

Also: fixed input shapes, fold BN, anchor-free head (a couple of 1×1 convs for cls and box), quantize to int8.

On your candidate list: OSTrack-lite, MixFormer-lite, and HiFT are attention-based, so correlation is already static MatMul/Softmax. Best starting points, just watch the multi-head batch-reshape. SiamFC++ and SiamRPN still use xcorr, same wall. LightTrack is mostly CNN, but check the head and rewrite any pointwise correlation as a MatMul.

Happy to share more on the attention-correlation block if it’s useful, so reach out if you want it.