Hi all,
we run YOLOv8 detection and pose models on a Hailo-10H with on-chip NV12 input conversion (input_conversion(input_layer1, nv12_to_rgb) in the model script). On the chip, the converted image comes out darker and lower in contrast than the DFC emulates: every gain of the BT.601 matrix is about 6–8 % low, while the offsets are right. The DFC’s emulation uses the correct matrix, so its accuracy and SNR estimates never show it; the loss appears only on hardware.
We work around it on the host, but the right correction is different for each input size and, on real models, for each HEF. We’d like to know whether this is expected, and whether there’s a supported way to make the chip and the emulation agree.
Environment
-
DFC 5.3.0 (
hailo_sdk_client 5.3.0, Docker image with CUDA 12.5.1), targethailo10h. -
HailoRT 5.3.0 on a Raspberry Pi 5 (Hailo-10H), InferModel API, UINT8 input.
-
Model scripts:
normalization([0,0,0],[255,255,255]),input_conversion(..., nv12_to_rgb), calibration set in BT.601 limited-range YUV 4:4:4.
1. Probe: the conversion itself, per input size
A probe HEF per size: NV12 input → nv12_to_rgb → /255 → identity 1×1 conv, so the output is the chip’s RGB. Fed random YUV, flat patches and real frames; least-squares fit per output channel and 2×2 pixel phase.
| Input size | Chip gain Y / V→R / U→B (BT.601 = 1) | Host correction (1 / mean) |
|---|---|---|
| 640×640 | 0.918 / 0.927 / 0.923 | ×1.085 |
| 1280×736 | 0.938 / 0.947 / 0.942 | ×1.061 |
| 2048×1536 | 0.930 / 0.930 / 0.934 | ×1.074 |
| 4064×3040 | 0.935 / 0.934 / 0.938 | ×1.068 |
Otherwise the conversion is exact: linear (fit rms 0.30, i.e. 8-bit rounding), offsets as BT.601 (R −223.1, G +135.9, B −276.8), chroma nearest with correct siting, identical across the frame width. The gain does not follow the input size smoothly (1280×736 needs less correction than both 640 and 2048).
2. Real models: the best correction is per HEF
Correction: every NV12 byte through a host LUT v → min(255, round(v × g)) before inference. For each HEF we sweep g and compare the chip’s raw output heads with the same model’s ONNX on the same frames (mean SNR over box and keypoint heads):
| Model | Input | Best g | Probe g for that size |
|---|---|---|---|
| YOLOv8s-pose, model A | 1280×736 | 1.0851 | 1.061 |
| YOLOv8s-pose, model B | 1280×736 | 1.0876 | 1.061 |
| YOLOv8s-pose, model C (two calibration sets) | 1280×736 | 1.0851 | 1.061 |
| YOLOv8n detection | 1280×736 | 1.061 | 1.061 |
| YOLOv8n-pose, model D | 2048×1536 | 1.074 | 1.074 |
| YOLOv8n-pose, model E | 2048×1536 | 1.06 (flat to 1.07) | 1.074 |
| YOLOv8n-pose | 4064×3040 | 1.068 | 1.068 |
-
At the same input size, different models need different corrections (1.061 vs 1.085 at 1280×736; 1.06 vs 1.074 at 2048×1536).
-
The optimum is narrow: 0.5 % off costs 3–8 dB on the output heads. Without any correction a 4064×3040 nano model matches 38 of 69 of its ONNX detections; with the right g, 68 of 69.
-
An RGB-input HEF of the same model, fed the same frames converted to RGB on the host, matches its ONNX without any correction, so 4:2:0 chroma itself costs nothing.
Our guess is that the DFC folds the input conversion into the first quantized layer, so the hardware’s fixed conversion error lands differently in each model’s quantization.
3. Folding the inverse into the model script doesn’t help
We built a HEF with the inverse gains folded into normalization (and the calibration set scaled to match). On the chip it behaved exactly like the uncorrected build: the hardware conversion seems to use its own fixed coefficients regardless of the normalization folded around it.
Questions
-
Is the gain difference between the on-chip
nv12_to_rgband the DFC emulation known? Is it fixed in a later DFC/HailoRT release? -
Which coefficients does the hardware conversion use, and why do they differ by input size?
-
Is there a supported way to make chip and emulation agree, e.g. a model-script option, a different conversion layer, or calibrating with the chip’s coefficients?
-
If a host-side correction is the intended way, is there a recommended method to find it per HEF?
We can share the probe HEFs and the probe script (identity network, no proprietary weights) if that helps.
Thanks!