How can I compose yaml file for hailomz eval?

General
1

Hi, I’m trying to evaluate custom 1-class model.
I don’t know how should I compose my yaml file and alls file.

I excuted this command.

hailomz eval --har /local/workspace/compiled_files/yolov8s_custom/20250502_imgsz_160_320_opset_11/yolov8s_custom_quantized_model.har --target emulator --data-path /local/shared_with_docker/tfrecord/20250502_output.tfrecord --yaml /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov8s_custom.yaml

But, I get strange APs.

<Hailo Model Zoo INFO> Start run for network yolov8s ...
<Hailo Model Zoo INFO> Initializing the runner...
<Hailo Model Zoo INFO> Chosen target is emulator
<Hailo Model Zoo INFO> Initializing the dataset ...
<Hailo Model Zoo INFO> Running inference...
[info] CPU postprocess does not exist in the model, ignoring the given argument `nms_score_threshold`
Processed: 536images [00:29, 18.14images/s]
creating index...
index created!
Loading and preparing results...
Converting ndarray to lists...
(53600, 7)
0/53600
DONE (t=0.45s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.02s).
Accumulating evaluation results...
Please run evaluate() first
DONE (t=0.00s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
<Hailo Model Zoo INFO> Done 536 images AP=-100.000 AP50=  alls_script: yolov8s_custom.alls
-100.000

The content of custom yaml file is below.

base:
- base/yolov8.yaml
postprocessing:
  device_pre_post_layers:
    nms: true
  hpp: true
network:
  network_name: yolov8s_hybo
paths:
  network_path:
    - /local/workspace/hailo/2025-04/shared_with_docker/20250428_yolov8s_custom/weights/20250428_yolov8s_custom.onnx
  alls_script: yolov8s_custom.alls

The content of custom alls file is below.

normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
change_output_activation(conv42, sigmoid)
change_output_activation(conv53, sigmoid)
change_output_activation(conv63, sigmoid)
nms_postprocess("../../postprocess_config/yolov8s_nms_config.json", meta_arch=yolov8, engine=cpu)

My Questions:

  • Am I missing any required keys or parameters? If so, could you point out the missing parts?
  • Is there a specific way to debug this problem further, or should I take a different approach?
2

Hey @yuto_arai,

Why You’re Seeing AP = -1.000

This usually means the model didn’t detect any objects that matched the ground truth. Most of the time, it comes down to one of these:

  1. The class name isn’t set correctly in your YAML.
  2. The TFRecord label mapping doesn’t match the class name your model expects.
  3. The postprocessing config doesn’t match the architecture or output format.

What to Check and Fix

1. Set Your Class Name in the YAML

Looks like your .yaml is missing the evaluation.classes section. For custom models, this needs to be defined explicitly. Here’s what it should look like:

evaluation:
  classes:
    names: ["my_custom_class"]

Make sure "my_custom_class" matches exactly what’s used in your TFRecord.

2. Check Your TFRecord Label Map

Your TFRecord should use a simple label map like this:

{
  "1": "my_custom_class"
}

The index ("1") needs to match how the class was encoded during training, and the class name must match what’s in your YAML. If these are off, all detections will be treated as background — and AP = -1.0 will follow.

3. Explicitly Set meta_arch in Postprocessing

Even though you’ve got NMS and HPP configured, it’s a good idea to explicitly declare the model architecture to avoid misalignment. Update your postprocessing section to:

postprocessing:
  meta_arch: yolov8
  device_pre_post_layers:
    nms: true
  hpp: true

This ensures Hailo runs the right postprocessing logic for YOLOv8.