hailomz compile: ValueError: Tried to convert 'input' to a tensor and failed. Error: None values not supported.

tommymcphillips · December 4, 2024, 9:01am

Hi guys, I facing the following problem, I retrained a model yolov8s with my custom dataset (licence-plates).

Compiling my onnx file with this yaml hailo_model_zoo/cfg/networks/yolov8s.yaml file I having some issue

this is my command
hailomz compile --ckpt /local/shared_with_docker/yolov8s.onnx --calib-path /local/shared_with_docker/licence-plate/train/images --yaml hailo_model_zoo/cfg/networks/yolov8s.yaml

Error:

This are my outputs from neutron en in the image

Im not sure what im doing wrong

chansheunglong · December 15, 2024, 9:20am

I am having the same or similar issue, I trained my own model base on Yolov8m following the instruction from hailo-rpi5-examples/doc/retraining-example.md at main · hailo-ai/hailo-rpi5-examples · GitHub. The training is success and I can run it on a Nvidia GPU using Yolo in Python, however when I try to run the hailomz compile, I got an error:

ValueError: Tried to convert 'input' to a tensor and failed. Error: None values not supported.
Call arguments received by layer "yolov8_nms_postprocess" (type HailoPostprocess):
  • inputs=['tf.Tensor(shape=(None, 80, 80, 64), dtype=float32)', 'tf.Tensor(shape=(None, 80, 80, 3), dtype=float32)', 'tf.Tensor(shape=(None, 40, 40, 64), dtype=float32)', 'tf.Tensor(shape=(None, 40, 40, 3), dtype=float32)', 'tf.Tensor(shape=(None, 20, 20, 64), dtype=float32)', 'tf.Tensor(shape=(None, 20, 20, 3), dtype=float32)']

chansheunglong · December 15, 2024, 9:43am

Here is the complete terminal output:

<Hailo Model Zoo INFO> Start run for network yolov8m ...
<Hailo Model Zoo INFO> Initializing the hailo8l runner...
[info] Translation started on ONNX model yolov8m
[info] Restored ONNX model yolov8m (completion time: 00:00:00.28)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.12)
[info] NMS structure of yolov8 (or equivalent architecture) was detected.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.22/cv2.0/cv2.0.2/Conv /model.22/cv3.0/cv3.0.2/Conv /model.22/cv2.1/cv2.1.2/Conv /model.22/cv3.1/cv3.1.2/Conv /model.22/cv2.2/cv2.2.2/Conv /model.22/cv3.2/cv3.2.2/Conv.
[info] Start nodes mapped from original model: 'images': 'yolov8m/input_layer1'.
[info] End nodes mapped from original model: '/model.22/cv2.0/cv2.0.2/Conv', '/model.22/cv3.0/cv3.0.2/Conv', '/model.22/cv2.1/cv2.1.2/Conv', '/model.22/cv3.1/cv3.1.2/Conv', '/model.22/cv2.2/cv2.2.2/Conv', '/model.22/cv3.2/cv3.2.2/Conv'.
[info] Translation completed on ONNX model yolov8m (completion time: 00:00:01.49)
[info] Saved HAR to: /home/ubuntu/Desktop/hailo_convertion/yolov8m.har
<Hailo Model Zoo INFO> Using generic alls script found in /home/ubuntu/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov8m.alls because there is no specific hardware alls
<Hailo Model Zoo INFO> Preparing calibration data...
[info] Loading model script commands to yolov8m from /home/ubuntu/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov8m.alls
[info] Loading model script commands to yolov8m from string
[info] Starting Model Optimization
[warning] Reducing optimization level to 0 (the accuracy won't be optimized and compression won't be used) because there's less data than the recommended amount (1024), and there's no available GPU
[warning] Running model optimization with zero level of optimization is not recommended for production use and might lead to suboptimal accuracy results
[info] Model received quantization params from the hn
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/hailodfc/bin/hailomz", line 33, in <module>
    sys.exit(load_entry_point('hailo-model-zoo', 'console_scripts', 'hailomz')())
  File "/home/ubuntu/hailo_model_zoo/hailo_model_zoo/main.py", line 122, in main
    run(args)
  File "/home/ubuntu/hailo_model_zoo/hailo_model_zoo/main.py", line 111, in run
    return handlers[args.command](args)
  File "/home/ubuntu/hailo_model_zoo/hailo_model_zoo/main_driver.py", line 250, in compile
    _ensure_optimized(runner, logger, args, network_info)
  File "/home/ubuntu/hailo_model_zoo/hailo_model_zoo/main_driver.py", line 91, in _ensure_optimized
    optimize_model(
  File "/home/ubuntu/hailo_model_zoo/hailo_model_zoo/core/main_utils.py", line 326, in optimize_model
    runner.optimize(calib_feed_callback)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 2093, in optimize
    self._optimize(calib_data, data_type=data_type, work_dir=work_dir)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 1935, in _optimize
    self._sdk_backend.full_quantization(calib_data, data_type=data_type, work_dir=work_dir)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py", line 1045, in full_quantization
    self._full_acceleras_run(self.calibration_data, data_type)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/sdk_backend.py", line 1229, in _full_acceleras_run
    optimization_flow.run()
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py", line 306, in wrapper
    return func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py", line 326, in run
    step_func()
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py", line 250, in wrapped
    result = method(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py", line 123, in parent_wrapper
    self.build_model()
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/tools/orchestator.py", line 250, in wrapped
    result = method(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/flows/optimization_flow.py", line 240, in build_model
    model.compute_output_shape(shapes)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/acceleras/model/hailo_model/hailo_model.py", line 1039, in compute_output_shape
    return self.compute_and_verify_output_shape(input_shape, verify_layer_inputs_shape=False)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/acceleras/model/hailo_model/hailo_model.py", line 1073, in compute_and_verify_output_shape
    layer_output_shape = layer.compute_output_shape(layer_input_shapes)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/keras/engine/base_layer.py", line 917, in compute_output_shape
    outputs = self(inputs, training=False)
  File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/tmp/__autograph_generated_file_wj_ekob.py", line 41, in tf__call
    outputs = ag__.converted_call(ag__.ld(self).call_core, (ag__.ld(inputs), ag__.ld(training)), dict(**ag__.ld(kwargs)), fscope)
  File "/tmp/__autograph_generated_filehmooo37f.py", line 90, in tf__call_core
    ag__.if_stmt(ag__.ld(self).postprocess_type in [ag__.ld(PostprocessType).NMS, ag__.ld(PostprocessType).BBOX_DECODER], if_body_3, else_body_3, get_state_3, set_state_3, ('do_return', 'retval_'), 2)
  File "/tmp/__autograph_generated_filehmooo37f.py", line 22, in if_body_3
    retval_ = ag__.converted_call(ag__.ld(self).bbox_decoding_and_nms_call, (ag__.ld(inputs),), dict(is_bbox_decoding_only=ag__.ld(self).postprocess_type == ag__.ld(PostprocessType).BBOX_DECODER), fscope)
  File "/tmp/__autograph_generated_file2uuxdpzr.py", line 99, in tf__bbox_decoding_and_nms_call
    ag__.if_stmt(ag__.ld(self).meta_arch in [ag__.ld(NMSOnCpuMetaArchitectures).YOLOV5, ag__.ld(NMSOnCpuMetaArchitectures).YOLOX], if_body_4, else_body_4, get_state_4, set_state_4, ('decoded_bboxes', 'detection_score', 'do_return', 'retval_', 'inputs'), 4)
  File "/tmp/__autograph_generated_file2uuxdpzr.py", line 96, in else_body_4
    ag__.if_stmt(ag__.ld(self).meta_arch == ag__.ld(NMSOnCpuMetaArchitectures).YOLOV5_SEG, if_body_3, else_body_3, get_state_3, set_state_3, ('decoded_bboxes', 'detection_score', 'do_return', 'retval_'), 4)
  File "/tmp/__autograph_generated_file2uuxdpzr.py", line 93, in else_body_3
    ag__.if_stmt(ag__.ld(self).meta_arch == ag__.ld(NMSOnCpuMetaArchitectures).YOLOV8, if_body_2, else_body_2, get_state_2, set_state_2, ('decoded_bboxes', 'detection_score'), 2)
  File "/tmp/__autograph_generated_file2uuxdpzr.py", line 69, in if_body_2
    (decoded_bboxes, detection_score) = ag__.converted_call(ag__.ld(self).yolov8_decoding_call, (ag__.ld(inputs),), None, fscope)
  File "/tmp/__autograph_generated_filewapk051t.py", line 82, in tf__yolov8_decoding_call
    decoded_bboxes = ag__.converted_call(ag__.ld(tf).expand_dims, (ag__.ld(decoded_bboxes),), dict(axis=2), fscope)
ValueError: Exception encountered when calling layer "yolov8_nms_postprocess" (type HailoPostprocess).

in user code:

    File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/acceleras/hailo_layers/base_hailo_none_nn_core_layer.py", line 45, in call  *
        outputs = self.call_core(inputs, training, **kwargs)
    File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/acceleras/hailo_layers/hailo_postprocess.py", line 123, in call_core  *
        is_bbox_decoding_only=self.postprocess_type == PostprocessType.BBOX_DECODER,
    File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/acceleras/hailo_layers/hailo_postprocess.py", line 157, in bbox_decoding_and_nms_call  *
        decoded_bboxes, detection_score = self.yolov8_decoding_call(inputs)
    File "/home/ubuntu/anaconda3/envs/hailodfc/lib/python3.10/site-packages/hailo_model_optimization/acceleras/hailo_layers/hailo_postprocess.py", line 367, in yolov8_decoding_call  *
        decoded_bboxes = tf.expand_dims(decoded_bboxes, axis=2)

    ValueError: Tried to convert 'input' to a tensor and failed. Error: None values not supported.


Call arguments received by layer "yolov8_nms_postprocess" (type HailoPostprocess):
  • inputs=['tf.Tensor(shape=(None, 80, 80, 64), dtype=float32)', 'tf.Tensor(shape=(None, 80, 80, 3), dtype=float32)', 'tf.Tensor(shape=(None, 40, 40, 64), dtype=float32)', 'tf.Tensor(shape=(None, 40, 40, 3), dtype=float32)', 'tf.Tensor(shape=(None, 20, 20, 64), dtype=float32)', 'tf.Tensor(shape=(None, 20, 20, 3), dtype=float32)']
  • training=False
  • kwargs=<class 'inspect._empty'>

tommymcphillips · December 15, 2024, 6:43pm

I found the solution of this in this thread Problem With Model Optimization - #25 by Nadav

what you need to do is to add this line to the alls script
quantization_param([conv42, conv53, conv63], force_range_out=[0.0, 1.0])

for this do the following search this file in your docker container

github.com

hailo-ai/hailo_model_zoo/blob/master/hailo_model_zoo/cfg/alls/generic/yolov8m.alls

normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
model_optimization_config(calibration, batch_size=2)
change_output_activation(conv58, sigmoid)
change_output_activation(conv71, sigmoid)
change_output_activation(conv83, sigmoid)
post_quantization_optimization(finetune, policy=enabled, learning_rate=0.000025)
nms_postprocess("../../postprocess_config/yolov8m_nms_config.json", meta_arch=yolov8, engine=cpu)

add the line before nms line and save the file
quantization_param([conv42, conv53, conv63], force_range_out=[0.0, 1.0])

in the yaml file that you are passing through argument in the compile, for example this one
hailomz compile --ckpt yolov8m.onnx --calib-path /local/shared_with_docker/licence-plate/train/images/ --yaml hailo_model_zoo/cfg/networks/yolov8m.yaml --classes 1

modify the hailo_model_zoo/cfg/networks/yolov8m.yaml in your docker container

github.com

hailo-ai/hailo_model_zoo/blob/master/hailo_model_zoo/cfg/networks/yolov8m.yaml

base:
- base/yolov8.yaml
postprocessing:
  device_pre_post_layers:
    nms: true
  hpp: true
network:
  network_name: yolov8m
paths:
  network_path:
  - models_files/ObjectDetection/Detection-COCO/yolo/yolov8m/2023-02-02/yolov8m.onnx
  alls_script: yolov8m.alls
  url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov8m/2023-02-02/yolov8m.zip
parser:
  nodes:
  - null
  - - /model.22/cv2.0/cv2.0.2/Conv
    - /model.22/cv3.0/cv3.0.2/Conv
    - /model.22/cv2.1/cv2.1.2/Conv
    - /model.22/cv3.1/cv3.1.2/Conv

This file has been truncated. show original

and change this

paths:
  network_path:
  - models_files/ObjectDetection/Detection-COCO/yolo/yolov8m/2023-02-02/yolov8m.onnx
  alls_script: yolov8m.alls
  url: https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ObjectDetection/Detection-COCO/yolo/yolov8m/2023-02-02/yolov8m.zip

alls_script: /the/path/to/the/file/that/you/changed/previusly

and re run the hailomz compile

you can find more information in the post shared.

chansheunglong · December 16, 2024, 5:06pm

Thanks for the info, I changed both the YAML and alls files, re-run the hailomz compile but the output still returned the same ValueError.

ValueError: Exception encountered when calling layer “yolov8_nms_postprocess” (type HailoPostprocess).
ValueError: Tried to convert ‘input’ to a tensor and failed. Error: None values not supported.

I am unsure if the issue might be because I am using a trained model instead of teaching from 0 in the halio docker container. But I assume any valid onnx should be able to be compiled using the tool.

Might need tech staff member from Hailo to check on this.

Here are my full command

hailomz compile --ckpt /home/ubuntu/Desktop/dataset/runs/detect/train5/weights/best_ir8.onnx --hw-arch hailo8l --calib-path /home/ubuntu/Desktop/dataset/test/images/ --yaml /home/ubuntu/hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov8m_fixed.yaml --classes 1

If anyone would like to have the onnx file to have a try, please let me know

tommymcphillips · December 16, 2024, 6:31pm

from what I understand is that your model should have specific output that the post process nms is looking for, you can check in the file

github.com

hailo-ai/hailo_model_zoo/blob/master/hailo_model_zoo/cfg/postprocess_config/yolov8n_nms_config.json

{
	"nms_scores_th": 0.2,
	"nms_iou_th": 0.7,
	"image_dims": [
		640,
		640
	],
	"max_proposals_per_class": 100,
	"classes": 80,
	"regression_length": 16,
	"background_removal": false,
	"bbox_decoders": [
		{
			"name": "yolov8n/bbox_decoder41",
			"stride": 8,
			"reg_layer": "yolov8n/conv41",
			"cls_layer": "yolov8n/conv42"
		},
		{
			"name": "yolov8n/bbox_decoder52",

This file has been truncated. show original

[
		{
			"name": "yolov8n/bbox_decoder41",
			"stride": 8,
			"reg_layer": "yolov8n/conv41",
			"cls_layer": "yolov8n/conv42"
		},
		{
			"name": "yolov8n/bbox_decoder52",
			"stride": 16,
			"reg_layer": "yolov8n/conv52",
			"cls_layer": "yolov8n/conv53"
		},
		{
			"name": "yolov8n/bbox_decoder62",
			"stride": 32,
			"reg_layer": "yolov8n/conv62",
			"cls_layer": "yolov8n/conv63"
		}
	]

you can check with https://www.neutron.app/ the output of your model, but if you model dont have 3 outputs yolov8letter/convTheNumber it will fail in the post-process

you are right the solution that I gave you is for some error when we are retraining and these outputs (yolov8x/convNumbers) don’t appear.

and maybe it’s ok that a model that you grab don’t have that ouputs and maybe it’s possible to add to that model theses output for the nms process but not sure how to do it, @Nadav maybe can clarify, why a model dont have this outputs, and how we can added it.

tommymcphillips · December 16, 2024, 9:24pm

this is the app sorry https://netron.app/

Topic		Replies	Views
Compile custom ONNX models on Hailo8 General hailo8	1	58	December 12, 2024
Compiling custom onnx to hef on hailo8 General hailo8	1	66	December 12, 2024
Conversion error from yolov8n.onnx to HEF file General	18	1101	August 21, 2024
Help-YOLOv8n Custom Model: Training to Hailo HEF Conversion Failure General	4	111	November 23, 2024
Failing to compile onnx to hef General hailort , raspberry-pi , hailo8	3	36	January 14, 2025

hailomz compile: ValueError: Tried to convert 'input' to a tensor and failed. Error: None values not supported.

Related topics