Error while compiling the optimized file for converting tflite model to HEF file


i have followed the below steps
Step1: Training,I have trained my custom dataset and obtained tensorflow model(.h5).Then i converted .h5 to .tflite (without quantization).
This is the training code which i have used,

import os
import numpy as np
import xml.etree.ElementTree as ET
from PIL import Image
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, BatchNormalization, ReLU, MaxPooling2D, Flatten, Dense
from tensorflow.keras.optimizers import Adam
from sklearn.model_selection import train_test_split

Define image size

IMAGE_SIZE = (320, 320)

def parse_annotation(xml_path):
# Parse XML file to extract annotations
tree = ET.parse(xml_path)
root = tree.getroot()
for obj in root.findall(‘object’):
# Extract class label
class_name = obj.find(‘name’).text
if class_name == ‘person’:
return 1 # Person is present
return 0 # Person is not present

def preprocess_image(image_path):
# Preprocess image: resize and normalize
img = Image.open(image_path).convert(‘RGB’)
img = img.resize(IMAGE_SIZE)
img_array = np.array(img) / 255.0
return img_array

def load_data(image_dir, annotation_dir):
image_files = [f for f in os.listdir(image_dir) if f.endswith(‘.jpg’)]
X, y = ,

for image_file in image_files:
    img_path = os.path.join(image_dir, image_file)
    xml_path = os.path.join(annotation_dir, image_file.replace('.jpg', '.xml'))
    
    if not os.path.exists(xml_path):
        continue
    
    img_array = preprocess_image(img_path)
    label = parse_annotation(xml_path)
    
    X.append(img_array)
    y.append(label)

X = np.array(X, dtype=np.float32)  # Ensure data type consistency
y = np.array(y, dtype=np.float32)
return X, y

Define the model

def create_model():
model = Sequential()

# Add Conv2D layer with batch normalization
model.add(Conv2D(32, (3, 3), padding='same', input_shape=(320, 320, 3), dtype=tf.float32))
model.add(BatchNormalization(dtype=tf.float32))
model.add(ReLU())
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3), padding='same', dtype=tf.float32))
model.add(BatchNormalization(dtype=tf.float32))
model.add(ReLU())
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(128, dtype=tf.float32))
model.add(BatchNormalization(dtype=tf.float32))
model.add(ReLU())
model.add(Dense(1, activation='sigmoid', dtype=tf.float32))  # Single output for binary classification

return model

Load and preprocess data

image_directory = ‘/home/ubuntu/Downloads/Dataset/person’
annotation_directory = ‘/home/ubuntu/Downloads/Dataset/annotations’
X, y = load_data(image_directory, annotation_directory)

Check if data is loaded correctly

if len(X) == 0 or len(y) == 0:
print(“No data found. Please check your directories and annotations.”)
else:
# Split the data into training and validation sets
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.1, random_state=42)

# Compile and train the model
model = create_model()
model.compile(optimizer=Adam(), loss='binary_crossentropy', metrics=['accuracy'])

model.fit(
    X_train, y_train,
    epochs=10,
    validation_data=(X_val, y_val)
)

# Save the model
model.save('/home/ubuntu/Downloads/tf_model_new.h5')
print("Model training and saving completed successfully.")

After training using above code,i have converted .h5 to .tflite
using below code

import os
import tensorflow as tf

Load the trained model

model = tf.keras.models.load_model(‘/home/ubuntu/Downloads/tf_model_new.h5’)

Ensure all layers use float32

for layer in model.layers:
if hasattr(layer, ‘dtype’):
layer._dtype = tf.float32

Convert the model to TensorFlow Lite format without quantization

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.target_spec.supported_types = [tf.float32] # Ensure float32 is used
tflite_model = converter.convert()

Save the TFLite model

tflite_model_path = ‘/home/ubuntu/Downloads/tf_model_new.tflite’
with open(tflite_model_path, ‘wb’) as f:
f.write(tflite_model)

print(f"TFLite model saved at: {tflite_model_path}")

The training part and conversion from .h5 to .tflite i have done in different environment not hailo environment

step2: I am using ubuntu

Hi @shreeyaj444,

To facilitate our investigation, could you please send me the model file through a direct message?

Thanks!

Hi, I have mailed the model files and queries i have regarding those files.
please let me know if you have received the model files and please let me know the solution.
Thank you in advance

Hello,

Thank you for your response,First of all i have two model files which i did training under two circumstances… so please i want solutions for both…I have attached both model files with their respective names as i have mentioned below.If u require any other information please let me know.

1.detect_11.tflite : For this i have done normal training in collab without using any batch normalization techniques during training process.I did parsing successfully but during optimization process I am getting error saying "This error raises when the data or weight range are not balanced. Mostly happens when using random calibration-set/weights, the calibration-set is not normalized properly or batch-normalization was not used during training." So my question is should we compulsorily do Batch normalization during training ?.

Sometimes i do get error saying input shapes are not matching…the error goes like this "hailo_model_optimization.acceleras.utils.acceleras_exceptions.BadInputsShape: Data shape (11, 320, 320, 3) for layer detect_11/input_layer1 doesn’t match network’s input shape (320, 320, 3)…should i ensure that input shapes matches for both model file and calibration.npy file?

2.new_model_4.tflite : For this i have done batch normalization during training,I did this because it was giving me error for the previous detect_11.tflite as mentioned above.For this model i am getting stuck at last step before converting it to HEF file i.e complier step.The error goes like this "[error] Mapping Failed (allocation time: 1s)
Can’t find mutual format for fc1_d3 → ew_add1_ew_add_n_fc1

[error] Failed to produce compiled graph
[error] BackendAllocatorException: Compilation failed: Can’t find mutual format for fc1_d3 → ew_add1_ew_add_n_fc1".
For this i dont know how to proceed.

Hi @shreeyaj444,

I haven’t received the model. To send the file to me through a dm, please open my profile and you will have a “message” button there.

Thanks for sending the files, @shreeyaj444.

Regarding your question:

It seems that the issue with the quantization of detect_11.tflite might be related to the lack of input normalization.

If the code you shared for preprocessing the training set was indeed used for training detect_11, then the input normalization with a mean of [0, 0, 0] and a standard deviation of [255, 255, 255] should also be applied during both calibration and inference. You can add these operations to the device by including the following command in the model script (.alls):

normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])

As for the error:

Unfortunately, one of the fully connected layers you’re using is currently not supported.

Could you provide more details about the task for this model? Have you considered starting with a well-known architecture that’s already pre-trained for similar tasks?

The task of my model is to object detection i.e person in my case , it should be able to detect in every light conditions and even in occlusion.I am using tensorflow lite framework and architecture EfficientNet.

I want to use custom dataset and custom trained model. I guess i cannot go for pre-trained models as Im afraid detection wont happen accurately for the conditions i have mentioned above. And for the framework i want to use only tensorflowlite because it is lightweight so that i can interface with RPI5.

Can u give me any solution so that i can move forward with the conditions i have stated above?
Just want to mention,I dont have in-depth knowledge about CNN like connected layers error which i have got.I have just followed the Document which Hailo has provided in this link Dataflow Compiler v3.28.0 and sometimes if CLI commands doesnt work i am using the code provided in jupiter notebook in visual studio and running as a python code combining all as one code.

We have EffficientNet as part of our officially supported models, and we also have EfficientDet, which is used for detection. It seems like the issue is in the changes you have made to the model.

Re-training a pre-trained model is a good idea because the weights start from a point that is better than random. We have a variety of training dockers that we offer for re-training, you can check the list here.

Hello ,
I have installed Hailo AI Software suite-self extractable in ubuntu not the Docker one.So,how should i proceed further ?Should i install Docker inside Hailo_env or outside? can u give me guidence please.

Hi @shreeyaj444,

I would recommend using our Journey to explore next steps:
https://hailo.ai/developer-zone/journey-develop-a-sw-for-hailo-8/

There you will also find a link to the documentation of the Suite, which contains the installation steps.

Sorry,i didnt find the journey documentation.Are u saying me to look at hailo model zoo user guide?And if i have to install and use hailo model zoo in hailo AI suite…should i connect hailo-8 device physically to my desktop with ubuntu?As i have already fixed my hailo8 device to rpi5 through pcle driver…should i train and test my custom dataset in rpi5 only?

make sure to be logged in to the developer zone before trying to open the link

yes i have logged in

The suite includes all of our SW packages - HailoRT fw, ModelZoo, TAPPAS and DFC.

The only thing missing is the pcie driver which needs to be installed outside of the container.

The pcie driver is only relevant for inference and it’s not necessary for re-training the model. It is important to notice that the Hailo device is not used for training, only inference.

ok got it.I thought documentation named journey is there…
So i can also re-train the pre-trained model using hailo model zoo through hailo ai suite only right? i dont need to again install docker right?

or if i have to install docker…should i do it outside or inside the hailo_env?

Hello i have retrained the pre-trained yolov8s model using my custom dataset successfully.I have exported it to onnx also. But in last step when i am compiling my onnx model using model_zoo by this command
“hailomz compile --ckpt best_320.onnx --calib-path /home/ubuntu/Documents/hailo_ai_sw_suite/sources/model_zoo/training/yolov8/calibration_dataset/ --yaml /home/ubuntu/Documents/hailo_ai_sw_suite/sources/model_zoo/training/yolov8/Yolo_annotate/config.yaml”.
I am getting this error

I generated a separate config.yaml file using the below code by following the instruction given in this link hailo_model_zoo/docs/YAML.rst at master · hailo-ai/hailo_model_zoo · GitHub
The code to generate config.yaml is

import onnx
import yaml

def get_input_output_nodes(onnx_model):
input_nodes = [node.name for node in onnx_model.graph.input]
output_nodes = [node.name for node in onnx_model.graph.output]
return input_nodes, output_nodes

def generate_yaml(onnx_file, yaml_file):
# Load the ONNX model
model = onnx.load(onnx_file)

# Extract node details
input_nodes, output_nodes = get_input_output_nodes(model)

# Prepare YAML content
yaml_content = {
    'network': {
        'network_name': 'yolov8s'  # Update if necessary
    },
    'paths': {
        'network_path': [onnx_file],
        'alls_script': None
    },
    'parser': {
        'nodes': [input_nodes[0], output_nodes],  # Start and end nodes
        'normalization_params': {
            'normalize_in_net': False,
            'mean_list': [123.68, 116.78, 103.94],
            'std_list': [58.395, 57.12, 57.375]
        },
        'start_node_shapes': {
            input_nodes[0]: [1, 3, 320, 320]  # Adjust as necessary
        }
    },
    'preprocessing': {
        'network_type': 'object_detection',
        'meta_arch': 'yolov8',  # Update if applicable
        'padding_color': 114
    },
    'quantization': {
        'calib_set': ['/path/to/calibration_dataset/'],  # Update path if necessary
        'calib_set_name': None
    },
    'postprocessing': {
        'meta_arch': 'yolov8',  # Update if applicable
        'postprocess_config_file': None,
        'device_pre_post_layers': {
            'bilinear': False,
            'argmax': False,
            'softmax': False,
            'nms': True
        }
    },
    'evaluation': {
        'dataset_name': 'your_dataset_name',  # Update if applicable
        'data_set': '/path/to/evaluation_dataset.tfrecord',  # Update path if necessary
        'classes': 1,
        'labels_offset': 0,
        'network_type': 'object_detection'
    },
    'hn_editor': {
        'yuv2rgb': False,
        'flip': False,
        'input_resize': {
            'enabled': False,
            'input_shape': [320, 320]
        },
        'bgr2rgb': False
    }
}

# Write YAML content to file
with open(yaml_file, 'w') as f:
    yaml.dump(yaml_content, f, default_flow_style=False)

print(f"YAML configuration file '{yaml_file}' has been generated.")

Example usage

onnx_file = ‘/home/ubuntu/Documents/hailo_ai_sw_suite/sources/model_zoo/training/yolov8/best_320.onnx’ # Update with your ONNX model path
yaml_file = ‘/home/ubuntu/Documents/hailo_ai_sw_suite/sources/model_zoo/training/yolov8/Yolo_annotate/config.yaml’ # Update with the desired output YAML path
generate_yaml(onnx_file, yaml_file)

I will be sharing the link for config.yaml which i used for compiling and dataset.yaml which i used for training. So i have used two different yaml files for training and compiling respectively.

I will also share the link for my onnx model.

Please tell me where i have gone wrong and give me solution.

If u want any additional information please do ask.
Thankyou

I tried using the yolov8s.yaml provided in the model_zoo by running this command
“hailomz compile --ckpt best_320.onnx --calib-path /home/ubuntu/Documents/hailo_ai_sw_suite/sources/model_zoo/training/yolov8/calibration_dataset/ --yaml /home/ubuntu/Documents/hailo_ai_sw_suite/sources/model_zoo/hailo_model_zoo/cfg/networks/yolov8s.yaml”

I am getting this error

please provide solution for this error also.

Your start and end node names are incorrect. They should be:
image

The screenshot is taken of netron.app, a good tool for visualizing models.