Error when trying to convert a TensorFlow regression model to Hailo

I have an existing sample regression model which I based of the Melbourne Dataset. It runs using TensorFlow, but during conversion to the Hailo extension I came across the error I have attached in the error log. I have also attached the accompanying code to this error.

https://file.io/D3VVhFdVarn0

  1. Is it possible to migrate a regression model to run on the Hailo chip, as it is said to support TensorFlow
    Here’s an example of what I was trying to do: https://www.youtube.com/watch?v=vzWbbqgjV9g

  2. Is there any documentation on this subject if possible?

  3. Do I need to run this conversion on the Pi

Thank you for any help!

Hi @seansamu.code,
I’m unable to download the files, but out recommendation is to convert the model to ONNX or TFLite format, and then try to run the Dataflow Compiler (DFC) steps on it.
You can use the tf2onnx Python package in case you want to convert it to ONNX, or use the instructions in the DFC user guide under "Common Conversion Methods from Tensorflow to Tensorflow Lite
".

Regards,

Hi Omer,

Thanks for the prompt reply.

I will attach the files, here is the main.py -


main.py

import tensorflow as tf
import pandas as pd
import numpy as np

from tensorflow.keras.optimizers import Adam

from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score

from hailo_sdk_client import ClientRunner

Path

save_path = “./model”

Hailo Hardware Architecture

chosen_hw_arch = “hailo8”

def load_data():
melb_data = ‘melb_data.csv’

base_frame = pd.read_csv(melb_data)

df = base_frame[['Rooms', 'Bathroom', 'Landsize', 'Lattitude', 'Longtitude', 'Price']].copy()
df = df.dropna()

return df

def split_data(data):
x = data[[‘Rooms’, ‘Bathroom’, ‘Landsize’, ‘Lattitude’, ‘Longtitude’]].copy()
y = data[‘Price’].copy()

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)

return x_train, x_test, y_train, y_test

def create_model(normalizer, adam_optimizer):
model = tf.keras.Sequential([
normalizer,
tf.keras.layers.Dense(64, activation=‘relu’),
tf.keras.layers.Dense(128, activation=‘relu’),
tf.keras.layers.Dense(32, activation=‘relu’),
tf.keras.layers.Dense(1)
])
model.compile(loss=‘mean_squared_error’, optimizer=adam_optimizer)

return model

def reset_weights(model):
for layer in model.layers:
if hasattr(layer, ‘kernel_initializer’) and hasattr(layer, ‘bias_initializer’):
# Reinitialize weights and biases
layer.kernel.assign(layer.kernel_initializer(tf.shape(layer.kernel)))
layer.bias.assign(layer.bias_initializer(tf.shape(layer.bias)))

def main():
# Load the data from csv
data = load_data()

# Split the Data into Training and Testing
x_train, x_test, y_train, y_test = split_data(data)

# Normalize the Data and Create Optimizer
normalizer = tf.keras.layers.Normalization(axis=-1)
normalizer.adapt(tf.convert_to_tensor(x_train))

optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)

# Create the Model
model = create_model(normalizer, optimizer)

# Fit the Model
model.fit(x_train, y_train, epochs=10)

# Predict Model and Evaluate Results
y_pred = model.predict(x_test)
print(r2_score(y_test, y_pred))

# Summary
print("------------------")
model.summary()
print("------------------")

# Save the Model
model.save(save_path+"/saved_model.keras")

# Converting the Model to tflite
model_name = "action_forecast"
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.target_spec.supported_ops = [
    tf.lite.OpsSet.TFLITE_BUILTINS,  # enable TensorFlow Lite ops.
    tf.lite.OpsSet.SELECT_TF_OPS,  # enable TensorFlow ops.
]
tflite_model = converter.convert()  # may cause warnings in jupyter notebook, don't worry.
tflite_model_path = "./model/action_forecast.tflite"
with tf.io.gfile.GFile(tflite_model_path, "wb") as f:
    f.write(tflite_model)

# Parsing the model to Hailo format
runner = ClientRunner(hw_arch=chosen_hw_arch)
hn, npz = runner.translate_tf_model(tflite_model_path, model_name)

# Reset the weights of the model
# reset_weights(model)

if name == “main”:
main()

Here is the errror :

python3 main.py
2024-10-19 10:43:38.242391: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2024-10-19 10:43:38.283544: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2024-10-19 10:43:38.283932: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-19 10:43:38.977621: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Epoch 1/10
449/449 [==============================] - 1s 1ms/step - loss: 569119735808.0000
Epoch 2/10
449/449 [==============================] - 0s 1000us/step - loss: 302432059392.0000
Epoch 3/10
449/449 [==============================] - 0s 986us/step - loss: 300883083264.0000
Epoch 4/10
449/449 [==============================] - 0s 965us/step - loss: 300806963200.0000
Epoch 5/10
449/449 [==============================] - 0s 994us/step - loss: 300236603392.0000
Epoch 6/10
449/449 [==============================] - 0s 974us/step - loss: 299848499200.0000
Epoch 7/10
449/449 [==============================] - 0s 978us/step - loss: 300602425344.0000
Epoch 8/10
449/449 [==============================] - 0s 1ms/step - loss: 299893424128.0000
Epoch 9/10
449/449 [==============================] - 0s 1ms/step - loss: 299760451584.0000
Epoch 10/10
449/449 [==============================] - 1s 1ms/step - loss: 299863048192.0000
113/113 [==============================] - 0s 840us/step
0.31117518939460487

Model: “sequential”


Layer (type) Output Shape Param #

normalization (Normalizatio (None, 5) 11
n)

dense (Dense) (None, 64) 384

dense_1 (Dense) (None, 128) 8320

dense_2 (Dense) (None, 32) 4128

dense_3 (Dense) (None, 1) 33

=================================================================
Total params: 12,876
Trainable params: 12,865
Non-trainable params: 11



2024-10-19 10:43:49.369142: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:364] Ignored output_format.
2024-10-19 10:43:49.369170: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:367] Ignored drop_control_dependency.
2024-10-19 10:43:49.369451: I tensorflow/cc/saved_model/reader.cc:45] Reading SavedModel from: /tmp/tmpfby9tuv9
2024-10-19 10:43:49.370513: I tensorflow/cc/saved_model/reader.cc:89] Reading meta graph with tags { serve }
2024-10-19 10:43:49.370527: I tensorflow/cc/saved_model/reader.cc:130] Reading SavedModel debug info (if present) from: /tmp/tmpfby9tuv9
2024-10-19 10:43:49.375186: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:353] MLIR V1 optimization pass is not enabled
2024-10-19 10:43:49.376356: I tensorflow/cc/saved_model/loader.cc:231] Restoring SavedModel bundle.
2024-10-19 10:43:49.429489: I tensorflow/cc/saved_model/loader.cc:215] Running initialization op on SavedModel bundle at path: /tmp/tmpfby9tuv9
2024-10-19 10:43:49.444673: I tensorflow/cc/saved_model/loader.cc:314] SavedModel load for tags { serve }; Status: success: OK. Took 75221 microseconds.
2024-10-19 10:43:49.470922: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var MLIR_CRASH_REPRODUCER_DIRECTORY to enable.
[info] Translation started on Tensorflow model action_forecast
Traceback (most recent call last):
File “/home/ss/Projects/test/tf_action/main.py”, line 108, in
main()
File “/home/ss/Projects/test/tf_action/main.py”, line 101, in main
hn, npz = runner.translate_tf_model(tflite_model_path, model_name)
File “/home/ss/Projects/test/tf_action/.venv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py”, line 16, in wrapped_func
return func(self, *args, **kwargs)
File “/home/ss/Projects/test/tf_action/.venv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py”, line 1222, in translate_tf_model
parser.translate_tf_model(
File “/home/ss/Projects/test/tf_action/.venv/lib/python3.10/site-packages/hailo_model_optimization/tools/subprocess_wrapper.py”, line 124, in parent_wrapper
func(self, *args, **kwargs)
File “/home/ss/Projects/test/tf_action/.venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py”, line 101, in translate_tf_model
parsing_results = self.parse_model_to_hn(
File “/home/ss/Projects/test/tf_action/.venv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py”, line 367, in parse_model_to_hn
fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
File “/home/ss/Projects/test/tf_action/.venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py”, line 98, in convert_model
self._validate_nan_values_in_params()
File “/home/ss/Projects/test/tf_action/.venv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py”, line 412, in _validate_nan_values_in_params
if hasattr(layer, “kernel”) and not layer.dynamic_weights and np.any(np.isnan(layer.kernel)):
File “/home/ss/Projects/test/tf_action/.venv/lib/python3.10/site-packages/hailo_sdk_common/hailo_nn/hn_layers/normalization.py”, line 135, in kernel
kernel, _ = self._convert_normalization_params_to_dw_params()
File “/home/ss/Projects/test/tf_action/.venv/lib/python3.10/site-packages/hailo_sdk_common/hailo_nn/hn_layers/normalization.py”, line 130, in _convert_normalization_params_to_dw_params
kernel, bias = calc_normalization_params(self._mean, self._std, self.kernel_shape)
File “/home/ss/Projects/test/tf_action/.venv/lib/python3.10/site-packages/hailo_sdk_common/numeric_utils/normalization_params.py”, line 12, in calc_normalization_params
kernel /= std.reshape(kernel_shape)
ValueError: cannot reshape array of size 5 into shape (1,1,1,1)

Hi Omer,

Do you have any example of a dnn regression model, we are trying to evaluate if we can switch some of our models to use Hailo.

I am using the melb_data.csv | Kaggle

Best Regards
SS

Omer,

This matches somewhat closely to our model -

Do you know it we will be able to run this?

Best Regards
SS

Hi Omer,

Any updates? or directions?

Best Regards
SS

Hi @seansamu.code,
You can look in our Hailo Model Explorer:

There you can see all the available Hailo models, their accuracy and performance.

Regards,