[BUG] Parser exception with square output being used multiple times

General
1

Steps to reproduce:

import tensorflow as tf
from hailo_sdk_client import ClientRunner
from math import factorial

def sin_series(x, order=6):
    # Hailo doesn't support sin and cos and...
    # Hailo supports only square and fractional powers in (0, 1) so we'll build hiegher powers sequentially 
    assert order > 1

    result = x_prev = x
    x_sq = x * x
    for k in range(1, order):
        x_prev = x_prev * x_sq
        result = result + ((-1)**k / factorial(2*k+1)) * x_prev
        
    return result

pos = tf.keras.Input(shape=(2048, 1, 2), name="kps0")
ft_proj = tf.keras.layers.Conv2D(32, 1, use_bias=False)
freqs = sin_series(ft_proj(pos))

model = tf.keras.Model(
    {'pos': pos },  {'freqs': freqs}, 
    name="ff_test"
)

model_name = "ff_test"
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.target_spec.supported_ops = [
    tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
    tf.lite.OpsSet.SELECT_TF_OPS, # enable TensorFlow ops.
]

tflite_model = converter.convert() # may cause warnings in jupyter notebook, don't worry.
tflite_model_path = "models/ff_test.tflite"
with tf.io.gfile.GFile(tflite_model_path, "wb") as f:
    f.write(tflite_model)
    
# Parsing the model to Hailo format
runner = ClientRunner(hw_arch='hailo8l')
hn, npz = runner.translate_tf_model(tflite_model_path, model_name)

You should see an error with the following stack trace in the end:

FuserHelper.run_broadcast_ew(self, layers)
    172 pred0_shape = preds[0].output_shapes[index]
    174 index = preds[1].outputs.index(layer.name)
--> 175 pred1_shape = preds[1].output_shapes[index]
    177 does_apply_broadcast = FuserHelper.is_feature_broadcast(
    178     pred0_shape,
    179     pred1_shape,
    180     is_two_sided=True,
    181 ) or is_spatial_broadcast(pred0_shape, pred1_shape, is_two_sided=True)
    183 if does_apply_broadcast:

IndexError: list index out of range

From my brisk stroll through the DFC codebase, I deduced that the problem is likely because for square operation multiple outputs are not created (and as a result multiple output shapes). Also, the fact that changing x_sq = x * x to x_sq = x * (x*0.9999999) makes it work points to the problem somewhere in the code of the operation.

2

Is there any update on this issue? I found that a similar issue still exists in DFC3.30. The cause seems to be in the broadcast_ew processing, when trying to obtain the shape of the previous node that has the same output shape, there is only one value in the list due to the identical shape, resulting in an “index out of range” error.

4

Hey @user135 @v11,

Sorry you’ve been dealing with this issue. We’re aware of this error. Our recommendation is related to the SQUARE operation - it’s supported but has known issues where multiple outputs aren’t created correctly.

This aligns with your finding that changing x_sq = x * x to x_sq = x * 0.9999999 * x makes it work, as it avoids an exact square computation.

I will update on a timeline to fix this issue.