I have to withdraw my statement. I didn’t notice I was working with the changes I mentioned earlier in this forum.
[my fix]
I still get the error:
spatial_reshape_sizes = [output_shape[1], output_shape[2]]
IndexError: list index out of range
@nina-vilela, do you have another idea where my error is?
Please try again from a fresh clean venv, and make sure that you install clip the recommended way:
pip install git+https://github.com/openai/CLIP.git
Also. please try exactly this snippet:
Hey @nina-vilela,
I made a fresh venv and installed the following packages:
pip install torch
pip install torch.onnx
pip install git+https://github.com/openai/CLIP.git
Then i executed your Code but i again had to adjust the opset_version = 14`
The result is this:
Hi @lukasOST
You can compile the model using one of the following options:
Hello, @lukasOST
Sorry for the late reply. I got a message from you asking “How did you get the ONNX file?”, I’ll answer it below.
I found an ONNX file that openai pretrained on FP32 at here.
I only succeed the ResNet50 Image Encoder of CLIP.
And on each step I set on this way.
[ Parsing (ONNX->HAR1) ]
hn, npz = runner.translate_onnx_model(
onnx_path,
onnx_model_name,
start_node_names=['input'],
end_node_names=['output'],
net_input_shapes={'input': [1, 3, 224, 224]})
[ (Optimization) HAR1 → HAR2 ]
I set it up as follows by referring to the config file and CLIP_CIFAR100 in hailo_model_zoo github.
I resized CIFAR100 test dataset to 224x224 and converted to .npy format for optimization.
And the model script was as below.
set_seed(seed=23)
normalization1 = normalization([122.7709383, 116.7460125, 104.09373615000001], [68.5005327, 66.6321579, 70.32316304999999])
model_optimization_flavor(optimization_level=0, compression_level=0)
model_optimization_config(checker_cfg, batch_norm_checker=False)
When I used optimization_level=3, I was running out of RAM memory, so I specified it like above.
Hey @lihis,
i tried to compile the visual part with DFC 3.28.0.
I got this error:
hailo_sdk_client.model_translator.exceptions.ParsingWithRecommendationException: Parsing failed. The errors found in the graph are:
UnsupportedShuffleLayerError in op /attnpool/Transpose_5: Failed to determine type of layer to create in node /attnpool/Transpose_5
Please try to parse the model again, using these end node names: /attnpool/MatMul_4
so i tried this code to compile it:
hn, npz = runner.translate_onnx_model(
str(onnx_path),
onnx_model_name,
end_node_names="/attnpool/MatMul_4"
)
This compiled. My question now is did i clip the rest of the graph which would come after /attnpool/MatMul_4 ?
Hi @lukasOST,
Package Version
------------------------ ----------
clip 1.0
filelock 3.16.1
fsspec 2024.10.0
ftfy 6.3.1
Jinja2 3.1.4
MarkupSafe 3.0.2
mpmath 1.3.0
networkx 3.4.2
numpy 2.1.3
nvidia-cublas-cu12 12.4.5.8
nvidia-cuda-cupti-cu12 12.4.127
nvidia-cuda-nvrtc-cu12 12.4.127
nvidia-cuda-runtime-cu12 12.4.127
nvidia-cudnn-cu12 9.1.0.70
nvidia-cufft-cu12 11.2.1.3
nvidia-curand-cu12 10.3.5.147
nvidia-cusolver-cu12 11.6.1.9
nvidia-cusparse-cu12 12.3.1.170
nvidia-nccl-cu12 2.21.5
nvidia-nvjitlink-cu12 12.4.127
nvidia-nvtx-cu12 12.4.127
onnx 1.17.0
packaging 24.2
pillow 11.0.0
pip 22.0.2
protobuf 5.28.3
regex 2024.11.6
setuptools 59.6.0
sympy 1.13.1
torch 2.5.1
torchvision 0.20.1
tqdm 4.67.0
triton 3.1.0
typing_extensions 4.12.2
wcwidth 0.2.13
Also, please double-check that you’re using this code for exporting and that everything is being done in a clean environment.
import torch
import clip
from torch.autograd import Variable
vision_arch = "RN50x4"
device = "cpu"
model, preprocess = clip.load("RN50x4", device=device)
vit = model.visual
dummy_input = Variable(torch.randn(1, 3, 288, 288, device="cpu"))
# ==================== #
# Export Model as ONNX #
# ==================== #
vit.eval()
torch.onnx.export(vit, # model being run
dummy_input, # model input (or a tuple for multiple inputs)
"clip.onnx", # where to save the model
opset_version=14, # the ONNX version to export the model to
do_constant_folding=True, # whether to execute constant folding for optimization
)
Hey @lihis,
First i tried to compile the onnx you have send me. It failed at the same point as my onnx file.
Parsing failed. The errors found in the graph are:
UnsupportedShuffleLayerError in op /attnpool/Transpose_5: Failed to determine type of layer to create in node /attnpool/Transpose_5
Please try to parse the model again, using these end node names: /attnpool/MatMul_4
Then i checkd the installed packages.
My list is way longer than yours:
Package Version
---------------------------- --------------
absl-py 2.1.0
anyio 4.6.2.post1
argcomplete 3.5.1
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asttokens 2.4.1
astunparse 1.6.3
async-lru 2.0.4
attrs 24.2.0
babel 2.16.0
beautifulsoup4 4.12.3
bleach 6.2.0
cachetools 5.5.0
certifi 2024.8.30
cffi 1.17.1
charset-normalizer 3.4.0
cloudpickle 3.1.0
coloredlogs 15.0.1
comm 0.2.2
contextlib2 21.6.0
contourpy 1.3.1
cycler 0.12.1
debugpy 1.8.8
decorator 5.1.1
defusedxml 0.7.1
disjoint_set 0.8.0
dm-tree 0.1.8
exceptiongroup 1.2.2
executing 2.1.0
fastjsonschema 2.20.0
flatbuffers 23.5.26
fonttools 4.55.0
fqdn 1.5.1
future 1.0.0
gast 0.4.0
google-auth 2.36.0
google-auth-oauthlib 1.0.0
google-pasta 0.2.0
grpcio 1.68.0
h11 0.14.0
h5py 3.12.1
hailo-dataflow-compiler 3.28.0
httpcore 1.0.7
httpx 0.27.2
humanfriendly 10.0
idna 3.10
importlib_metadata 8.5.0
ipykernel 6.29.5
ipython 8.29.0
ipywidgets 8.1.5
isoduration 20.11.0
jax 0.4.30
jaxlib 0.4.30
jedi 0.19.2
Jinja2 3.1.4
json5 0.9.28
jsonpointer 3.0.0
jsonref 1.1.0
jsonschema 4.23.0
jsonschema-specifications 2024.10.1
jupyter 1.1.1
jupyter_client 8.6.3
jupyter-console 6.6.3
jupyter_core 5.7.2
jupyter-events 0.10.0
jupyter-lsp 2.2.5
jupyter_server 2.14.2
jupyter_server_terminals 0.5.3
jupyterlab 4.2.6
jupyterlab_pygments 0.3.0
jupyterlab_server 2.27.3
jupyterlab_widgets 3.0.13
keras 2.12.0
kiwisolver 1.4.7
libclang 18.1.1
Markdown 3.7
markdown-it-py 3.0.0
MarkupSafe 3.0.2
matplotlib 3.9.2
matplotlib-inline 0.1.7
mdurl 0.1.2
mistune 3.0.2
ml_dtypes 0.5.0
mpmath 1.3.0
msgpack 1.1.0
nbclient 0.10.0
nbconvert 7.16.4
nbformat 5.10.4
nest-asyncio 1.6.0
networkx 3.4.2
notebook 7.2.2
notebook_shim 0.2.4
numpy 1.23.3
oauthlib 3.2.2
onnx 1.14.0
onnx-tf 1.10.0
onnxruntime 1.12.0
onnxsim 0.4.17
opt_einsum 3.4.0
overrides 7.7.0
packaging 24.2
pandas 2.2.3
pandocfilters 1.5.1
parso 0.8.4
pbr 6.1.0
pexpect 4.9.0
pillow 11.0.0
pip 22.0.2
platformdirs 4.3.6
prettytable 3.5.0
prometheus_client 0.21.0
prompt_toolkit 3.0.48
protobuf 3.20.3
psutil 6.1.0
ptyprocess 0.7.0
pure_eval 0.2.3
pwlf 2.3.0
py 1.11.0
py-cpuinfo 9.0.0
pyasn1 0.6.1
pyasn1_modules 0.4.1
pycparser 2.22
pydantic 1.10.8
pyDOE 0.3.8
Pygments 2.18.0
pygraphviz 1.14
pyparsing 2.4.7
python-dateutil 2.9.0.post0
python-json-logger 2.0.7
pytz 2024.2
PyYAML 6.0.2
pyzmq 26.2.0
referencing 0.35.1
requests 2.32.3
requests-oauthlib 2.0.0
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rich 13.9.4
rpds-py 0.21.0
rsa 4.9
scipy 1.13.1
Send2Trash 1.8.3
setuptools 68.0.0
six 1.16.0
sniffio 1.3.1
soupsieve 2.6
stack-data 0.6.3
sympy 1.13.3
tabulate 0.9.0
tensorboard 2.12.3
tensorboard-data-server 0.7.2
tensorflow 2.12.0
tensorflow-addons 0.23.0
tensorflow-estimator 2.12.0
tensorflow-io-gcs-filesystem 0.37.1
tensorflow-probability 0.20.1
termcolor 2.5.0
terminado 0.18.1
testresources 2.0.1
tflite 2.10.0
tinycss2 1.4.0
tomli 2.1.0
tornado 6.4.1
tqdm 4.67.0
traitlets 5.14.3
typeguard 2.13.3
types-python-dateutil 2.9.0.20241003
typing_extensions 4.12.2
tzdata 2024.2
uri-template 1.3.0
urllib3 2.2.3
verboselogs 1.7
wcwidth 0.2.13
webcolors 24.11.1
webencodings 0.5.1
websocket-client 1.8.0
Werkzeug 3.1.3
wheel 0.37.1
widgetsnbextension 4.0.13
wrapt 1.14.1
zipp 3.21.0
I am using a fresh new enviroment and deleted all global packages with pip freeze | xargs pip uninstall -y.
I am using hailo_dataflow_compiler-3.28.0-py3-none-linux_x86_64.whl with python 3.10.12 to install the DFC.
Of the packages on my list and your list, the following are different:
My version | Your version
numpy==1.23.3 | 2.1.3
onnx==1.14.0 | 1.17.0
protobuf==3.20.3 | 5.28.3
regex not installed | 2024.11.6
I also checked again with the DFC 3.29.0 and tried to compile your onnx but there i get the old error:
raise UnexpectedNodeError(f"Failed to find reshape node in format conversion layer near {self.name}.")
output_shape = consumed_vertices[-1].get_output_shapes()[0]
spatial_reshape_sizes = [output_shape[1], output_shape[2]]
return consumed_vertices, [output_shape], spatial_reshape_sizes