If you are able to train a model using your GPU, but not to optimize the model when running Hailo DFC, the issue is likely a mismatch of the CUDA/CUDNN and driver version with respect to the Tensorflow version used by the DFC. In fact, the DFC uses Tensorflow (version 2.12.0 for DFC 3.30) as backend, which has its own requirements (Build from source | TensorFlow).
When using the Docker, you are sure that you have the right CUDA/CUDNN dependencies. If you still have the issue, you can try the following:
- Installing Nvidia driver 525 outside the docker, which offers better compatibility with CUDA 11.8.
- Make sure that to install nvidia-docker2, as explained in the Hailo AI SW Suite User Guide