Hailo Optimization Fine-Tuning not using GPU?

General
1

Hi,

I am using the Hailo DFC with optimization level 2 to optimize my Yolox detection model with the model script outlined below. I am not getting a warning message that would indicate that Hailo cannot detect the GPU (I’ve seen [warning] Reducing optimization level to 0 (the accuracy won’t be optimized and compression won’t be used) because there’s no available GPU in other posts). Still, the fine-tuning stage does not allocate any VRAM which indicates that it is not using the GPU at all (which it should, right ? ). I would greatly appreciate some help with this! Thanks!

model_optimization_config(calibration, batch_size=16, calibset_size=99996)
post_quantization_optimization(finetune, policy=enabled, dataset_size=99996, batch_size=16, epochs=10, learning_rate=0.0001)
nms_postprocess("nms.json", yolox, engine=cpu)
2

Hi @stwerner,
What is the system that you’re using (OS version, GPU model, DFC version)?
Also, are you using the Docker or standalone whl?

In addition, we highly reccomment to refrain from changing the calibset size, as in here:
model_optimization_config(calibration, batch_size=16, calibset_size=99996)

This will cause optimization issues down the road.

1 Like
3

Hi @Nadav , thanks for the response !

I have installed the standalone wheel in a a docker container.

  • Ubuntu 22.04.5 LTS
  • NVIDIA H100 GPU (Driver Version 550.144.03, CUDA Version 12.4)
  • Hailo DFC version 3.28.0

I could switch to another GPU Architecture (A100s and T4s) if need be. I cannot change the driver versions (since this is managed GPU infrastructure in a cluster). Thanks for the advice.

Best Regards

4

EDIT: In some cases the GPU is also a MIG instance (e.g. H100 40GB MIG)

5

Okay, I’ve fixed the issue. I changed some things around in my docker build. I’ve changed the source image to use a different CUDA version and am using Ubuntu 20.04 LTE now. Also, using the Hailo Docker image as a source image works fine for me.

1 Like