Two optimization parameters -- what is the difference

Hi, I see these two lines in model scripts:
performance_param(compiler_optimization_level=max)
model_optimization_flavor(optimization_level=2, compression_level=2)

What is the difference between “compiler_optimization_level” and “optimization_level”? or they may be the same after all? If not, could there be conflicts between values of these two, or one will overwrite another?

Thanks!

Hey @TT2024 ,

Summary of Differences and Recommendations


  1. compiler_optimization_level

    • Purpose: Controls hardware-specific optimizations during compilation.
    • Use Case: Improves hardware execution by optimizing processes like layer fusion and memory management.
    • Levels:
      • default: Balanced speed and performance.
      • max: Maximizes performance but increases compilation time.
  2. optimization_level

    • Purpose: Optimizes the model before compilation through quantization, pruning, and compression.
    • Use Case: Reduces model complexity while preserving accuracy.
    • Levels:
      • optimization_level: Adjusts the extent of model optimizations.
      • compression_level: Balances compression and accuracy.
  3. Key Differences:

    • compiler_optimization_level is used during the compilation stage, while optimization_level applies during the model optimization stage.
    • They complement each other and do not conflict.
  4. Recommendations:

    • Use optimization_level and compression_level to balance performance and accuracy during optimization.
    • Set compiler_optimization_level="max" for latency-critical applications.

Let us know if you have more questions!

Best regards,
Omria

Thank you very much for your explanation!

Where you able to change them? How? Which script do you use?