Two optimization parameters -- what is the difference

General
1

Hi, I see these two lines in model scripts:
performance_param(compiler_optimization_level=max)
model_optimization_flavor(optimization_level=2, compression_level=2)

What is the difference between “compiler_optimization_level” and “optimization_level”? or they may be the same after all? If not, could there be conflicts between values of these two, or one will overwrite another?

Thanks!

2

Hey @TT2024 ,

Summary of Differences and Recommendations

  1. compiler_optimization_level

    • Purpose: Controls hardware-specific optimizations during compilation.
    • Use Case: Improves hardware execution by optimizing processes like layer fusion and memory management.
    • Levels:
      • default: Balanced speed and performance.
      • max: Maximizes performance but increases compilation time.

  2. optimization_level

    • Purpose: Optimizes the model before compilation through quantization, pruning, and compression.
    • Use Case: Reduces model complexity while preserving accuracy.
    • Levels:
      • optimization_level: Adjusts the extent of model optimizations.
      • compression_level: Balances compression and accuracy.

  3. Key Differences:

    • compiler_optimization_level is used during the compilation stage, while optimization_level applies during the model optimization stage.
    • They complement each other and do not conflict.

  4. Recommendations:

    • Use optimization_level and compression_level to balance performance and accuracy during optimization.
    • Set compiler_optimization_level="max" for latency-critical applications.

Let us know if you have more questions!

Best regards,
Omria

3

Thank you very much for your explanation!