How to evaluate Skymizer Intelligent Compiler for your projects

According to our study, the bigger performance gap between O0 to O3 is, the higher performance Skymizer Intelligent Compiler can achieve eventually.

Skymizer Intelligent Compiler combines machine learning, high-performance computing and compiler technologies to find the best way to optimize your projects. Its high level concept is a sort of iterative compiler which drives performance guide optimization (PGO) and linker optimization (on linker level, not LTO). Our machine learning system studies the whole compilation process of your project, from compiler to linker, and recommends you the best optimization approach.

Evaluate tool helps you to evaluate Skymizer Intelligent Compiler for your project. We found a simple truth: the bigger performance gap between O0 to O3 is, the higher performance Skymizer Intelligent Compiler can achieve eventually. `evaluate` can change default compiler flags in a very easy way. You can use it to calculate the performance gap between O0 to O3. Moreover, you can use it to do manual compiler flags optimization as well.

The Step of Evaluation

  1. Set up a clear platform for performance evaluation.
  2. Write benchmark program or script.
  3. use evaluate to calculate the performance gap.
  • Set gcc/clang at low optimization level, -O0.
  • Compile your project and run the benchmark. now your get one benchmark score, we call it low watermark.
  • Set gcc/clang at high optimization level, O3, and then build your project again.
  • Re-run the benchmark and get the score as the low watermark.

4. Calculate performance gap

(high watermark — low watermark) / low watermark

If the performance gap > 0.3, then Skymizer Intelligent Compiler has good chance to optimize your project.

Example: scimark2

1. Download scimark2 (Assume we install scimark2 at ~/scimark2)

2. Set gcc’s default compiler flags to low watermark (-O0)

$ sudo evaluate -mark-low `which gcc`

3. Compile your program

$ cd ~/scimark2 && make clean && make

4. Calculate benchmark score

$ cd ~/scimark2 && ./scimark2 -large

5. Set default flags to -O3

$ sudo evaluate -mark-high `which gcc`

6. Compile your program again

$ cd ~/scimark2 && ./scimark2 -large

7. Receive benchmark result on the platform

$ cd ~/scimark2 && ./scimark2 -large

Here you can see the change of the Composite Score, from 607 to 1738. The difference between these two score is so called performance gap between watermarks.

8. Clean up the settings on gcc

$ evaluate -unmark `which gcc`

9. Calculate Performance Improvement

(1738–607)/607 = 1.86 is much bigger than 0.3

In this case, scimark2 has a great chance to be optimized by Skymizer Intelligent Compiler.

Up to now, Skymizer Intelligent Compiler can optimize scimark2 up to Composite Score 3560, that is 5 times better than O3, and the best is: you don’t need to change anything, neither upgrading your hardware nor revising your software.

Reference

Introduction to Evaluate

Originally published at https://blog.skymizer.com/