Introduction to Evaluate

Evaluate is a versatile tool, or say a Swiss army knife, to help you revise the default behavior of softwares. For example, you can enable ‘-O3’ as a default option of your gcc/clang. With Evaluate, you don’t need to handle with all details of Makefiles and building system any more.

Revising Makefiles and building system is complex and error-prone. You never know what stupid mistakes you will make until things end up as disasters. Skymizer ever spent two full months on figuring out performance issues of an open source project, and found it just simply failed to pass the compiler correct flags. We once made a lot of changes on Nijia of Android just for applying different compiler flags on different source code. What we learned is, fighting with building system is interesting in the beginning but purely painful at last.

Spent your time with your family, not with revising Makefiles.

And, to save your time — your life — we released Evaluate.

Install Evaluate

Please approach us to receive the installment of Evaluate. Don’t stop here. I promise it’s free. We just haven’t got the time to design the ”disagree” button of the agreement interface while the law requires us to do so.

So send us a mail through little.sky@skymizer.com and you will find Evaluate on your doorstep.

Install via rpm

$ sudo rpm -ivh ./evaluate-1.0–1.rpm

Install via deb

$ sudo dpkg -i ./evaluate-1.0–1.deb

After installment, the executive file of Evaluate will be under /usr/bin . As to README and other files will be under /usr/share/evaluate .

You may validate your install through

$ evaluate -h

And below will appear

Usage:
evaluate -mark-high [path of compiler]
Skymizer Evaluator version 1.0.3
http://www.skymizer.com

Congrats! It’s done!

Getting Started with Evaluate

The most popular case of using evaluate is to evaluate the opportunity of compiler flag optimization.

To help you to evaluate it, we define two concepts: high watermark (-O3) and low watermark(-O0) to calculate the performance gap. You can use the following commands to change default options of gcc/clang.

Set high/low watermark on GCC

Supposedly we are going to add -O3 options to gcc, we can use the following instructions:

Set High Watermark

$ sudo evaluate -mark-high `which gcc`

Set Low Watermark

$ sudo evaluate -mark-low `which gcc`

After each settings, you should see the following message

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
• high-mark -O3
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

That means you’ve already set the high watermark.

List Current Setting

Use -l and -list to list current settings

$ evaluate -l -verbose=3

You should see message like these:

/// */// 
……… /// ……. … … …. …. … ……… ……… … …
///////// /// //// /// *//////*////,//// */// ///////// ///////// ///*///
/// /// //// /// */// /// /// /// */// //// /// /// ////
///////// /////// ///./// /// /// /// */// ///// ///////// ///
/// /// ///. ////// /// /// /// */// .//// /// ///
///////// /// ///* //// /// /// /// */// ///////// ///////// ///
/////

Copyright © 2017 Skymizer Taiwan, Inc.
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
• high-mark -O3
locate in file://usr/bin/gcc
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Evaluate allows multiple users have different settings on the same compiler. If one user sets gcc at high watermark, and the other sets gcc at low watermark, the setting difference won’t interfere each other or results.

Clear All Settings:

Simply put -unmark as below:

$ sudo evaluate -unmark `which gcc`

Unlike listing all settings, clearing settings will interfere multiple users. Please notify others before execute this step. If you are not sure about the current status, please enter below:

$ sudo evaluate -tool `which gcc` -mark

Show more information of Evaluate:

If you want Evaluate to show more information such as how to set location or option etc. You can use -verbose and -v to tune verbose level.

$ sudo evaluate -verbose=3

The number of occurrences of -v is the verbose level. For example, this sets verbose level to 3.

$ sudo evaluate -v -v -v

-quiet will set the verbose level to zero.

“Echo” — A Simple Example

Here, we use echo to explain how Evaluate works internally.

The following is our echo script:

#!/bin/bash 
echo “I got $*”;

This script simply lists all scripts it received.

You can add number one! as default options to the echo script like this:

$ evaluate -tool ./echo.sh -mark number one!

Then echo.sh is equivalent to echo.sh [original options] number one! as we execute it.

$ ./echo.sh Skymizer 
I got Skymizer number one!

We can use evaluate -list to check all settings on the echo script.

$ evaluate -list 
/// *///
……… /// ……. … … …. …. … ……… ……… … …
///////// /// //// /// *//////*////,//// */// ///////// ///////// ///*///
/// /// //// /// */// /// /// /// */// //// /// /// ////
///////// /////// ///./// /// /// /// */// ///// ///////// ///
/// /// ///. ////// /// /// /// */// .//// /// ///
///////// /// ///* //// /// /// /// */// ///////// ///////// ///
/////

Copyright © 2017 Skymizer Taiwan, Inc.
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
• high-mark -O3
locate in file://usr/bin/gcc
• echo.sh number one!
locate in file://home/luba/examine/echo.sh
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

FAQ

  1. What is the difference between using evaluate and alias?

Alias can only adds options in front of the command lines while Evaluate adds them behind the command lines. Unfortunately, the options in the front are usually covered by the latter ones. In short, they won’t work well.

2. Is there any other techniques similar to Evaluate?

MPICH MPI CC uses similar techniques to modify compiler options. This approach is actually commonly used in open source projects. We simply release this tool to make it even much easier.

Reference

How to evaluate Skymizer Intelligent Compiler for your projects

Originally published at https://blog.skymizer.com/