Linux Implementation

From my previous researches and tries at attempting to run main() on HPX thread, I now finally implemented it and it has been successfully integrated in HPX as well. The implementation consists of using Linker flag -Wl,-wrap to wrap __libc_start_main and change the program’s entry point. More on this can be read in my previous blogs.

Mac OSX Implementation

After taking care of my Linux implementation, I went ahead to look for ways to implement the same in Mac OSX. We need to first understand why my Linux implementation is not portable enough that we need to implement it differently for different OSes.

• Every OS has its own Linker with different Linker Flags. The -wrap linker flag is not available in both Windows and Mac OSX.
• Mac OSX uses its own version of libc and __libc_start_main is a libc function provided by glibc. Therefore Mac OSX does not contain that very same function to try to hook to.

This called for an alternate way of changing the entry point of the program. That’s when I stumbled upon -e linker flag. This changes the entry point of program to user defined entry point as opposed to main(). While this flag was available in Linux as well, it served no good use as it changed the entry point of the ELF from _start to that of the user defined entry point. This would have meant that I had to implement _start first and then create the function stack altogether. Clearly -wrap provided me with better alternatives.

Things will get clearer with the following example:

Create a file (let’s call it link.cpp) and copy the following code:

Now let’s create another file (let’s call it main.cpp) which will contain our main function.

Now open up the terminal and write the following commands:

$ clang++ -c -o link.cpp.o link.cpp
$ clang++ -c -o main.cpp.o main.cpp
$ clang++ -Wl,-e,_before_main -o result link.cpp.o main.cpp.o

From the above commands we can see the use of -e linker flag. Running the generated executable should give the following output:

$ ./result
before_main
main

It is quite clear from the above output that the entry point has been changed from main to before_main. I have used this method to implement all the initializations necessary for initiating the HPX runtime.

I was able to successfully implement it and is currently undergoing review by my mentors.

Let’s now shift gears to Window’s end.

MSVC Implementation

After failing to achieve expected output with MSVC, my mentor got me in contact with MSVC team. I shared my doubts and sought out ways to implement them.

Unfortunately, MSVC does not provide any elegant way of initializing the HPX runtime system. After exchanging a few emails, we came down to 2 ways of implementing it.

First Method

Re-implementing the mainCRTStartup which is the entry point of an exe file. Implementing the function in a separate file any compiling it to an obj file and further linking it at link time will make sure that my implementation of mainCRTStartup is run as opposed to MSVC’s implementation.

While this method will make for an elegant solution, there are some serious maintenance issues that needs to be resolved. Firstly, there is a question of portability amongst different architecture. There is a need to re-write code specific to every architecture. Secondly, the code for the function changes quite often. This would mean that I will have to keep the multiple implementation of the function for different versions of MSVC Compiler.

I’m currently trying to explore this method in depth to understand its feasibility.

Second Method

It involves using #pragma init_seg. It can allow users to inject code at various steps of the program. While this looks lucrative at first glance, we cannot force it to run after all the user defined global objects have been constructed. We can run, however, run before it.

While running it before all the global objects using hpx_start will register the kernel thread with HPX thread, I would still not be able to use all the functionality that it provides. With that said, one could always use run_on_hpx_thread() since global objects and main are registered with the HPX runtime system (but are not running on it!). This method is similar to the one provided in init_globally example.

I’m exploring this method in depth as well.

What next?

While implementing for Linux and Mac OSX, I came up with an idea of having a custom init function for the HPX library. If you’re not familiar with the purpose of init function, then in short, it is responsible for initializing the .init_array section and also calling the constructors of global objects. Basically, it deals with all the initializations relating to global scope variables and objects. If we implement an init function for HPX then we could deal with certain global scope initializations and run them on HPX thread as well.

That’s what I’ll be focusing on in the future along with MSVC implementation.

Until next time, I bid you farewell!

Pre Mid term evaluation status was originally published in GSoC'18 with STE||AR GROUP on Medium, where people are continuing the conversation by highlighting and responding to this story.

First Drawback:

My previous implementation worked only for a dynamically linked executable. This meant that the executable would result in a segmentation fault in case the user tried to link it statically.

Therefore, I had to address the same with either another implementation (a ground’s up approach) or updating the my current implementation. The use of dlfcn library meant that I could not update it any further, so I continued on finding another solution. This is when I stumbled upon the -Wl,-wrap method of wrapping functions. This method worked with statically and dynamically linked executable. Thus, this was an apt replacement of my current implementation.

An example of -Wl,-wrap with wrapping __libc_start_main to change the entry point is given as follows:

Step 1: Create a file named wrap_libc.cpp with the following code in it:

With the -Wl,-wrap method, you need to prepend __wrap_ with the function name (so __libc_start_main converts to __wrap___libc_start_main).

Step 2: Compile the code to create an object file to link to another file. In your terminal type:

$ g++ -c -o wrap_libc.o wrap_libc.cpp

This should create an object file by the name of wrap_libc.o which you can later link with.

Step 3: It is now time to create a test file:

Step 4: Open up the terminal and type:

$ g++ -static test.cpp wrap_libc.o -o static_working_file

This should result in the creation of a statically linked executable. Run it to check if you get the expected output:

$ ./static_working_file

This is our Custom Entry point

main

If your output matches the above output, things have worked out. And so now, we can address the second drawback.

Second Drawback

Now that I had made things working for statically linked executable, I had to handle another major drawback. The HPX Runtime system initializes it from the custom implemented Entry Point. This would mean, that any global scope initialization using HPX functionality will result in a segmentation fault.

This needed further investigation, which leads to another libc function known as __libc_csu_init which carries on the global scope initializations. So wrapping this function to initialize the runtime system should help (or result in what is known as Initialization Sequencing Fiasco). I’m currently working on testing this method. A simple implementation can be seen here (needs HPX installed). You will need to run the makefile to generate the executable.

My current goals are to test this implementation thoroughly before integrating it with the source code (Since it will require a lot of changes in the cmake structure). I will then switch back to Windows to complete the project.

Until next week, I bid you farewell

Week 4 & 5 status was originally published in GSoC'18 with STE||AR GROUP on Medium, where people are continuing the conversation by highlighting and responding to this story.

While I was successfully able to integrate my code into HPX (see the pull request), I am still working on producing working results for MSVC. I spent the week 2 understanding the MSVC runtime system and various methods to hook into runtime functions. I succeeded in hooking into a few runtime functions, but could not seem to hook into mainCRTStartup which is responsible to initialize the C runtime system in MSVC. After clearing doubts with my mentor, I decided to leave this section as of now and work on a different case scenario for gcc, clang based compilers.

As of now, my implementation will work perfectly for any function call from C main function. But, it would not work with global scope object or any section based function (formed using the “__attribute__ ((“section_name”))”). This would simply result in a runtime failure.

A very simple example of the same is given as:

The above code will break the HPX runtime system, throwing a null id exception, asking if the code actually ran on an HPX thread (which it did not since the runtime system was yet to initialize).

I am currently figuring out ways to make HPX functionality work for global scope objects. This will be my prime focus for this week and I will attempt to implement the same.

Until then, I bid you farewell!

Week 2, 3 status was originally published in GSoC'18 with STE||AR GROUP on Medium, where people are continuing the conversation by highlighting and responding to this story.

After the Community Bonding period ended, the Coding period kicked off and with full zeal and enthusiasm, I started to write code that could run HPX functionality on Compiler’s main itself (if you didn’t get the reference, read this).

I focused on Linux and Unix based systems first. My motive was to somehow run HPX functionality directly from Compiler’s main, but that would mean that the HPX system somehow needs to get initialized before it. And so my journey began to find out ways to make this happen (this dates back to March when I was writing my proposal) and I stumbled upon a solution that involved hooking. To start off with, hooking can be described as: (as per given in Wikipedia)

A range of techniques used to alter or augment the behavior of an operating system, of applications, or of other software components by intercepting function calls or messages or events passed between software components.

So with a clear picture of what a hook is, let’s move on to the function I decided to hook into to intercept the function call to alter (or override the function) the behavior, a libc function __libc_start_main. The work of __libc_start_main is to initialize the process, call the main with appropriate arguments, and then handle the return from main. The next thing would be to understand how to create a hook. The usual method to employ a hook is by using the dlfcn.h header which provides a programming interface to dynamic loader.

A clear picture of what I’ve written above can be established by explaining the following code snippet:

Starting from line 2, we see a classic definition of the __libc_start_main function. Next from line 13 to 26 we see a rather complex code. To break it into simpler chunks, what we want to do is to load this function with the dynamic linker instead of the actual function. This can be done using dlsym() (provided in the dlfcn.h header) function which takes two parameters handler and function name. For handler we use RTLD_NEXT which returns the next object after the actual one that defines __libc_start_main and store it in a pointer real_start_main and we use __libc_start_main as the function_name.

The pointer real_start_main now contains info to initialize the process, call the main function with appropriate arguments, and handle return from main. So if we were to return real_start_main from the defined function, the code will function exactly the same as pre-implemented __libc_start_main. One thing to note here is that this pointer contains the Compiler’s main function itself. So, if we were to switch this function with our own implementation of main (like initializing_main in the code snippet) then our pointer would call that function instead, and that too with appropriate arguments! This would mean that the new entry point of the program will now change from Compiler’s main to our implementation of main.

With that said, if the entry point of our system changes to our own implementation, then we could simply initiate the HPX system and call the main function from there. This would mean that for Linux and Unix based system this implementation would suffice!

With this work done, I’m now shifting to Windows implementation and it will therefore be my focus for the next week i.e. week 2.

Until then, I bid you farewell!

Week 1 Status was originally published in GSoC'18 with STE||AR GROUP on Medium, where people are continuing the conversation by highlighting and responding to this story.

Hello World!

This year I got selected into the Google Summer of Code Program with STE||AR GROUP. My project requires me to investigate ways to start the HPX run time system with the compiler’s version of main (i.e int main).

This is first of the many blogs that I aspire to write. With the advent of Community Bonding Period, I was overwhelmed with the work. I had my End Term Examinations from 25th April to 3rd May, due to which I was a little inactive on IRC. After 3rd May, I gained momentum and started working on my Project with full enthusiasm. I made contact with my fellow GSoCers and also attended a Skype meet with my mentors. In this meet we discussed about the possible implementation of my project and the potential fall backs for the same.

An Overview of my Project

Currently the HPX functionalities can be invoked only after initiating the HPX run time environment. This can be done in one of three ways:

• Re-using the main() function as the main HPX entry point
• Supplying our own main HPX entry point while blocking the main thread
• Supplying our own main HPX entry point while avoiding to block the main thread

For further details on their implementation, click here.

Seamless integration in the first way of implementation can be done using:

#define main hpx_startup::user_main

The above macro could however result in an unexpected behavior. This is where my GSoC project comes in. My project is to seamlessly integrate the HPX run time system such that any user can use any of the HPX functions directly from Compiler’s main than to explicitly call functions to initiate the run time system.

What I’ve done so far

After the discussion with my mentor, I created a repository by the name of GSoC-experimental-code. This repository contains my experimental code that is essential to make my GSoC project a success. I will also create a repository for study material/notes.

What Next

In the next few days to come I will try to integrate my experimental code with HPX. I will also continue to learn more about hooking.

With this I bid Good-Bye!

GSoC’18 with STE||AR GROUP was originally published in GSoC'18 with STE||AR GROUP on Medium, where people are continuing the conversation by highlighting and responding to this story.