The binary used for analysis in this post is from a hackthebox challenge called exatlon. I will not be solving the challenge in this post but only unpacking it. So head on over to hackthebox.com, make yourself an account and grab the challenge binary exatlon so you can follow along or use your own UPX-packed binary. I hope you enjoy the read and please feel free to drop me a message with questions or criticism, both are welcome :)
I have decided to write this post about unpacking a UPX packed elf binary using GDB for two reasons. First, I struggled to find decent information regarding this technique in a linux environment as most write-ups I found were set in a Window’s environment and perhaps anyone who also is having a similar issue will find this post and learn from it. Secondly, I am hoping to cement the knowledge for myself and add to my ever-growing collection of work, as the main intention of this blog is personal.
The big issue is that I do not know how to start out writing this. After successfully extracting the binary, I began procrastinating, which has made the task of writing this blog more stressful by the day. Sometimes,
it can be hard to just sit down and start writing when faced with just a blank page. To break this cycle, I have decided to just start free-writing this and see where it takes me.
I am going to assume that anyone reading this already understands at a basic level what a packer does and that
the main goal when dealing with a packed binary is to recover the original entry point (OEP). After we have found the OEP, we then need to extract it. Lets start by talking about how we can find the OEP.
Finding the OEP requires dynamic analysis. We need to be able to understand how the packer is designed and what techniques it uses to unpack the binary and then transfer control. How can we discover these techniques? The strategy that I used for this binary, since it was stripped, was to load it up in GDB and monitor the systems calls that were being made. You can also, if you would like, statically analyze the packing stub (the code that handles the unpacking routine) and search for those calls.
To monitor the system calls I made use of GDB’s catchpoint functionality. If you are unfamiliar with what a catchpoint is then to explain briefly, a catchpoint will break at a syscall and also break upon returning from that syscall. You can set a catchpoint like so:
It is also worth mentioning at this point that the packing stub has no actual main(), so GDB wont be able to just break on main.
When working with binaries such as this it is useful to tell GDB to break upon the first instruction by using the starti command like so
So, now that we have a break on the first instruction and our catchpoint set lets just step through the binary while keeping note of the syscalls being used. We are especially interested in any api that works with memory because the stub must allocate a new page in the binary’s memory, handle the permissions of that page, write the origional binary to it, and then ultimately transfer control over. At the end of this tutorial I will give you the fast way of handling UPX packed binaries specifically with one single catchpoint. But first lets let the binary run and observe what is happening, since a great deal of reverse engineering is observing the binary in action, generating a hypothesis and then testing that. You can let the binary run by simply using the ‘continue’ keyword.
So now we have hit our first syscall, which is open():
You can see that the first instruction inside the syscall open() is located at 0x00000000004918ed in memory. It is important to remember that this address is the address of the first byte inside the open function NOT the address of the instruction to make the syscall. We can view that instruction by using the examine command but instead of examining instructions at the address stored in rip, we will examine instructions starting 30 bytes before rip like so:
x/5i $rip-30
and we can see the instruction syscall as well as the instructions before it. How did I know to make the offset -30 from rip? Well, I just played around until I got an offset that I liked. Sorry but not much more magic than that. You have to be willing to accept that you are walking through a forest of unkown trees and you need to be curious enough to examine those trees and be ok with having an exploratory mindset.
So, here is what instructions we have right before the syscall:
This is pretty cool but lets put on our thinking cap here for a moment and try and think of a more efficient way of investigating the parameters of this
syscall. First, we know that we have stopped execution right at the entry point of open() so all of our registers will be the same as they were when the call
was made. For those of you new to reverse engineering, I like to think of the binary as a new universe to explore and the registers contain the state of that
universe. Getting to know the registers is vital.
OK, lets have a look at these registers by issuing the command i r (info registers):
Lets also have a look at the docs for the syscall open, which can be found here: https://man7.org/linux/man-pages/man2/open.2.html Get use to this process in general and it will make your life easier.
We can see that the signature for open() is:
int open(const char *pathname, int flags)
According to the docs the int that is returned is a file descriptor or handle to the file that was opened. The man page provides further information as to the flags available and the mode.
Now, in order to make sense of the information that is held in the registers we need to know the calling convention used by this binary as well as the linux sytem call table for the architecture that we are on. For finding such information, we can do one of three things:
- Use the file utility
- Use the readelf utility with -headers flag
- Just examine the bytes of the executable’s header manually in GDP (which we will do later)
Lets start with the readelf -headers command. Output is as follows:
I cut off some of the output for brevity but what you see first is extracted from the elf header in this binary. You can find the actual structure of an elf header as well as information about the complete structure of an elf file here: https://man7.org/linux/man-pages/man5/elf.5.html
From the elf header we can see that the OS/ABI (application binary interface) is UNIX — GNU. This tells us that this binary was compiled to be run on a UNIX OS and the ABI used is the GNU ABI, which is really a C++ ABI. You can find more detailed information here:
https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html
Because we know know the ABI that is used, and we have located the docs for that ABI, we can now determine the calling convention, which will tell us what registers are used when making a call to any function. You can find the docs here: https://itanium-cxx-abi.github.io/cxx-abi/abi.html#calls
I strongly encourage you to bear through reading these docs. I am only going into detail on this so that you can understand how to go about investigating a binary that you haven’t seen before.
So after reading the docs we know that function calls will follow the base C calling convention, which for us will be the System V ABI since we are on
x86 architecture. After scouring through docs we will find that on a 64-bit platform parameters to functions are passed in the registers rdi, rsi, rdx, rcx, r8, r9, and further values are passed on the stack in reverse order.
Now we could’ve taken a short cut here by just looking at the linux system call table and that will tell us which register the parameters for each system call
are stored in. But again, it is just as important to understand where these conventions come from and how we can research them. The more we understand, the better off we will be.
The linux system call tables can be found here: https://blog.rchapman.org/posts/Linux_System_Call_Table_for_x86_64/ This blog post is awesome and worth reading. If you skip over the information at the top you will really miss out on a lot. Anyways, back to syscall open().
You can see from the table that a pointer to the pathname is passed in rdi, the flags are passed in rsi, and the mode is passed in rdx.
Lets examine the first two of those registers to get our parameters to open(). First we will look at what rdi points to, since it contains an address. We can do this with x/s $rdi as such:
We see that it contains the string “/proc/self/exe”. I'm not going to get into the details of the proc filesystem in this post but do please go and research it. Basically this points back to the executable that we are in right now.
Lets check the value inside rsi.
It’s 0. But wait! When we read the docs on open() we saw only a bunch of macros! Correct. We need to investigate the header file to see what macro represents the value 0 in order for us to determine the flag used.
Taking a step back to the man page for open() we see this include statement #include <fcntl.h>. So the macros we are interested in will be defined in fcntl.h. After finding the source code for the header file we see that 0 is represented by the macro O_RDONLY: #define O_RDONLY 00000000. So now we know the second parameter passed to open().
Going back to our original signiture we can gather that the equivalent C code here would be something like:
int fd = open(“/proc/self/exe”, O_RDONLY);
Now how do we get the value returned by this call? Well according to the ABI, any value returned by a function call will be stored in the rax register. So we
simply continue and, because we are using catchpoints, we will break right after the return from open(). We can then examine the register to find that rax
contains the value 3.
I hope the above steps made sense. I wanted to show how you can go about investigating a syscall. You can apply this same methodology to any syscall that you come across. I’m not going to walk through each syscall in this binary like I did the open() call, that’s for you to do, but I will share how I took note of them when I was stepping through. Below are the syscalls that this stub made:
1.open
2.mmap
1. returned pointer to mapped area 0x7ffff7f66000
2. this is repeated twice in a row
3.mprotect
1. starting address 0x7ffff7ff7000
2. length 0x116b (4459)
3. set the memory protections to read
4.readlink
1.passing in /proc/self/exe into buffer* 0x7fffffffd3e0
5.mmap
1. returned pointer to mapped area at 0x400000
2. called twice on this address
3. possible ELF header beginning
6.mprotect
1. sets the above range memory protections to read
7.mmap
1.Creating mapping starting at 0x401000 with length 0x149661 (1349217)
2.Setting protections (rbx) to 0x7 or read, write and execute
8.mprotect
1.Changing the protection on the above memory region to 5 or read and execute
9.mmap
1. creating memory at 0x54b000 with length 0x51e57 (335447)
2. prot is set to 3 — read and write
10.mprotect
1. Changing protection on the above memory region to 0x1 — read
11.mmap
1. creating memory at 0x59e000 with length 0xc610
2. setting protections to read and write
12.mprotect
1. setting protections to read and write on above memory map
13.mmap
1.creating memory map starting at address 0x5ab000 for length 0x3ca8 with read and write prot
14.mmap
1.creating memory map with rdi value of 0x0 for length 0x1000 with read prot
15.close
1.closing the opened fd — 3
16.munmap
1.unmapping memory region 0x7ffff7f66000
a. see note 2
As you can see I just made some generic notes. Some more detailed than others. Since we aren’t interested in understanding how the packing algorithm works on a deeper level, I felt fine with that level of detail.
By looking at the trace of calls we can see a pattern of mmap() => mprotect() and then a close() to close out the file we opened in the beginning and then
finally munmap.
Now we are at the point that we have to start doing a little guesswork. I’m going to assume that after this munmap call the binary has been unpacked completely and the stub will now transfer control. First, lets examine all the memory regions that have been created by issuing the command info proc mappings, as such:
Take a look at the Start Addr, Size and Perms fields. You will notice that they correspond to the mappings created by mmap (which should go without saying). These are the ones that we will eventually dump because our hypothesis is that those mmap calls created the pages that the original binary was written to. But before we just dump all these regions and while we are still paused on the return from munmap, lets examine the code in the stub and see if we can spot where the control from the stub is transferred to the unpacked binary, which should be our OEP.
Lets examine the instructions following the current instruction, which is in the rip register: x/10i $rip
You can see that we are popping the value from the top of the stack into rdx and then executing a ret instruction. Lets have a look at the stack. To do so
we can just x/10w $rsp. Since we are in a 64 bit environment we know that our address space is going to be 8 bytes in length. So that pop instruction will
place 0x0000000000000000 into rdx and move the stack pointer down by 8 bytes. The ret instruction just pops whatever is on the top of the stack into rip,
which will now be 0x0000000000404990. My hypothesis at this point is that this is the OEP of the binary and the ret will be transferring control. If this is
correct then we should have some instructions at this address. Lets take a look with x/10i 0x0000000000404990.
Ok! Looks like we have valid instructions. Also notice that the address we are returning to falls within the 0x401000–0x54b000 page, which has read and execute permissions. Do you see now how we are testing our hypothesis and confirming?
So what about all those other regions?
Well lets make another hypothesis based on the structure of an elf file. If we found the above range to contain instructions, then the region above it should be all of our headers. Lets examine that region. If it is indeed an elf header then we know from the man pages that the first field in the struct ElfN_Ehdr
is an array of size 16 called e_ident. So we should be able to examine 16 bytes at 0x4000000 and see at the least our magic byte that defines this as an elf
file.
x/16x 0x4000000
We can see that the first byte should correspond with the macro EI_MAGO. In this case we can see that it is 0x7f, which is indeed the magic number for an elf file. The next 3 bytes should spell ELF, which they do. Now lets grab the entry point listed in the header to confirm the entry point we found earlier, which was the address loaded into rip at the ret instruction(0x00404990).
The entry point will be stored in the e_entry field of the struct. We can calculate the offset to this member by adding the size of each member in bytes like so:
1.e_ident[16] — 16 bytes
2.e_type — 2 bytes
3.e_machine — 2 bytes
4.e_version — 4 bytes
Total — 24 bytes
So since the member right after e_version is e_entry, which is what we want, we should be able to grab this value by accessing the address
of the struct (0x4000000) + 24 bytes. Lets try.
x/2w 0x4000000+24 (We are examining 8 bytes (2 words) because we know that this member contains an address in the 64-bit arch, which will have a width of 8 bytes)
Remember, we are in little-endian so reverse the bytes and we get 0x0000000000404990, which is the same as the address we found at that ret instruction. So now we can be even more confident about the OEP that we found. You can go on to examine the rest of the header in this manner but we are just going to jump straight into extracting this binary. Also, if you wanted to, you could find the elf header of the packing stub from the very beginning of this tutorial by looking at memory page 0x7ffff7f65000–0x7ffff7f66000. Why is it here? Well, notice that the first call to mmap returned us a pointer to
0x7ffff7f66000? Well, my assumption here is that how the packer’s unpacking process works is by first writing a copy of itself in memory, otherwise it
would just overwrite all the code in the stub during the unpacking process.
Ok! Lets dump these memory regions and get for ourselves the original binary.
We are interested in the following regions:
0x400000 0x401000 0x1000 0x0 r — p
0x401000 0x54b000 0x14a000 0x0 r-xp
0x54b000 0x59d000 0x52000 0x0 r — p
0x59e000 0x5af000 0x11000 0x0 rw-p
Did you notice that we skipped 0x59d000–0x59e000? This is because I have already done some playing around and the executable will not run if we include this region. Also, if you take note of all the regions created by the mmap calls, you will see that this is not one of them. I’m not sure what this region is for. My best guess is that it is left over from the unpacking process.
To dump these regions from inside GDB we simply issue the following commands:
dump binary memory dump1 0x400000–0x401000
dump binary memory dump2 0x401000–0x54b000
dump binary memory dump3 0x54b000–0x59d000
dump binary memory dump4 0x59e000–0x5af000
You will now have 4 files containing binary data: dump1, dump2, dump3, and dump4. Simply cat all those together like so:
cat dump1 dump2 dump3 dump4 > extractedBinary
That’s it! Make sure to chmod +x extractedBinary so that you can execute it.
At the beginning of this post I mentioned that I would show an easier way than just stepping through all of the syscalls. Now that we understand
how to find the OEP of a UPX packed binary, we can just simply set a catchpoint on the munmap call like so: catch syscall munmap. Or of course you can always just use the upx utility to unpack it for you but what fun is that?
As a follow-up exercise to anyone wishing to improve, I would challenge you to write a little script that unpacks this for you.
