Reverse Engineering: Let’s Hash This Out

Rick Harris
Apr 4, 2016 · 3 min read

In a recent post on LinkedIn, I shared how using the strcmp() function in C, or similar functions in other languages, for password validation presented a security risk. In the discussion on this post, it was suggested that hashing out the password would resolve the issue. A new reverse engineering challenge at Holberton School put this theory to the test.

If you would like to take a shot at the ‘crackme2’ program, you can obtain the file from the Holberton School Github.

Know Your Environment

crackme2 is checking the first 9 characters of every environment variable for “Passw0rd=”.

The first step in understanding the program was discovering how to input the password. Unfortunately, it was not as simple as putting in a command line argument. After confirming this, I cracked open the Linux tool bag. The ltrace tool runs the program and listens for standard library calls. Immediately, it was clear that crackme2 was checking for an environment variable called “Passw0rd=” (The ‘o’ is the number ‘0’). To get started, I added an environment variable and moved on to understanding the program.

Two Steps Forward, One Step Back

Program is using MD5 encryption from the standard library.

To dig into what the program was doing, I called upon objdump. I used the -j flag to only output the .text section, the -M flag to set the syntax to Intel, and the -d flag to disassemble the program. Initially the code indicated a lot happening in the program. However, one particular section that stood out was the calling of MD5_Init, MD5_Update, and MD5_Final. Instead of moving forward with investigating the Assembly code, I took a step back to look at ltrace now that a password was in place.

When Passwords Come Calling

This time ltrace picked up a lot more activity, but the critical answer was found just near the bottom. A strcmp() call containing two MD5 encrypted values. The first value was the encrypted password, and the second was my encrypted input.

Encrypted password was output during a strcmp function call and uncovered using ltrace.

After retrieving the encrypted string, I used Google to return MD5 decryption sites. According to the site, the decrypted hash value was ‘qwerty’. Quick and simple. To ensure this wasn’t a red herring, I had to test the password in the program itself. I set the environment variable equal to ‘qwerty’, launched the crackme2 program, and the result was ‘Access Granted’. The program was successfully hacked.

Showing The Hand

Encrypting passwords is good step toward securing data. MD5 encryption converts all strings to 16 two-character hexadecimal values (32 total characters, 128 bits). As a result, using time variances to identify the password one character at a time is ineffective since the full string would only validate if the correct password was entered. For example, if you encrypt the string ‘qwert’ it returns ‘a384b6463fc216a5f8ecb6670f86456a’ which is nowhere near the same as ‘d8578edf8458ce06fbc5bb76a58c5ca4’ (the encrypted version of ‘qwerty’). One missing character produces a completely different string. However, showing the hand using a function like strcmp that outputs readable text (even if it is encrypted) undermines the security of the program.


12 claps
Rick Harris

Written by

Full-stack Software Engineer | @holbertonschool Student | Fan of the @FightingIrish | Opinions represented are my own.