xorpd assembly riddle 0x03

Syscall59
Syscall59
Dec 5, 2018 · 3 min read

xorpd has some riddle-like pieces of assembly code here. In this post, I’ll analyze this one.

I present to you the riddle number 0x03:

sub      rdx,rax
sbb rcx,rcx
and rcx,rdx
add rax,rcx

Shall we begin our analysis?

Line 1: sub rdx, rax

This line is pretty straightforward and self-explanatory: It substracts rax from rdx and places the result on rdx. The only thing to take into consideration here is that the op could result in a negative value.

; Case 1: Let's suppose rdx = 0x20 and rax = 0x19
sub rdx,rax ; rdx = 0x20 - 0x19 = 0x07
; Case 2: Let's suppose rdx = 0x19 and rax = 0x20
sub rdx,rax ; rdx = 0x19 - 0x20 = 0xfffffffffffffff9 = -7

Line 2: sbb rcx, rcx

As we know from previous riddles this op performs subtraction with carry. From the previous operation, we’ll inherit a carry flag (CF) that will be 0 if rdx was greater or equal to rax, or 1 if rdx value was lower than rax. We can translate this operation to the following:

Non-assembly representation:
rcx = rcx — rcx - CF
rcx = 0 - CF
|--> if CF = 0 ==> rcx = 0 = 0x0000000000000000
|
|--> if CF = 1 ==> rcx = -1 = 0xffffffffffffffff
Notice that the initial value of rcx doesn't affect the end result
Assembly with some comments:
; Case 1: rdx = 0x07 and rax = 0x19, CF = 0
sbb rcx,rcx ; rcx = 0x00
; Case 2: rdx = 0xfffffffffffffff9 and rax = 0x20, CF = 1
sbb rcx,rcx ; rcx = 0xffffffffffffffff

Line 3: and rcx, rdx

The third line is using the rcx register as a mask against rdx:

; Case 1: 
; rdx = 0x07
; rax = 0x19
; rcx = 0x00
and rcx,rdx
; 0x00 & 0x07 = 0x00
; remember that 0 & x = 0
; Case 2:
; rdx = 0xfffffffffffffff9
; rax = 0x20
; rcx = 0xffffffffffffffff
and rcx,rdx
; 0xfffffffffffffff9 & 0xffffffffffffffff = 0xfffffffffffffff9;
; remember that 1 & x = x

Line 4: add rax, rcx

Fourth and last line just adds rcx and rdx.

; Case 1: 
; rdx = 0x07
; rax = 0x19
; rcx = 0x00
add rax,rcx ; rax = 0x19 + 0x00 = 0x19
; Case 2:
; rdx = 0xfffffffffffffff9
; rax = 0x20
; rcx = 0xfffffffffffffff9
add rax,rcx ; rax = 0x20 + 0xfffffffffffffff9 = 0x19

Conclusions

So, what did we end up with?

  • If rax > rdx => rax is assigned rdx’s value
  • If rax ≤ rdx => rax keeps its original value

In other words: Given two numbers we are returning the minimum. This can be translated into a high-level language as:

fn min(a,d) {
if(a > d){
a = d
}
return a
}

syscall59

Shellcode for the masses

Syscall59

Written by

Syscall59

Twitter: @syscall59 | medium.syscall59.com | syscall59@protonmail.com

syscall59

syscall59

Shellcode for the masses

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade