Emulating Ghidra’s PCode: Why/How

NSA’s Ghidra Software Reverse Engineering (SRE) Suite Logo

As reverse engineers, we spend an absurd amount of time analyzing disassembled and decompiled code trying to make sense of what’s going on. The cognitive complexity involved in this process is extremely high, leading to inevitable (and understandable) mistakes and misinterpretations.

A simple mistake and one poor assumption can snowball into a huge time sink. So any options we have to make the process less cognitively complex should be seen as a welcome opportunity. One such option is code emulation.

Reverse engineers have several options when it comes to emulation. For full-system and binary emulation, QEMU (x86, ARM, MIPS, etc) and Bochs (x86) are great choices.

In many cases, full system emulation is overkill and not worth the effort. In part, this has led to the Unicorn Engine becoming an extremely popular emulation choice among reverse engineers and security researchers in the past several years.

Let me pause and give a shout out to Radare2 and its Evaluable Strings Intermediate Language (ESIL) which I don’t mention elsewhere in this article. Despite Radare2 having a steep learning curve, ESIL is extremely easy to use and fits really well into the RE process if you’re using Radare2. Much love devs. ❤

With Unicorn, we can carve out small program slices and emulate them in isolation, tracking state with each emulated instruction. This allows us to initialize our own values for registers and memory to observe how sequences of instructions function as a whole.

You might be wondering why we don’t debug code instead. In some cases setting up a debugger can be faster than using an emulator, and always faster in terms of execution time. In other cases, debugging can be infeasible. For example, when analyzing the firmware of a complex embedded system with no exposed debugging pins, analyzing a targetted algorithm in a large software suite with numerous dependencies, processes and components, or analyzing compiled libraries without access to software that imports and uses it.

While QEMU, Bochs, and Unicorn are all great emulation choices, they require breaking out of your normal RE process. This means reverse engineers typically need to develop testing harnesses in a sort of Frankenstein's Monster approach to get specific test cases running. For the most part, this is OK, but this causes a disconnection from all our hard work in variable naming, symbol identification, and inline comments we’ve added in our disassemblers.

With Ghidra, however, we can initialize an emulator instance, select any location, and start emulating code. We can run, pause, single step, hook functions, and modify anything we want on the fly. All while retaining access to our documented disassembly right there in front of us.

Ghidra v9.1 (released September 2019) added improved emulator support via the EmulatorHelper class¹. This gave us easy access to Ghidra’s emulation features via the API — though this isn’t yet exposed through Ghidra’s UI. If you want to use this functionality, you need to script it with Java or Python.

What exactly are we emulating?

PCode is Ghidra’s intermediate representation. It’s a register transfer language that describes the semantics of machine code instructions in a generic but explicit manner. This means it doesn’t matter if you’re analyzing x86, ARM, MIPS, SH4, or any other supported architecture — PCode describes what’s happening all the same.

Let’s consider the following x86 instruction.

SUB   ESP,0x10

This instruction subtracts 16 (0x10) from the stack pointer (ESP). This seems simple enough, but there’s a lot more going on under the hood.

In PCode, nothing is left implied. When the above instruction is “lifted” to PCode, we get the following representation.

(1)  CF = INT_LESS ESP, 16:4 
(2)  OF = INT_SBORROW ESP, 16:4 
(3)  ESP = INT_SUB ESP, 16:4 
(4)  SF = INT_SLESS ESP, 0:4 
(5)  ZF = INT_EQUAL ESP, 0:4

Yikes, that seems a lot more complicated than simple subtraction and it is. When subtraction is performed in x86 it causes flags to be updated depending on the result of that subtraction. Let’s break down what’s going on here.

1. The carry flag (CF) is set if ESP is less than 16 as an unsigned integer.
2. The overflow flag (OF) is set if subtracting 16 from ESP would produce a signed borrow.
3. ESP is set to the result of ESP minus 16.
4. The signed flag (SF) is set if ESP is less than 0 as a signed integer.
5. The zero flag (ZF) is set if ESP equals 0.

The x:y notation here means value:size. For example, 16:4 is a value of 16 as a four-byte data type.

This clearly describes what’s happening in the instruction. It’s often the case that PCode results in a one-to-many translation. Meaning you’ll often have multiple PCode operations for each native assembly instruction.

When we emulate code in Ghidra, we aren’t emulating the target architecture’s native code — because that would require developing emulation support for each and every architecture, for each and every implied operation the instructions perform.

All of this information is already tracked by the PCode translation, and it exists because it’s a critical step in the decompilation process in Ghidra’s decompiler.

Where does PCode come from?

PCode was not specifically developed for emulation, rather is was developed for Ghidra’s decompiler — the component that converts disassembled code into a C-like representation.

PCode’s pedigree can be traced back to the early 1990s in a journal article² by Norman Ramsey (University of Virginia) and Mary F. Fernandez (AT&T Labs) titled “Specification Language for Encoding and Decoding”³.

In this article, the authors present SLED, a “Specification Language for Encoding and Decoding”. This work was implemented in Ghidra in its early development, focusing on SLED’s use in reverse engineering. This would later become developed into SLEIGH³, a tongue-in-cheek name for a bigger, better version of SLED.

The distinction between SLEIGH and PCode

SLEIGH and PCode are distinct languages. SLEIGH is a processor specification language that allows developers to write new processor modules for Ghidra. So when a new processor architecture is released, a developer will define the processor’s instruction semantics using SLEIGH. The output of SLEIGH processor modules is PCode.

Getting down to emulation

Enough on the why, let’s get down to the how. The general process works like this:

1. Instantiate an EmulatorHelper
2. Setup registers and memory
3. Define a starting address
4. Run emulation until some event happens
5. Dispose of the EmulatorHelper instance

In this article we use Python to emulate PCode. Everything you see here can be done in Java. If you’d like to see examples of Java-based PCode emulation, check out Ghidra’s snippets ‘EmuX86DeobfuscateExampleScript.java’ and
‘EmuX86GccDeobfuscateHookExampleScript.java’ shipped with Ghidra.

Instantiate an EmulatorHelper

Instantiating an EmulatorHelper is as easy as importing the class and initializing it with the current program context.

from ghidra.app.emulator import EmulatorHelper
emuHelper = EmulatorHelper(currentProgram)

Note that Ghidra automatically defines `currentProgram` for all non-headless script invocations. If you plan to use this headless, you can get a reference to the current program using getState() and getCurrentProgram().

Setup registers and memory

By default, your emulation state will be completely zero. All memory and registers will contain zero. Most of the time you’ll have an idea of starting values for registers, and you’ll almost certainly need to define a stack frame for your emulation run. Let’s set up some registers.

emuHelper.writeRegister("RAX", 0x20)    emuHelper.writeRegister("RSP", 0x000000002FFF0000)    emuHelper.writeRegister("RBP", 0x000000002FFF0000)

Here we’ve set RAX to 0x20 and picked a random value for RSP and RBP to represent a collapsed stack frame. Now, let’s assume we have a buffer that needs to contain some pre-defined byte sequence. We’ll set that up now.

emuHelper.writeMemoryValue(getAddress(0x000000000008C000), 4, 0x99AABBCC)  # writes big endian  
emuHelper.writeMemory(getAddress(0x00000000000CF000), b'\x99\xAA\xBB\xCC') # writes little endian

Here we’ve used two methods of writing to memory. One writes in little-endian, the other in big-endian. Typically you’ll use writeMemoryValue when writing typed values, like char, unit32_t, etc. If you’re writing a lot of data, say from a debugger memory dump, a file, or a network packet, you’ll want to use writeMemory.

Now that we have registers and memory setup for our emulation run, we can define starting and ending addresses.

Define a starting addresses

To set the starting address for emulation, we simply write the address where we want to start to the program counter. In this case, it’s the RIP register since we’re emulating x86–64.

But wait, I thought we were emulating PCode, not native assembly? You’re right, we are. PCode abstracts operations while treating registers as a sort of variable name. So specific registers to the underlying architecture are still important to us. Ghidra does currently offer a method that allows us to generically reference the program counter on all supported architectures, as we’ll see below, using getPCRegister.

from ghidra.program.model.symbol import SymbolUtilities
def getSymbolAddress(symbolName):
    symbol = SymbolUtilities.getLabelOrFunctionSymbol(currentProgram, symbolName, None)
    if (symbol != None):
        return symbol.getAddress()
    else:
        raise("Failed to locate label: {}".format(symbolName))
mainFunctionEntry = getSymbolAddress("main")
mainFuncLong = int("0x{}".format(mainFunctionEntry), 16)
emuHelper.writeRegister(emuHelper.getPCRegister(), mainFuncLong)

There’s a lot going on here, but the last two lines are what’s really important. We first use a very janky method to obtain the address for the main function (the function we’re emulating). We then use the EmulatorHelper method getPCRegister to reference the program counter (RIP) and writeRegister to set it. At this point, we’re ready to run.

Run emulation

It’s time to run our emulation test. We just need to get a reference to a monitor (this lets us cancel emulation if we choose) and single-step until our program counter equals zero.

Since our emulation state is initialized to zero when the main function returns, it’ll end up popping zero off the stack to return from main and we’ll stop emulation there before an exception occurs (due to 0x00000000 being an invalid address).

from ghidra.util.task import ConsoleTaskMonitor
monitor = ConsoleTaskMonitor()
controlledReturnAddr = getAddress(0)
while monitor.isCancelled() is False:
        executionAddress = emuHelper.getExecutionAddress()  
        if (executionAddress == controlledReturnAddr):
            print("Emulation complete.")
            return

        # single step emulation
        success = emuHelper.step(monitor)
        if (success == False):
            lastError = emuHelper.getLastError()
            printerr("Emulation Error: '{}'".format(lastError))
            return
reg_value = emuHelper.readRegister('RAX')
print("  {} = {:#018x}".format('RAX', reg_value))

Here we use the step method exposed by EmulatorHelper to execute one instruction at a time checking to see if the program counter equals zero. When it does we stop emulating and print the return value (RAX).

Dispose of the EmulatorHelper instance

All that’s left to do is clean up the emulation state. We do this by calling the dispose method on our EmulatorHelper instance.

emuHelper.dispose()

A complete example

Here’s a complete Python example script that will emulate a function names “main” after configuring registers and memory state.

emulate_pcode.py — a complete PCode emulation example written in Python

Conclusion

Since Ghidra came out I’ve slowly been transitioning my work off of IDA Pro but find the lack of a debugger an issue for some projects. The emulation capabilities here are helping me bridge that gap in some cases until the debugger is released (fingers crossed).

While Ghidra isn’t the end all be all (I still use Binary Ninja’s amazing IL suite on a regular basis), it’s truly impressed me in so many ways. It’s clear the developers are in tune with the challenges of software reverse engineering and I’m excited to see what the future holds for Ghidra. Thanks, NSA!

Footnotes & References

[1]: Change Log Ghidra v9.1 (September 2019): “Emulator. Added improved emulation support at the API level including a simplified API exposed via the EmulatorHelper class. Sample GhidraScripts, which utilize this API, have been provided. (GT-3066)”

[2]: “Specifying Representations of Machine Instructions”, Norman Ramsey, Mary F. Fernandez, ACM Transactions on Programming Languages and Systems (TOPLAS), May 1997, Volume 19, Number 3, pgs 492–524.

[3]: SLEIGH, A Language for Rapid Processor Specification, History. From Ghidra’s documentation on the SLEIGH specification.

About the author

John Toterhi is a security researcher at Battelle Memorial Institute where he specializes in reverse engineering, vulnerability research, and tool development. When he’s not being a computer nerd, he’s probably picking up a new hobby or telling bad dad jokes. You can connect with John on LinkedIn, GitHub, Twitter, and Medium.