Setting Up a Windows 7+ Virtualbox VM for Kernel Mode Debugging

I’ve been following along with “Practical Malware Analysis” to prepare for my GREM exam, and realized that the guide for Windows debugging was not usable on a Windows 7 setup, nor one based on Virtualbox. For this reason, I decided to make my own, both to teach me and to help others. This is not a very complicated task, I wanted to amalgamate some sources into one convenient example. I also ended up figuring out some of this on my own, as no tutorial as of now covers WinDBG Preview.

Before we start, we must enable kernel debugging in the Windows Virtual Machine. This means opening an administrator command prompt within the VM and typing the following commands:

bcdedit /debug on

bcdedit /dbgsettings serial debugport:1 baudrate:115200 (assuming the port is COM1)

Next, we must shut down the VM

Then, we go into the settings for our Virtualbox Windows VM.

In the “Serial Ports” menu, we must check “Enable Serial Port”, select “Host Pipe”, uncheck “Connect to existing pipe/socket”, and enter our pipe’s name into the address.

(Author’s Note: This is the exact opposite of how it’s usually done. Normally the pipe is set up through the debugger instead of through the v irtual machine. I find this option much better for convenient snapshotting, however. Every time I open my kernel debugging snapshot, it will automatically start the pipe. This also gives me the ability to swap out, turn off, or modify the kernel debugger while the virtual machine is live)

Now, when we start our virtual machine, it will be kernel debuggable.

Setting up our debugger:

(Author’s note: Originally, I wanted to do this with the regular version of WinDBG. This does not install correctly on Windows 10, however, so I was forced to use WinDBG Preview. I’m not complaining, it turned out to be quite user friendly. WinDBG Preview has no command line option, but that’s the same for any WinDBG install on Windows 10. There is no working command line option as of now)

We may install WinDBG Preview from the Windows Store (As of now there is no direct download).

After this, we will be able to launch it from the windows start menu.

Once both WinDBG and the Windows VM are open, we can go to “File>Start Debugging” and select “Attach to Kernel”.

We can then input the name of the Virtualbox pipe and select “Pipe” and “Reconnect”

This will open us up into a gui command line window. If we are sucessful, the bottom where the console is should say “Debugee is Running”. To start the console and pause the guest, we press the “Break” button.

At this point, we will have a kd prompt at the bottom of the screen to start typing kernel commands. To unpause the Virtual Machine and close the console, we can press “Go” at any time. We may also take advantage of the other debugging options shown near the “Go” button.

Thank you for reading, this is the first of many posts I’ll be making related to Malware Analysis. Stay tuned for tutorials and analyses that will appear here in the future!

Sources:

https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/attaching-to-a-virtual-machine--kernel-mode-

https://blahcat.github.io/2017/08/07/setting-up-a-windows-vm-lab-for-kernel-debugging/

Enjoys edev, cyber forensics, hardware hacking, and RE, former CACI BIT Systems intern, GREM, Security+