Setting Up a Windows 7+ Virtualbox VM for Kernel Mode Debugging

Elias Augusto
Feb 25, 2019 · 4 min read

I’ve been following along with “Practical Malware Analysis” to prepare for my GREM exam, and realized that the guide for Windows debugging was not usable on a Windows 7 setup, nor one based on Virtualbox. For this reason, I decided to make my own, both to teach me and to help others. This is not a very complicated task, I wanted to amalgamate some sources into one convenient example. I also ended up figuring out some of this on my own, as no tutorial as of now covers WinDBG Preview.

Before we start, we must enable kernel debugging in the Windows Virtual Machine. This means opening an administrator command prompt within the VM and typing the following commands:

bcdedit /debug on

bcdedit /dbgsettings serial debugport:1 baudrate:115200 (assuming the port is COM1)

Next, we must shut down the VM

Then, we go into the settings for our Virtualbox Windows VM.

In the “Serial Ports” menu, we must check “Enable Serial Port”, select “Host Pipe”, uncheck “Connect to existing pipe/socket”, and enter our pipe’s name into the address.

(Author’s Note: This is the exact opposite of how it’s usually done. Normally the pipe is set up through the debugger instead of through the v irtual machine. I find this option much better for convenient snapshotting, however. Every time I open my kernel debugging snapshot, it will automatically start the pipe. This also gives me the ability to swap out, turn off, or modify the kernel debugger while the virtual machine is live)

Now, when we start our virtual machine, it will be kernel debuggable.

Setting up our debugger:

(Author’s note: Originally, I wanted to do this with the regular version of WinDBG. This does not install correctly on Windows 10, however, so I was forced to use WinDBG Preview. I’m not complaining, it turned out to be quite user friendly. WinDBG Preview has no command line option, but that’s the same for any WinDBG install on Windows 10. There is no working command line option as of now)

We may install WinDBG Preview from the Windows Store (As of now there is no direct download).

It will say launch when downloaded, I downloaded mine ahead of time

After this, we will be able to launch it from the windows start menu.

Once both WinDBG and the Windows VM are open, we can go to “File>Start Debugging” and select “Attach to Kernel”.

We can then input the name of the Virtualbox pipe and select “Pipe” and “Reconnect”

This will open us up into a gui command line window. If we are sucessful, the bottom where the console is should say “Debugee is Running”. To start the console and pause the guest, we press the “Break” button.

Don’t worry about all of the NatVis stuff, that may not show up for you, I did some work beforehand

At this point, we will have a kd prompt at the bottom of the screen to start typing kernel commands. To unpause the Virtual Machine and close the console, we can press “Go” at any time. We may also take advantage of the other debugging options shown near the “Go” button.

Thank you for reading, this is the first of many posts I’ll be making related to Malware Analysis. Stay tuned for tutorials and analyses that will appear here in the future!


Elias Augusto

Written by

Enjoys edev, cyber forensics, hardware hacking, and RE, former Northrop Grumman intern, GREM, Security+

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade