Volatility 3 — Downloading Windows Symbols for Volatility 3 on Air-gapped Machines
For those who does or had done memory analysis before would most likely have heard of volatility, and are most likely using it for your own analysis work.
In 2019, Volatility 3 which is a complete re-write of the previous framework, is released. The new version aims to address many of the technical and performance challenges see in previous versions. For me, I feel that the biggest benefit of transiting to the use of Volatility 3 is that there is no need to worry about the windows profiles to be used.
Most of the time when I have the time, I usually try to hone my knowledge in malware analysis. Seldom do I try out memory analysis actually. But the few times which I tried in the past, especially when dealing with memories captured from Windows 10, I remember having the struggles of getting the right profile. Also, if the Windows 10 has the latest update, and the corresponding profiles and structures has not been released in Volatility, then I would not be able to analyse the memory unless I am able to create the structures on my own which is too technical for me xp. Imagine if this happens in a real investigation, then the memory (which may contain many significant evidence) may be deemed useless.
So when I realised, a few days ago when I try out memory analysis again, that there is a new Volatility 3, and that profiles had been done away with, I was super excited.
True enough, Volatility 3 is much more straightforward and I do not have to worry about the type of OS the memory capture was from. In order for Volatility to examine the memory capture correctly, the corresponding symbol table for the OS are needed. The symbol tables for various OS had been pre-packed into symbol table packs available for download at the github of Volatility. However, according to Volatility, the pre-packaged symbol table pack is only complete at the point of creation of the OS. Simply said, the pre-packaged symbol table pack is not complete. In fact, I tried doing analysis on a few different memory images from different Windows OS versions and of different updates, and most of the times, there are missing windows symbols.
So what happens if there is missing windows symbols?
According to the documentation on Volatility 3, for Windows systems,
“Volatility accepts a string made up of the GUID and Age of the required PDB file. It then searches all files under the configured symbol directories under the windows subdirectory. Any that match the filename pattern of <pdb-name>/<GUID>-<AGE>.json (or any compressed variant) will be used. If such a symbol table cannot be found, then the associated PDB file will be downloaded from Microsoft’s Symbol Server and converted into the appropriate JSON format, and will be saved in the correct location.”
But what if, you do not have internet connection? Obviously Volatility 3 would not be able to download the required windows symbols, and you will get the following error without having your memory capture analysed.
So what do we do? We would have to do what Volatility would do, manually. I.e. we would have to find the name of the required pdb, the pdb’s GUID and AGE, download it from Microsoft public symbol server, convert it into json and put it in Volatility’s symbols folder.
In case you need, I had described the steps in details below.
*Note: If anyone has better idea, please feel free to share.
Step 1: Run Volatility with -vvv
I am not exactly sure what -vvv does, but apparently it outputs all the debug messages of Volatility. Using this, you will be able to find out the exact windows symbols Volatility needs.
“http://msdl.microsoft.com/download/symbols” is Microsoft public symbols server. This is the location where Volatility will try to download the missing symbols from. “ntkrnlmp.pdb” is the pdb file of the symbol Volatility 3 requires, while “A641F14B11335DF730C980BC383D20291” is the combination of the GUID (first 32 characters) and AGE (last digit) of the pdb file.
Step 2: Create Manifest File
Although Microsoft do not provide offline windows symbols packages anymore, they still recognise that there are cases where one would need to retrieve symbols for file on an isolated computer.
Therefore, SymChk (a utility in Windows debugging kit), is made available, and when used with the /om parameter, it creates a manifest file that describes the files for the symbols to be retrieved. Using SymChk again, but with the parameter /im instead, it takes in the Manifest file and downloads the required from the public server. In normal circumstances, this is what you can do. Unfortunately, for Volatility 3, SymChk with /om parameter cannot be used. Having said that, even though we cannot create the manifest file automatically, we can create our own manifest file manually.
The format of the manifest file is very simple. Each row correspond to the respective pdb file to be download. It contains the name of the pdb, the combination of GUID and AGE, and the last field an integer number which I am not sure what it represents (I just put “1”). An example of the manifest file I had created based on the required symbol file shown in Step 1 is as follows:
Step 3: Download the Required Windows Symbols PDB
To do so, move the created Manifest file onto a machine with internet connection. Make sure you have Windows debugging kit installed.
Traverse to the folder of your debugging kit. For me, it is “C:\Program Files (x86)\Windows Kits\10\Debuggers\x64”.
Then run the SymChk command as follows:
symchk.exe /im <path of manifest file> /s srv*<path to store the downloaded pdb>*https://msdl.microsoft.com/download/symbols
Note that you need to make sure the path to store the downloaded pdb exists, else the download would fail.
Step 4: Convert the PDB to JSON
The next step is to copy and move the downloaded pdb back to your isolated machine with Volatility 3.
In Volatility 3, the python script that is responsible to convert pdb to json is “volatility/framework/symbols/windows/pdbconv.py”. So feel free to use this script to convert your newly downloaded pdb to json manually. Example shown below.
Step 5: Move the converted JSON to Volatility Symbols folder
Once successfully converted, name the converted JSON to “<GUID>_<AGE>.json”. Then move the file into the respective odb folder in “volatility\symbols\windows\”.
Note: It does not matter if the file is “.json” (not compressed) or “.json.xz” (compressed). If it is compressed, Volatility would uncompressed it before using.
Step 6: Examine Your Memory Capture
Now with all things set, you can go ahead and examine your memory capture without any issues.
Although I could not really think of situations where we would need to run Volatility on an isolated machine, I still continue to write this post in case someone really needs it, as I find that there is a lack of solutions and answers online with regards to the problem I tried to address in this post. In fact, most of the solutions I found to solve the problem are just “connect your machine to internet”. But what if someone really do not want or could not connect their analysis machine with internet? For this someone, I hope my post would be of help to you. =)
~Back to geumland
