Decrypting SMB3 Traffic with just a PCAP? Absolutely (maybe.)

Khris Tolbert
Maveris Labs


TL;DR: Given just a PCAP of an SMB3 session, the encrypted SMB3 could be decrypted by cracking the NetNTLMv2 hash and computing the Random Session Key

While poking around on the Hacky Holiday’s Space Race CTF, I found a very interesting challenge, “Stolen Research”, revolving around recovering stolen data from a PCAP and a memory dump. The final task of the 5-part challenge requested that the challenger attempt to recover the stolen data (and later a hint was provided strongly encouraging challengers to use the provided PCAP to do so).

The challenge “Stolen Research” on the Space Race CTF

Initial observation of the PCAP shows that we, the players of this wonderful CTF, are provided a capture of setup and execution of an encrypted SMB3 session.

Screenshot displaying the Encrypted SMB3 packets

Somehow, some way, an interesting concept popped into my head. Was it possible to decrypt these packets just provided with the data in the packet? Well, there’s only one way to find!

Not going to lie, this was definitely me for a bit

So… onto this quest. Where to start? Was it even possible to decrypt SMB3 traffic? A visit to Wireshark’s wiki suggests that this was indeed possible with the Session Key (and confirmed by this link at Samba Wiki). Seems easy enough, right? Let’s see if this is an easy win by using the Session Key (session key b35056d4bb7f4382ee7aabd0f055bad0) in the packet (Edit -> Preferences -> Protocols -> SMB2 -> Secret Session Key for decryption [EDIT…]):

Screenshot of the Session ID (fe3f6db000000000)
Screenshot of the Session Key (b35056d4bb7f4382ee7aabd0f055bad0)
Insertion of Session ID and Key into Wireshark
Whomp Whomp :-/ No dice. Looks like the Session key in the packet is not able to decrypt SMB3 traffic…

Nope :-/ Ok, maybe there is more to this. Pouring through the countless documents online provided by Microsoft and elsewhere on the internet, I finally started wrapping my head around what was needed. Essentially, what Wireshark displays in the capture as the Session Key != the Session Key needed to decrypt the SMB3 traffic and that I would have the “real” Session Key. The key printed in the packet is actually the Encrypted Session Key, where as the Session.Session_Key aka Random Session Key was indeed required. Additionally, I had found that the SMB traffic was actually SMB version 3.1.1, so I would also need to ensure my version of Wireshark is at least version 3.0.0.

SMB Dialect is 3.1.1 according to Server Response

More scouring of the internet commenced. Eventually, I stumbled into someone attempting to calculate the session key for SMB3 and straight up not having a good time with it.

My assumption of that poor soul trying to mess with Session Keys in meme format

The top answer by the amazing Obaid Farooqi (thank you where ever you are!) lays out start to finish how to calculate the needed Random Session Key. Provided the following, the Random Session Key could be calculated:

-User’s password or NTLM hash
-User’s domain
-User’s username
-Key Exchange Key (Also known as the NTLMv2 Session Base Key)
-Encrypted Session Key

Most of what was required did not need computation and was provided clearly in the PCAP (domain, username, NTProofStr, and Encrypted Session Key). The other promising news is that the Key Exchange Key could be computed with knowledge of the password and/or hash of the user. In summary, the Random Session Key can be calculated by:

-Unicode (utf-16le) of password-MD4 hash of the above (This is also the NTLM Hash of the password)-Unicode(utf-16le) and Uppercase of Username and Domain/Workgroup together -Calculating the ResponseKeyNT via HMAC_MD5(NTLM Hash, Unicode of User/Domain above)-NTProofStr (can be calculated but not needed as it is present in the PCAP)-Calculating the KeyExchangeKey via HMAC_MD5(ResponseKeyNT,NTProofStr)-Decrypt the Encrypted Session Key via RC4 and the Key Exchange Key to finally get the Random Session Key

Seems like a lot to digest, so here it is again with some sample values in psuedo-code(also present in the forum answer linked above):

user= “test” 
domain= “workgroup”
password = “test”
NTProofStr = a0e42a75c54bbb0fab814593569faa22
EncryptedSessionKey = C914ADCEB0F1C32FB7C2548D8D959F01
hash = MD4(password.encode(‘utf16-le’))
# hash is 0cb6948805f797bf2a82807973b89537
ResponseKeyNT(HMAC_MD5(hash, (user.toUpper()+domain.toUpper()).encode(‘utf16-le’)))
# ResponseKeyNT is f31eb9f73fc9d5405f9ae516fb068315
KeyExchangeKey=HMAC_MD5(ResponseKeyNT, NTProofStr)
# KeyExchangeKey is fd160d4ed7f2cb38f64262d7617f23b3
RandomSessionKey = RC4(KeyExchangeKey,EncryptedSessionKey)
# RandomSessionKey is 4462b99bb21423c29dbb4b4a983fde03

Now that I had the steps logically laid out, it dawned on me that I, duh, did not have the password… yet. The authors of the challenge did state that if someone were to need to crack a password, they should use rockyou-75.txt. Using as a guide, I decided, why not, let’s crack some hashes. Required for the crack were the username, domain name, Server Challenge, and the NTLMv2 Response (which contains the NTProofStr) from the PCAP.

NTProofSTR, Domain, and Username all provided by Client Request
NTLM Server Challenge from Server Response

For this challenge, the hashcat format ended up looking like such:


Spinning up hashcat and using some rules, the password was quickly discovered … just kidding. No, actually it was not quick. Using simple rules and rockyou-75.txt did not seem to uncover the password. Eventually, using the mask processor, building a 7 digit (and 8 digit that I didn’t use) “wordlist”, plus playing with some of the hashcat rules for a bit, the password was uncovered. If you are wondering why 7 and 8 digits were chosen, I had noticed some of the hashcat rules would go up to 6 digits, and additionally, in the early days of password complexity, users would just append numbers (and a special character if needed) to get their passwords up to 14 characters. To implement, I had to use the combinator mode (-a 1) with a left-side rule (-j/--rule-left) of “c” (capitalize first letter). I tried again with “t” (toggle all), yet hashcat did not seem to crack the hash (and yes, I had --potfile-disable).

.\mp64.exe -o 3ight.txt ?d?d?d?d?d?d?d?d
.\mp64.exe -o Se7en.txt ?d?d?d?d?d?d?d
Mask Processor output. Look how large 8 digits alone is.
hashcat.exe -a 1 -m 5600 crackme.txt rockyou-75.txt Se7en.txt --rule-left=c
Screenshot of successful crack of NTLMv2 Hash for the challenge

The memory dump that associated the PCAP also confirmed the password as it was found cleartext with strings.

Oh look! It’s a bird, it’s a plane, it’s… a PASSWORD!!!

Now armed with the password, I needed to craft some python together. Using bits and pieces of the impacket code, I threw together sloppy python to calculate the Random Session Key:

Python2 I can’t quit you. You’ve gotten me through so much. If anyone wants to upgrade this to Python3, please!

Feeding the script the username, domain, password, NTProofSTR, and Encrypted Session Key from the PCAP, I was given the Random Session Key (bold for emphasis):

python2 --user administrator --domain jupiter \
--password Shuttle9812983 \
--ntproofstr c9396ea910bd92e58c60de983db0bcf7 \
--key b35056d4bb7f4382ee7aabd0f055bad0 -v
PASS HASH: f00ec4c6994a282a82cb03c0a981a014
RESP NT: 3b1c956f98de231d413af796f9fd681a
NT PROOF: c9396ea910bd92e58c60de983db0bcf7
KeyExKey: eab9e089692dd2dc6496142a78e692ad
Random SK: ba05e83790ffc59a5ada30becc4ea8c8

Inputting this into Wireshark, I was relieved to see that my hard work paid off! The SMB3 packets were decrypted and the challenge could be completed by extracting “research.png” from the PCAP:

Screenshot showing decryption was indeed possible!!!
It was then possible to export files from the encrypted session
Contents of research.png

And voila! We have the flag!

In conclusion, it is very possible to decrypt an encrypted SMB3 session from just the information in a packet capture with a little luck. During my brief research into the topic, I hadn’t seen an article on application of this technique, but if one exists I’m sorry and I would be delighted to read such an article! Additionally, not sure if this was the intended solve as the some of the other challengers released a method using Kerberos, but I had fun figuring this out nevertheless. Lastly, it should be noted that this technique of decrypting SMB3 doesn’t necessarily require knowledge of the user’s password, as knowledge of the NTLM Hash could also work.



Maveris exists to help your organization reach its fullest potential by providing thought leadership in IT and cyber so you can connect fearlessly. To learn more visit us at:



Khris Tolbert
Maveris Labs

Sometimes things break and I happen to be behind the keyboard. I’m just as confused as you are.