Running hashcat v4.0.0 in Amazon’s AWS new p3.16xlarge instance

Amazon released their new GPU rigs a couple of days ago. The top of the line options is the p3.16xlarge instance. Hashcat released v4.0.0 yesterday, I decided to make them play together.

This is the first time a Tesla based GPU is faster than the top GTX models for crypto operations. It is truly fast — 633.7 GH/s for NTLM. And expensive — $24.48 per hour.

Before giving you the required steps that are needed in order to run hashcat on p3.16xlarge, here are some notable benchmarks. The full list can be found at the end of this post.

• NetNTLMv2: 28912.2 MH/s
• MD5: 450.0 GH/s
• SHA-256: 59971.8 MH/s
• MS Office 2013: 163.5 kH/s
• bcrypt $2*$, Blowfish (Unix): 434.2 kH/s

Setting up p3.16xlarge for hashcat

First of, you may need to request a limit increase so that you are allowed to launch the new p3.16xlarge instances. AWS was pretty responsive, it only took them 20 mins or so.

Before we proceed a friendly heads up: I will not be replying to any angry emails coming from people who lost their job/spouse/sanity. Be careful with your internet clicks. If you forget to spin-down your p3.16xlarge instance for a month, you will be reminded with an AWS invoice of over $18k.

p3.16xlarge is available hey in US East (N. Virginia), US West (Oregon), EU West (Ireland) and Asia Pacific (Tokyo) regions.

Launch an a default Ubuntu p3.16xlarge instance with the “Ubuntu Server 16.04 LTS (HVM), SSD Volume Type — ami-cd0f5cb6” ami.

You can try getting a spot request one, these are usually 50%-80% cheaper, depending on availability and demand.

On your first login, update everything and install the required packages:

sudo apt-get update && sudo apt-get install -y build-essential linux-headers-$(uname -r) p7zip-full linux-image-extra-virtual

Then, sudo vi /etc/modprobe.d/blacklist-nouveau.conf and input the following:

blacklist nouveau
blacklist lbm-nouveau
options nouveau modeset=0
alias nouveau off
alias lbm-nouveau off

After:

echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf
sudo update-initramfs -u
sudo reboot

bash script for non-interactive automated deployments:

Once the instance reboots (it should take 2–3 minutes), download the latest NVIDIA Tesla drivers (these worked for me) and hashcat:

wget http://us.download.nvidia.com/tesla/410.104/NVIDIA-Linux-x86_64-410.104.run
sudo /bin/bash NVIDIA-Linux-x86_64-410.104.run --ui=none --no-questions --silent -X

wget https://hashcat.net/files/hashcat-5.1.0.7z
7za x hashcat-5.1.0.7z

You should be good to go. To verify that the drivers are working you can run sudo nvidia-smi

output of nvidia-smi

Full benchmarks: