This article is an incomplete draft, but I figured there are some interesting snippets that could help someone.
This post documents how I set up my own low cost GPU Render Farm on Amazon Web Services.
Obviously, you first need and AWS account. The registration is pretty straight forward. You need to enter your credit card, but don’t worry, you will be only charged a dollar or so as a verification, which will be refunded. Head over to https://aws.amazon.com/ and set up an account.
AWS offers multiple services, but we will only use their EC2 service; it’s similar to renting a virtual server. There are many types of those servers, varying by amount of cores/CPUs, available RAM, network bandwidth etc. We will be using P2 instances which have NVIDIA Tesla K-80. A single P2 instance may contain up to 16 of those GPUs!
P2 Instance Pricing
P2 are AWS EC2 instances dedicated for working with GPUs. There is a simple trick to saving over 70 % costs: Spot Instances. There are many differences to traditional instances, but none effect us when used for rendering. For $0.25 / hour, we can basically rent a machine with 61 GB RAM and a single K-80. Given that retail price of K-80 is $5000, you would need to render for 20,000 hours to get even; that is over 2 years of continuous rendering. Also, there would be power costs, which at 300W are nontrivial.
There are even larger instances: namely p2.8xlarge and p2.16xlarge. Those contain 8 and 16 K-80s respectively, for linear increase in price.
Note that AWS bills per-hour. This means you pay the same price whether you terminate the instance after 10 minutes and 59 minutes. Meaning that if your scene takes an hour to render on xlarge, it will presumably take an eighth of that on 8xlarge: roughly 8 minutes. You would still pay for a full hour. The render would be faster, but also more costly. However, if the original render would take 8 hours on xlarge, you would completely render it in one hour on 8xlarge and for the same price!
Applying for P2 Instances
Most resources on AWS are available on-demand out of the box. Alas, P2 instances are presumably in high demand and are very resource constrained, so you need to issue a one-time support request to gain access to those. In the AWS Dashboard, open Services, navigate to EC2, and select Limits in the left-side panel. Click any of the Request limit increase links on top of the page. This will take you to a support ticket editor.
First thing in the support request you need to fill is a Region. It denotes a place on Earth and you should generally pick the one closest to you that actually has P2 instances. If you are on the West coast go for Oregon, on the East coast select Northern Virginia and for Europe, pick Ireland. For other continents consult these tables.
Next up you need to pick a Primary Instance Type: pick p2.xlarge. Afterwards, a new field labeled Type Limit will appear already populated with value Instance Limit, leave it at that. Finally, fill in a New limit value. It’s unlikely you will need more than 1 instance at a time, so enter 1.
Click Add another request beneath and repeat those steps for p2.8xlarge. (You don’t need to also add p2.16xlarge as it’s very costly and not available for spot requests.)
In the Use Case description, truthfully enter what you need the instances for. I submitted this:
Hi, I’m setting up a spot request based personal render farm. Thanks
The next day the ticket was resolved, my request was approved and I was given access to P2 instances.
Preparing Instance
TODO HOW TO EXEC AND CONNECT TO AWS
To fully utilise P2 instances, K-80 drivers need to be installed. AWS has great tutorial here, or you can follow this script (it involves multiple reboots so don’t just blindly paste it):
Finally, install Blender:
Rendering
Thanks to Ritwik and Ben Jaguar Marshall, I came up with this setup for rendering on GPU on P2 EC2 AWS instance:
