Building your own Docker Images like that looks like a lot of work. I am not there yet, but have in mind to take what was done here, https://github.com/aws/sagemaker-tensorflow-containers and merge it with the ones provided for the contest, https://github.com/openai/retro-contest/blob/master/setup/agent.docker
Also, I am keen to try out mxnet… so instead I will merge the following, https://github.com/aws/sagemaker-mxnet-containers in the way described above.
Edit: On further reflection, Amazon Sagemaker could be a good place to run some of the training, see https://aws.amazon.com/sagemaker/pricing/:
“As part of the AWS Free Tier, you can get started with Amazon SageMaker for free. For the first two months after sign-up, you are offered a monthly free tier of 250 hours of t2.medium notebook usage for building your models, plus 50 hours of m4.xlarge for training, and 125 hours of m4.xlarge for hosting your machine learning models with Amazon SageMaker.”