How To: Deploy GPT2 NLG with Flask on AWS ElasticBeanstalk
At CaliberAI we use GPT2 natural language generation to aid our data annotation. We have been running a Python Flask app using Hugging Face’s Transformers on DigitalOcean. That setup was fairly simple; start a server, configure Ubuntu and Apache, deploy the Python Flask code, and leave it alone. If it stopped working I would just restart the server or SSH in and look at the logs. This is no way to run a service for the long term and as we scaled up our data annotation it became clear we needed a more resilient setup. We could have kept on using DigitalOcean but around the same time we also received some Amazon Web Services credits and realised Elastic Beanstalk could provide all of the production goodness we needed.
Now if you read the AWS Elastic Beanstalk guides you’ll think this is a very simple task and it should all just work. However we encountered quite a learning curve in getting from a hello world Python Flask app to a Python Flask app running a GPT2 machine learning model. In the end the setup is not complicated and it taught us some important Elastic Beanstalk concepts. You can see the working code here;
