Installing the requirements for GPT-2. TensorFlow, PyTorch and Transformers.
“I am become TensorFlow, the destroyer of environments.”
Hello fellow data enthusiasts. I’m currently working a fun AI writing project that will use GPT-2. I had a lot of heartache over installing the requirements to do this and got lost in an abundance of information that did not seem to help. So I thought I would share my success story in case it could help someone else also going through this struggle fest.
Step 1: Create a virtual environment
I am assuming you’ve already completed step 0 of having anaconda installed and in my case I am using gitbash for my command line interface. I am also running on Windows 10 and this may change things up if you are on a different system, especially if you intend to install pytorch. I’ll show how it is done in step 3. Okay disclaimer over. Lets roll.
- Go to your home directory in GIT BASH using
$ cdwhich stands for ‘change directory’.
- Then go to Anaconda environments using
$ cd anaconda3/envsif you installed your anaconda in your home directory. If it’s not there you can find where you installed anaconda using
$ where conda
- Create a new environment. I’m calling my new environment gpt_env but you can substitute that with whatever you would like. IMPORTANT: USE PYTHON 3.7 or earlier model. 3.9 which is as of now the most up to date version of python will not work effectively with TensorFlow. 3.8 will work with TensorFlow 2 but if you intend to train gpt-2 you will need an older TensorFlow and an older python version.
$ conda create --name gpt_env python=3.7
Step 2: Activate the new environment
Activate the new environment by using conda activate.
$ conda activate gpt_env/
You should be able to determine that we are in the new environment in your CLI. My environment appears before the username.
Step 3: Begin installing necessary libraries
I recommend installing TensorFlow first, because it has a lot of dependencies that can be messed up if the installation process is in the wrong order. I learned this the hard way, so you don’t have to… hopefully.
It is highly important to note that TensorFlow 2 (the current version of TF as of this post) does not allow for training/fine-tuning of gpt2 without some creative modifications. If you intend to fine-tune gpt2 I recommend installing TensorFlow version 1.15. More details are available here : Install TensorFlow with pip. As I mentioned above TensorFlow 2 works on Python 8 and above, while TensorFlow 1.15 works on Python 7 and below.
#If you intend to just generate text
$ pip install tensorflow #If you intend to fine-tune gpt-2
$ pip install tensorflow==1.15
Next I installed PyTorch. PyTorch is great with gpt-2 for text generation which is why I’m using it. You will need to go to Start Locally | PyTorch and list your installation specifications.
$ pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio===0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
Next I installed Transformers from huggingface.co. Here is the installation guide : Installation — transformers 4.3.0 documentation (huggingface.co).
$ pip install transformers
Step 4: Installing GPT-2
I found several methods of doing this. For my simple use case I decided to install gpt-2 on my system. It is available on the python project index here : pygpt-2-simple · PyPI. It can be installed by entering this in the CLI.
pip install gpt-2-simple
Step 5: Run Jupyter Lab, and use GPT-2 to Take Over the World!
Now you are off to the fun part! Well I hope this works for you. If you have any issues comment below and let me know!
For a test run you can follow along with this video : Language Generation with OpenAI’s GPT-2 in Python from a fellow named James Briggs. It is pretty sweet what GPT-2 can do! Good Luck!