Installing the requirements for GPT-2. TensorFlow, PyTorch and Transformers.

Cm Vigil April
Feb 15 · 4 min read

“I am become TensorFlow, the destroyer of environments.”

United States Department of Energy — Trinity and Beyond: The Atomic Bomb Movie

Hello fellow data enthusiasts. I’m currently working a fun AI writing project that will use GPT-2. I had a lot of heartache over installing the requirements to do this and got lost in an abundance of information that did not seem to help. So I thought I would share my success story in case it could help someone else also going through this struggle fest.

Step 1: Create a virtual environment

I am assuming you’ve already completed step 0 of having anaconda installed and in my case I am using gitbash for my command line interface. I am also running on Windows 10 and this may change things up if you are on a different system, especially if you intend to install pytorch. I’ll show how it is done in step 3. Okay disclaimer over. Lets roll.

  1. Go to your home directory in GIT BASH using $ cd which stands for ‘change directory’.
  2. Then go to Anaconda environments using $ cd anaconda3/envs if you installed your anaconda in your home directory. If it’s not there you can find where you installed anaconda using $ where conda
  3. Create a new environment. I’m calling my new environment gpt_env but you can substitute that with whatever you would like. IMPORTANT: USE PYTHON 3.7 or earlier model. 3.9 which is as of now the most up to date version of python will not work effectively with TensorFlow. 3.8 will work with TensorFlow 2 but if you intend to train gpt-2 you will need an older TensorFlow and an older python version.
    $ conda create --name gpt_env python=3.7

Step 2: Activate the new environment

Activate the new environment by using conda activate.
$ conda activate gpt_env/
You should be able to determine that we are in the new environment in your CLI. My environment appears before the username.

Step 3: Begin installing necessary libraries

I recommend installing TensorFlow first, because it has a lot of dependencies that can be messed up if the installation process is in the wrong order. I learned this the hard way, so you don’t have to… hopefully.

It is highly important to note that TensorFlow 2 (the current version of TF as of this post) does not allow for training/fine-tuning of gpt2 without some creative modifications. If you intend to fine-tune gpt2 I recommend installing TensorFlow version 1.15. More details are available here : Install TensorFlow with pip. As I mentioned above TensorFlow 2 works on Python 8 and above, while TensorFlow 1.15 works on Python 7 and below.

#If you intend to just generate text
$ pip install tensorflow

Next I installed PyTorch. PyTorch is great with gpt-2 for text generation which is why I’m using it. You will need to go to Start Locally | PyTorch and list your installation specifications.

$ pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio===0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Next I installed Transformers from huggingface.co. Here is the installation guide : Installation — transformers 4.3.0 documentation (huggingface.co).

$ pip install transformers

Step 4: Installing GPT-2

I found several methods of doing this. For my simple use case I decided to install gpt-2 on my system. It is available on the python project index here : pygpt-2-simple · PyPI. It can be installed by entering this in the CLI.

pip install gpt-2-simple

Step 5: Run Jupyter Lab, and use GPT-2 to Take Over the World!

Now you are off to the fun part! Well I hope this works for you. If you have any issues comment below and let me know!
For a test run you can follow along with this video : Language Generation with OpenAI’s GPT-2 in Python from a fellow named James Briggs. It is pretty sweet what GPT-2 can do! Good Luck!

Credits:
Jesse Tao was awesome in giving sage advise and guiding me through this process after much struggle on my end. He helped me figure this out. Check out his website at Jesse’s Data Science Journey (jesseptao.com)
And : James Briggs — YouTube

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data…

Sign up for Analytics Vidhya News Bytes

By Analytics Vidhya

Latest news from Analytics Vidhya on our Hackathons and some of our best articles! Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Cm Vigil April

Written by

Data scientist, Physicist, Rock Climbing and Running Enthusiast. A big nerd for space , sci-fi , and electricity.

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Cm Vigil April

Written by

Data scientist, Physicist, Rock Climbing and Running Enthusiast. A big nerd for space , sci-fi , and electricity.

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store