Youtube GPT, start a chat with a video

In this article, I will show you how YouTube GPT works

YouTube GPT is a tool that allows you to transcribe videos and then start a conversation about the content of the video.

Note that to run Youtube GPT you must have credits in your OpenAI account.

Demo

Start a chat with this video of Microsoft CEO Satya Nadella’s interview.

Youtubegpt - a Hugging Face Space by davila7

Code GPT

All the code of this project was written with Code GPT 💪

Visit https://codegpt.co to get the extension

To start you must have an OpenAI account and add your API Key, then paste the URL of any video on Youtube.

In this example I am using the following Streamlit video:

Click on Start Analysis, so that Youtube GPT performs the transcription of the video with OpenAI Whisper.

OpenAI Whisper Transcription

The transcript will be uploaded in the “Transcription” tab.

# Get the video 
youtube_video = YouTube(youtube_link)
streams = youtube_video.streams.filter(only_audio=True)
mp4_video = stream.download(filename='youtube_video.mp4')
audio_file = open(mp4_video, 'rb')

# whisper load base model
model = whisper.load_model('base')

# Whisper transcription
output = model.transcribe("youtube_video.mp4")

OpenAI Embedding with text-embedding-ada-002

In the same process, the “embedding” of each segment of the video is carried out. You can see each segment with its respective embedding tokens in the “Embeddings” tab

# Embeddings
segments = output['segments']
for segment in segments:
    openai.api_key = user_secret
    response = openai.Embedding.create(
        input= segment["text"].strip(),
        model="text-embedding-ada-002"
    )
    embeddings = response['data'][0]['embedding']
    meta = {
        "text": segment["text"].strip(),
        "start": segment['start'],
        "end": segment['end'],
        "embedding": embeddings
    }
    data.append(meta)
pd.DataFrame(data).to_csv('word_embeddings.csv')

OpenAI GPT-3

And finally, you can start asking questions about the video in the “Chat with the video” tab.

completions = openai.Completion.create(
    engine = "text-davinci-003",
    prompt = one_shot_prompt,
    max_tokens = 1024,
    n = 1,
    stop=["Q:"],
    temperature=0.5,
)
message = completions.choices[0].text
return message

Github Repo: https://github.com/davila7/youtube-gpt

Follow me if you want to see more projects with this incredible technology

LatinX in AI (LXAI) logo

Do you identify as Latinx and are working in artificial intelligence or know someone who is Latinx and is working in artificial intelligence?

• Get listed on our directory and become a member of our member’s forum: https://lxai.app/
• Become a writer for the LatinX in AI Publication: http://bit.ly/LXAI-Volunteer
• Learn more on our website: http://www.latinxinai.org/

Don’t forget to hit the 👏 below to help support our community — it means a lot!

Thank you :)