Abstractive Summarization of Text using OPENAI GPT — 2
Here we will discuss how to use OPEN AI GPT-2 model to generate Abstractive Summaries using the hugging face library
To generate summaries we do-not have a summarisation task to be used in the pipeline for GPT-2. We rather generate summaries using the “text-generation” task by adding the “TL;DR:” text at the end of the text string that we want to summarise.
from transformers import pipeline, set_seed
set_seed(42)
summarizer = pipeline("text-generation", model="gpt2")
ARTICLE = "New York (CNN)When Liana Barrientos was 23 years old, she got married in Westchester County, New York."
ARTILCE = ARTICLE + "TL;DR:"
ans = summarizer(ARTICLE, min_new_tokens=50, max_new_tokens=120, top_k = 2 )
print(ans)
print(ans[0]['generated_text'])You can refer the link1, link2 from hugging face for full list of parameters in the text-generation task
We can find more details about the summarization in the research paper which introduced GPT-2 “Language Models are Unsupervised Multitask Learners” link
Quoting the section about summarization from the paper “To induce summarization behaviour we add the text TL;DR: after the article and generate 100 tokens with Top-k random sampling with k = 2 which reduces repetition and encourages more abstractive summaries than greedy decoding. We use the first 3 generated sentences in these 100 tokens as the summary”
The text generation strategies that can be used whilegenerating text can be accessed in the link
References: