OpenAI Releases 1.5 Billion Parameter GPT-2 Model
In a blog post today OpenAI today announced the final staged release of its 1.5 billion parameter language model GPT-2, along with all associated code and model weights.
GPT-2 is a large language model that can generate realistic paragraphs of text. Since its debut this February, GPT-2 has attracted both praise for its performance and concerns regarding its potential misuse for example in generating fake news stories. The latter fears initially prompted Open AI to take the unusual step of withholding the language model’s code and training dataset. Over the past nine months, OpenAI researchers have made the following observations regarding GPT-2:
1. Humans find GPT-2 outputs convincing.
2. GPT-2 can be fine-tuned for misuse.
3. Detection is challenging.
4. We’ve seen no strong evidence of misuse so far.
5. We need standards for studying bias.
Even with no task-specific training data, GPT-2 still reaches state-of-the-art performance on machine translation, question answering, reading comprehension, and summarization.
When the GPT-2 model debuted in February, OpenAI told Synced it was withholding the model’s code and training dataset: “it’s clear that the ability to generate synthetic text that is conditioned on specific subjects has the potential for significant abuse.” OpenAI’s decision came under fire in the AI community, as did the company’s decision the following month to launch a new, for-profit division, OpenAI LP. The San Francisco-based research organization started in 2015 as a non-profit company with a goal of achieving a safe and socially beneficial artificial general intelligence. The company said its for-profit division was a necessary move to balance huge compute and AI talent costs, but some, like software engineer Brian Baliat, suspected an ulterior motif: “I now see why you didn’t release the fully trained model of #gpt2.”
OpenAI says it is now releasing the GPT-2 model to “aid the study of research into the detection of synthetic text, although this does let adversaries with access better evade detection.”
The AI community reacted quickly to today’s PT-2 release. Within an hour, machine learning engineer Adam King had updated his GPT-2 powered interactive text generating website: “The ‘too dangerous to release’ GPT-2 text generator is finally fully released! I’ve put it up on http://TalkToTransformer.com for those who like to live dangerously.”
The GPT-2 output detection model is available on GitHub, and OpenAI’s report on potential malicious use can be downloaded in pdf.
Journalist: Fangyu Cai | Editor: Michael Sarazen
We know you don’t want to miss any stories. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.
Need a comprehensive review of the past, present and future of modern AI research development? Trends of AI Technology Development Report is out!
2018 Fortune Global 500 Public Company AI Adaptivity Report is out!
Purchase a Kindle-formatted report on Amazon.
Apply for Insight Partner Program to get a complimentary full PDF report.
