The emergence of A.I. for text generation
We have witnessed artificial intelligence’s ability to generate content again and again. This time by inputting just a few words of prompt, a language model experiment, TalkToTransformer.com, can complete your sentences and create coherent paragraphs of text one word at a time. This article is a part of the blog series, A.I. for content generation, where we’ll take a look at some highlights in A.I. and its ability to create various forms of content ranging from written text to audio-visual forms.
TalkToTransformer.com was released on November 5, 2019, by Adam D. King as his latest experiment. It’s a great demonstration of OpenAI’s new machine learning model dubbed GPT-2, which was released mid-February this year. King states that his experiment runs on the full-sized GPT-2 model, which is a successor of the previous three smaller scale, less coherent versions of the model.
It’s another form of machine learning that combines human- and machine-written text. King has prepared eight prompts as an exemplary starting point, but you can also have a bit of fun with it by customizing your own prompts.
Predicting the next word you type, but not limited to just that
The principle of the language model is simple. The basic objective is to generate paragraphs of sentences by just typing a few words. It essentially aims to predict the next word considering the context it was given in the previous words. OpenAI marks this as a huge achievement in the development of modern neural networks, where a large-scale unsupervised language model is able to perform tasks impressively without task-specific training.
In doing so, it was trained using abundant Internet data. Trained on datasets as large as eight million web pages, it can generate synthetic texts as outcomes across diverse domains. In other words, the language model is able to decipher any input and generate synthetic text of unmatched quality. And not only that; the language model can also respond to question-answer formats, reading comprehension, summarizations, and translations. Although it hasn’t reached its state-of-the-art benchmark, it provides an entryway of ideas of machine learning’s potential in text generation.
Testing out the model
The examples that have been laid out range in text categories. You can view them on the dropdown menu under ‘Custom prompt’. There are creative writing (e.g. Sci-fi; Elon Musk’s debut album; Unicorns that speak English; Lord of the Rings), day-to-day or note proses (e.g. Life advice; Cooking instructions; Packing for trip to Mars), and even HTML code, as pre-made human-written prompts. It’s safe to say that the model does work across diverse domains.
I played around with it and noticed that it took about five to sixteen seconds in duration for the model to finish generating the text. For most custom prompts, the model was able to generate lengthy text of a minimum of one paragraph. Apparently, it was also able to generate just one sentence of text, which took the fastest to finish.
Here are some of the custom prompts I tried out:
The example above took the fastest to finish.
This example didn’t seem to make much sense, mostly because the context of the text was set superficially. So, it lacked depth. Although, I can’t deny that it was quite an entertaining read.
This last example made me wonder if the language model would often copy a pre-made text — as found in any one of its trained data sets — in the case that a similarly-worded prompt was inputted by the user. The last paragraph gave confidence to my suspicion that it would indeed take a text that was, and probably still is, available on another web page.
King actively updates about this on Twitter, where he would respond to the many others that indulge in the modern neural network experiment. Upon its release, his followers agreed that the experiment was a great demonstration of the language model, and even shared their own prompts as a reply to King’s announcement Tweet. This resulted in a thread on King’s Twitter post about TalkToTransformer.com, which shaped a very well-informed engagement on what others think about this experiment.
One follower brought up a very interesting question, particularly why the text length was limited (see Picture 4). King was responsive and transparent with his experiment, which provided a clear outlook on the demonstration.
Implications for the public
With the prominence of text-generating A.I., we can only question the authenticity of its quality, leading us to question its credibility. When A.I. can generate full-length texts, how can we assure that its information is correct, objective, and authentic?
The development of this unsupervised language model has provided an outlook on what the future of content generation can potentially become. This innovation certainly creates opportunities and merits. For instance, TalkToTransformer.com can easily become a tool to overcome writer’s block. Additionally, the GPT-2 language model can create A.I. writing assistants. However, we cannot neglect to think about its potential detriments to society.
OpenAI has discussed the malicious ways that this language model can be used:
- Generate misleading news articles
- Impersonate others online
- Automate the production of abusive or faked content to post on social media
- Automate the production of spam/phishing content
And so, it’s becoming increasingly needed for government and industry officials to declare safety-use guidelines. Future of Life Institute has attempted to create an overarching guideline for A.I. through the publication of its 23 Asilomar AI Principles. However, there is a specific need to create guiding principles for the general language model alone considering the rapid advancements it’s making. There needs to be close monitoring of how these language models are used by the community and public, and thankfully OpenAI will continue to do so with its GPT-2.
This article has merely discussed just one form of content generation. A.I. can now finish your sentences by predicting the next word you type. The practical applications for this unsupervised language model are limitless, but questions arise concerning its integrity of use. When it comes to advancements like these, being mindful and forward-looking are two considerations that we need to always factor in.
Author: Neysa Tavianto
Technology photo created by sebdeck — www.freepik.com
AI Lab One is an artificial intelligence consulting agency. For information on what we do and what makes us tick, visit https://ailab.one
