BUILDING TOOLS TO ASSIST ARTIFICIAL INTELLIGENCE
The Internet has revolutionized the communication process for humans by enabling advancement of technological innovations. One such innovation includes the expansion into Artificial Intelligence or AI. The developments in this internet based technology involves the creation of human-like intelligence so as to enhance interaction with the digital world and amongst us.
OpenAI is a venture founded to conduct research for the development of Artificial General Intelligence or AGI that possesses the complexity to learn, perceive and reason in the ‘people’ language.
The latest product of OpenAI is the GPT-3 and has created quite a commotion amongst the programming and digital enthusiasts. As the name suggests the GPT-3 is the third successor in its line of design.
The first in line was the Generative Pretrained Transformer (GPT). A model that has been trained on a large amount of data using language modeling in an unsupervised manner, which is then used to solve specific tasks. Once trained on enough data, the model is fine-tuned to achieve satisfactory results on different supervised tasks. This model works on Semi-supervised sequence learning and has been used to success in tasks like commonsense reasoning and reading comprehension.
GPT-2 model, a successor to GPT is trained on a large data set comprising 8 million web pages and has about 1.5 billion parameters. It is a direct scale-up of GPT and is trained on more than 10 times the data having 10 times the parameters. The GPT-2 is tuned to predict the next word in 40GB of internet text and has the ability to outperform other language models such as Wikipedia or news that have been trained only on specific domains.
GPT-3 is the third in the series of the autocomplete tools generated by OpenAI and operates on a very large scale of dataset that comprises basically any data that has been uploaded on the internet, allowing the program to tackle a wide array of autocomplete tasks. With 175 billion parameters and a mammoth sized dataset used to train the program, the GPT-3 looks for and finds patterns without human guidance.
Although the GPT series of tools is not exactly AGI, they are a beginner’s step for OpenAI into achieving it.
The new found capabilities of the GPT programs include:
- Increasing compute and availability of raw data using unsupervised learning techniques.
- Priming is done with an arbitrary input. This displays the ability to generate synthetic text samples and adapts to the style and content of the conditioning texts.
The future implications of these capabilities in AGI include:
- Generating realistic continuations about a chosen topic.
- Improve ability to perform and reason through processing broader context.
- Using the language functionality to perform sentiment analysis.
- Improve downstream tasks with more compute and data in addition to generative pretraining.