Weekly Gen AI Nugget 1 — Introduction to AI, ML, DL, Generative AI and NLP and the relationship between them (with a focus on GenAI).
A brief overview of the main concepts and examples of artificial intelligence and its subfields.
What is AI?
AI is a branch of computer science that aims to create systems that can do tasks that normally need human intelligence. These tasks include learning, reasoning, problem-solving, perception, and language understanding. Some examples of AI in our everyday life are:
· Virtual assistants like Siri and Alexa, that can answer our questions, set reminders, and control smart devices.
· Chatbots that help us on websites, by providing customer service, booking appointments, or giving product recommendations.
· Self-driving cars that can navigate traffic, avoid obstacles, and follow road rules.
· Facial recognition systems that can identify people, unlock devices, and enhance security.
· Recommendation systems that can suggest products, movies, or music based on our preferences and behaviour.
What is ML?
ML is a way to understand machine learning as a method of teaching a computer to learn from examples and make decisions based on that learning, without having to program it explicitly. An example of this is a spam email filter. Rather than specifying every word or phrase that indicates spam, machine learning algorithms can learn from thousands of emails, identifying patterns that separate spam from non-spam emails. Then, when a new email arrives, the filter can use its learning to predict if it’s spam or not, based on those patterns.
What is DL?
DL is like teaching a computer to understand data by building complex layers of concepts, like how our brains work. For example, in image recognition, deep learning models can learn to recognize simple shapes like edges in early layers, and progressively more complex patterns like eyes and noses in deeper layers. Finally, in the last layer, the model can classify the image as a cat, a dog, or something else. Some examples of DL in our everyday life are:
· Image Recognition: It can recognize faces, objects, medical images, and road scenes.
· Natural Language Processing (NLP): It powers services like virtual assistants, language translation, sentiment analysis, and text summarization.
· Speech Recognition: It enables systems that can understand and transcribe voice commands, queries, and conversations.
· Computer Vision: It enables systems that can detect and track objects, faces, and gestures, and perform tasks like face swapping, video editing, and augmented reality.
· Artificial Neural Networks (ANNs): It enables systems that can learn from large amounts of data, and perform tasks like pattern recognition, classification, and prediction.
What is Generative AI?
Generative AI is a type of deep learning that refers to a type of artificial intelligence that is designed to create new content or data that resembles or is like existing examples. Unlike traditional AI systems that are primarily focused on classification or prediction tasks, generative AI can generate entirely new content, such as images, text, music, or even videos. Some examples of generative AI in our everyday life are:
· Image Generation: It can create realistic images of faces, animals, landscapes, and artworks, that do not exist in reality.
· Text Generation: It can write coherent and meaningful texts, such as stories, articles, captions, and summaries, based on a given prompt or topic.
· Music Generation: It can compose original and harmonious music, based on a given genre, mood, or style.
· Video Generation: It can create realistic videos of people, animals, or scenes, that do not exist in reality, or modify existing videos with effects like face swapping, lip syncing, or style transfer.
Two key components of generative AI are as follows:
- FM (Foundation Model): This is a term for a model or architecture that serves as the basis for other models or architectures. Foundation models are usually pre-trained on big datasets and then adapted for specific tasks or domains. They are useful for various generative AI applications, such as natural language processing (NLP) and image generation.
2. LLM (Large Language Model): This is a kind of model in natural language processing (NLP) that can understand and produce human language at a large scale. Large language models are often deep learning models with a huge number of parameters, trained on a lot of text data to learn the subtleties of language. For example, GPT (Generative Pre-trained Transformer) models by OpenAI, such as GPT-3, which has 175 billion parameters. These models can do tasks like text generation, translation, summarization, and question answering.
What is NLP?
NLP is an AI discipline that focuses on natural language, the primary mode of human communication. NLP applies ML and DL approaches to teach machines how to process and generate language. ML requires human experts to design features and models for language tasks, while DL leverages neural networks and transformers that can automatically learn from data and context.
- The goal of NLP is to make machines understand and produce natural language, which is how humans communicate. NLP uses ML and DL methods to train systems on large amounts of text data and perform various language tasks. ML involves using handcrafted features and models for language processing, while DL uses neural networks and transformers that can learn from data and context without human intervention.
Difference/Overlap between NLP and GenAI?
Generative AI, which builds upon LLMs and other algorithms like transformers, can create novel content based on the provided context. It merges techniques from NLP and other AI fields to extend beyond traditional capabilities. Popular architectures such as GPT (Generative Pre-trained Transformer) excel in various natural language processing tasks.
However, both LLMs and Generative AI face computational challenges, requiring large and diverse datasets and substantial computing resources. Additionally, ethical concerns arise regarding data privacy, plagiarism, and biases in training data.
In conclusion, while NLP primarily focuses on analyzing data and making predictions, Generative AI has the unique ability to generate new content similar to its training data. Generative AI is characterized by its capacity to create novel outputs based on context, differentiating it from conventional NLP models.
What are the concerns surrounding generative AI?
Generative AI has various challenges and risks associated with its outcomes and applications. Some of the main issues that arise from the current state of generative AI are:
· It has the potential to generate misleading and deceitful information, eroding trust when the origin and authenticity of data are unclear.
· This can lead to new instances of plagiarism, infringing upon the rights of original content creators and artists.
· Additionally, it may disrupt established business models reliant on search engine optimization and advertising.
· Furthermore, it could contribute to the proliferation of fake news and enable the dismissal of genuine photographic evidence as AI-generated fakes.
· Moreover, it poses a risk of impersonating individuals for more successful social engineering cyber attacks.
Research Papers to get an overall understanding
- Attention is all you need (2017): The original transformer work and research paper from Google Brain that started it all. (blog post)
- BERT: Pre-training of deep bidirectional transformers for language understanding (2018): One of the first publicly available LLMs, with many variants still in use today. (blog post)
- NLP reasoning survey (2023)
- Theory of Mind in LLMs (2023)
- Augmented LLMs with reasoning (2023)
- Eight Things to Know about Large Language Models (2023): Probably my favorite LLM survey position as it takes some strong stances and comprehensively cites the literature.
- A survey of large language models (2023): Comprehensive breakdown of current LLMs, including development timeline, size, training strategies, training data, hardware, and more.
Basic GenAI Courses :
My Next Story — Weekly GenAI Nugget 2 — Understanding GAN, one of the most widely used Gen AI models using a simple story.