Photo by Steve Johnson on Unsplash

Fundamentals of AI

In the ever-evolving landscape of technology, Artificial Intelligence (AI) stands as a revolutionary force, reshaping the way we perceive and interact with the digital realm. This article serves as a comprehensive guide, delving into the core principles that underpin generative AI’s capabilities while shedding light on the powerful tools that drive innovation.

Every day several unique generative AI tools are released and it has been hard to chase its speed. During this pursuit and readings, a great deal of notes have been piled up on my notes on the theory of AI and generative AI tools. I want to gather these notes in several serial articles to make the topics understandable and see what I am following. I hope it will be useful to you who read them.

Basic Concepts:

Algorithmic Bias: When the data used to teach a machine learning system reflects the values of humans involved in that data collection, selection, or use.

AI Chronology

Weak AI: Artificial intelligence that is specifically programmed and focused to execute a narrow task effectively.

Strong AI: The main vision is to develop artificial intelligence to the point where the machine’s intellectual capability is functionally equal to a human’s.

Turing Test: A machine’s ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human.

Generative AI: Machine learning models that generate new content. Generative AI also requires training. Computing power, dataset availability, competitive interests, and model design are the factors that drive generative AI development. What we can do by Gen AI is write code for a website, answer unique customer service questions, and draw a picture of a cat scuba diving.

Discriminative models answer closed-ended questions, learn from training data, and guess the correct answer or categorize. Predicting tomorrow’s weather, categorizing books, and determining if a picture is a puppy or a badge are some examples of this type of AI model.

Generative adversarial networks (GANs): Generators try to trick discriminators

• Compare notes and get better in multiple rounds
• A GAN consists of two models in competition: A generator model tries to create fake data that looks indistinguishable from real data A discriminator model tries to tell the difference between real and fake data

Artificial general intelligence (AGI): An AI that exhibits intelligence like a human would. It has the scope of knowledge and reasoning across domains, social skills, creative thinking, and other cognitive competencies (vision, language)

Computational power allowed large models. These powers are parallelization and specialized hardware, Graphics Processing Units (GPUs), Tensor Processing Units (TPUs), Cloud computing, and Hardware-software optimization.

Reinforcement Learning with Human Feedback (RLHF): Reinforcement learning trains models through trial and error. Human feedback comes from users scoring model responses

The 5 phases of an AI implementation

1. Education — understand what AI is and use cases it is great for

2. Identification — determine which use case in our business is ripe for AI

3. Proof of Concept — build out a small-scale AI solution to test and learn

4. Scale — implement a full AI solution

5. Align Culture and Skills — change processes and upskill the workforce to leverage the AI solution

Responsible AI throughout the implementation

Four Pillars of Business Value:

1. Efficiency 2. Productivity 3. Creativity 4. Skill

Limitations of AI: While AI can perform specific tasks with remarkable precision, it cannot fully replicate human intelligence and creativity. Therefore, understanding, empathy, ethics, embedded content
creativity, and transparency become its limitations. Not enough data, incomplete or biased data, inability to encode complexity cause the limitations.

LLM (Large Language Models) Large language models (LLM) are very large deep learning models that are pre-trained on vast amounts of data. Large means training data and computing power, language means human-like text, and models mean learning complex patterns using text data. Sentiment analysis, identifying themes, translating text or speech, generating code, and next-word prediction are some applications of LLM.

Identify use cases for AI

• Five principles and four questions of a good use case

• Questions to identify a use case: How are competitors using AI? What about partners in adjacent, or unrelated, industries? What are some areas of the business that are in clear need of support? Where would there be funding to support a small-scale project? Or a full-scale implementation?
• Start small with a POC (Proof of Concept) — POC tied to high ROI potential. — Most likely to be valuable for the business — ROI can be measured in multiple ways

Training Methodology of AI:

Text Pre-processing:

• Tokenization: Splits text into individual words or tokens and converts into a list
• Stop word removal: Stop words do not add meaning and are eliminated through stop word removal
• Lemmatization: Group slightly different words with similar meanings and reduce words to their base form. For instance; Talking — TalkTalked — TalkTalk — Talk

Text representation:

Generative pre-training

• It learns from general-purpose datasets and is not optimized for specific tasks.
• Trained using generative pre-training: Input data of text tokens and train to predict the tokens within the dataset
• Types: 1- Next word prediction (Supervised learning technique, model trained on input-output pairs, predicts next word and generates coherent text, captures the dependencies between words, which word relates more with pizza? 2- Masked language modeling (Hides a selective word, train model predicts the masked words)
• Transformers provide the relationship between words. The components are Pre-processing, PositionalEncoding, Encoders, and Decoders

Fine-tuning: It adapts a pre-trained model. Why do we need fine-tuning?

• Largeness challenges: Fine-tuning can help, Powerful computers, Efficient model training methods, Large amounts of training data
• Computing power:
• Efficient model training: Training time is huge. Efficient model training means faster training time
• Data availability: AI needs high-quality data. To learn the complexities and subtleties of language

Advanced fine-tuning: (Reinforcement Learning through Human Feedback)

• Pre-training learns underlying language patterns and doesn’t capture context-specific complexities
• Fine-tuning
• Reinforcement Learning through Human Feedback (RLHF)

Developing Generative AI Models:

• Model design and data collection Model Development Steps: 1. Research and design 2. Training data collection 3. Model training 4. Model evaluation
• Stable Diffusion’s R&D: Deciding on image generation by settling on a diffusion model.
• Training data preparation: It requires massive amounts of required diverse, context-rich data and preprocessing
• Model training: Key advanced techniques:1. Transfer learning and fine-tuning 2. RLHF 3. Custom embeddings

• Model evaluation: Discriminative model evaluation techniques measure performance on well-defined tasks. The disadvantage is not to capture the subjective nature of generated content. Generative model-specific metrics use nuanced criteria, like realism, diversity, and novelty, and many well-known metrics. It cannot capture many subjective elements and often does not generalize. Human performance comparison uses benchmarks against human abilities. It could cause some unfair comparisons. The gold standard captures subjective aspects but it is slow, costly, and difficult to standardize and subject to human biases and irregularity. Turing’s classic test is proposed by computer scientist Alan Turing. It is a human evaluator who judges AI-generated content. It passes if the evaluator cannot distinguish AI from humans but human behavior is not always the right standard

Building Blocks of LLM:

• Linguistic subtleties: Irony / Humor /Pun / Sarcasm / Intonation / Intent etc.
• How do LLMs understand? — Trained on vast amounts of data — Largeness of LLMs (parameters )— Parameters represent the patterns and rules (Small number of bricks -> limited structures; Larger number of bricks -> complex and detailed structures ) — The more parameters the more complex patterns — Generates sophisticated and accurate responses

Learning techniques

• Fine-tuning: training a pre-trained model for a specific task
• But, what if there is little to no labeled data?
• N-shot learning: zero-shot, few-shot, and multi-shot
• Transfer learning
• Learn from one task and transfer it to a related task: Transferring knowledge from piano to guitar ( Reading musical notes / Understanding rhythm / Grasping musical concepts)
• N-shot learning: — Zero-shot (no task-specific data) — Few-shot (little task-specific data) — Multi-shot (relatively more training data: For example, a model trained on Golden Retriever)

Using AI Models and Generated Content Responsibility:

Evaluating and mitigating social bias:

• Systematic unfairness in generative AI
• Where bias appears: It can happen during Training data because of Skewed or unrepresentative information in the training dataset. The model itself might be biased because of pursuing goals that result in biased outcomes. How the model is used also causes bias for instance applying AI in wrong or malicious ways
• Identifying bias in data and models: Representation analysis compares how the model refers to different groups. Fairness metrics evaluate models for equal treatment, opportunity, and accuracy across groups. Human audits ask real people to review a model’s outputs to identify bias
• Mitigating bias in data and models: Diversifying data collection, adjusting models to prioritize different data, adversarial training, and continuous improvement are some ways to mitigate bias in data and models.

Copyright and Ownership:

• Law vs. AI: The legal landscape is evolving to meet rapid AI advancement: Those are 1. Intellectual property 2. Privacy implications 3. Evolving norms and regulations
• Follow IP best practices like checking the copyright status of training data, seeking legal guidance about the use, staying informed of regulatory dynamic
• Privacy implications with every prompt: It should be required to read the terms of service, understand how data is stored and used, and consider what we share: user data may be included in future training and suppose local alternatives: many generative AIs can be run at home
• Evolving norms: Different response needs across industries and norms in one context might not apply in another
• Evolving regulations differ across jurisdictions and may depend on the location of users, servers, and developers.

Responsible generative AI applications:

• Types of malicious use: Deepfakes, misinformation campaigns, and AI-enhanced hacking are some types of misuse of AI.
• Detection and prevention: Key usage principles should be determined. Human-in-the-loop and continuous monitoring are some ways to detect and prevent damage.

Getting Ready For the Age of Generative AI

Preparing for a future of generative AI in education and jobs, media and entertainment (Creative explosion, Personalized media, Requires new forms of trust) and science and technological progress

A novel implementation: 1. Augmentation: AI suggests edits, human decides 2. Co-creation: AI and humans collaboratively write a novel 3. Replacement: AI generates and publishes social media posts

Building Proof of Concept

• Phases of POC:

• A Successful POC:

• Defining Clear Goals:

• Understanding the end-user
• Understanding the timelines
• Understanding costs and funding

Data: No data means no patterns and wrong data means wrong patterns to learn.

Infrastructure:

• Deployment Infrastructure depends on where the solution will be hosted, interacted with, monitored, and secured and includes MLOps and DevOps.
• Questions to consider: What is the end-user interface? Does it need to be built or will the AI solution be integrated into an existing system? Does the AI solution need to be real-time? Who should be able to access the AI solution?

Building the project team

In conclusion, our journey through the realms of Artificial Intelligence has been nothing short of enlightening, as we’ve navigated the fundamental principles. From understanding the bedrock concepts like machine learning and neural networks to exploring the diverse toolbox of AI applications, we’ve witnessed the sheer potential and transformative power that AI holds. As we stand at the intersection of human ingenuity and technological prowess, it’s clear that the future is coming with possibilities. The collaboration between human intellect and AI tools is not just a merger of algorithms and data; it’s a symphony of innovation that can revolutionize industries, solve complex problems, and enhance our daily lives. In the next article, I wish to write a series about AI tools based on the fundamentals above.

Call-to-action

If you gained from this, please leave as many claps as you can….