AIGP — Domain I: Understanding the Foundations of Artificial Intelligence — Final Part — BoK Version 1
In this final part of Domain-I, we will walk-through AI technology stack and history of AI.
Disclaimer-This blog features my study notes, shared with the intention of helping others who are exploring similar subjects.The information presented is drawn from a range of resources and personal experience. While I strive to credit sources and ensure accuracy, not all references may be explicitly cited. If you notice any errors or missing attributions, please inform me. This content is meant as a helpful resource but isn’t a substitute for professional advice.
AI platform and application
Commonly used AI models
AI Technology Stack — Infrastructure
As AI models become more powerful, the infrastructure needed to support them becomes increasingly vital. This refers to the different components — such as network, software, compute, storage and so on — that are essential to support and run AI systems to run effectively. We will explore the details of four key components as depicted below.
Compute
Refers to computational power to run algorithms, train model and perform data analysis. AI workloads require high-performance computing resources such as CPUs, GPUs , TPUs to handle intensive tasks. The infrastructure must provide these resources on demand while optimizing both cost and energy efficiency.
CPU (Central Processing Unit): Sequentially processing — general-purpose tasks. Used for less intensive AI workloads like data cleaning.
GPU (Graphics Processing Unit): Parallel processing — handling multiple tasks simultaneously. Performing complex computations in AI applications like training large neural networks.
TPU (Tensor Processing Unit): Custom-built AI accelerators designed to enhance the training and inference of large AI models. TPU is an application-specific integrated circuit (ASIC) designed by Google specifically for neural networks.
Types of Compute
-Serverless : Executes functions or code in the cloud without the need to manage server; hence the name “serverless” — automatic scaling and charged based on usage. It is a subset of cloud computing. e.g.— AWS Lambda
-High-performance compute (HPC) : Utilizes clusters of powerful processors to perform complex calculations quickly. e.g.— Supercomputers. As of 2024, the world’s fastest supercomputer is “Frontier” located at the Oak Ridge National Laboratory in the United States
-Trusted execution environments : Creates a secure area within a computer to run sensitive code and protects data from unauthorized access and tampering. Specifically used for tasks such as secure data processing, cryptographic operations. e.g. — Intel SGX
Storage
Storage is essential for AI because it holds the large amounts of data and models needed for training and running AI systems. AI systems require access to vast amounts of data, necessitating robust storage and management solutions. These solutions need to manage large volumes of data, guarantee data quality, offer reliable access.
General stages of AI — Typical stages of AI development depicted below
Key Storage Considerations for AI:
Capacity: Sufficient storage for large data volumes
Performance: Choose storage for efficient data handling
Scalability: Choose storage that scale seamlessly with growing data
Data Integrity: Robust backup to protect data from loss
Cost/Expense: Cost-effective storage solutions
Note that file type considerations are relevant to several aspects of AI storage mentioned above. Different file types include images, files, text, videos and so on.
Further there can be structured, unstructured, semi-structured data — Structured data are organized into fixed format (rows & columns) like database tables, making it easy to search and process. Unstructured data lacks a predefined format that makes it difficult to search like images, audio, articles shared on social media platforms. Semi-Structured data are data formats that combine structured and unstructured elements like JSON, XML, Logs.
Network
Network plays a crucial role in enabling data movement in AI systems. High-bandwidth, low-latency networks facilitate rapid data transfer between storage locations and processing units.
Complex AI models need high speed network like 10 Gigabit Ethernet (10GbE), InfiniBand, fiber channels and other low-latency networks.
AI systems are usually in the same data center for faster communication and improve performance.
Edge Computing processes data closer to the source (e.g. sensors, devices) to minimize latency, crucial for real-time AI applications — e.g. Traffic management systems use sensors and cameras to monitor traffic flow.
Internet of Things (IoT) refers to the network of interconnected devices and systems that communicate and share data with each other over the internet. Common in smart homes where devices collect and transmit data for various applications.
Also note TCP (Transmission Control Protocol) in AI ensures reliable transmission between devices and servers.
Software
Provides the tools, frameworks, and algorithms needed to develop, train, and deploy AI models.
Key terms
Tuning AI Systems — Process of adjusting parameters and algorithms to enhance model performance and accuracy. Fine-tuning is adjusting a pre-trained model to better perform a new or specific task — basically for customization. For e.g. fine-tuning a pre-trained model to recognize specific cat breeds say Siamese, Persian and so on instead of just identifying cats.
At a high level, fine-tuning includes:
- Preparing and uploading the training data.
- Training a new, adjusted model.
- Evaluating the results and revisiting the data preparation if necessary.
- Deploying the fine-tuned model.
Hyperparameters:
Settings or configurations that control the training process. Here are a few common hyperparameters listed below:
Learning rate : Controls how quickly or slowly a model updates its weights during training.
Number of Epochs: How many times the model sees the entire dataset during training.
Batch size : How many data examples the model looks at before making an update to its weights.
Challenges with tuning AI systems include:
Overfitting: Model performs well on training data but performs poorly on new unseen data.
Hyperparameter Selection: Identifying the best settings can be both time-consuming and complex.
Computational Cost: Requires significant computational resources and time.
Data Quality: Poor or biased data can result in ineffective tuning and reduced model performance.
Scalability: Modifying models for large datasets can be difficult.
Data Pre-processing & Data Post-processing
Data Pre-processing : Prepare and clean data for analysis that involves removing duplicates, handling missing values before it is used for analysis/modeling.
Data Post-processing : Refine and interpret results after analysis or modeling.
Data transformation
Converting data from its original format into a format that is more suitable for analysis or modeling improving data quality. This includes the following but the list is not exhaustive:
Data cleaning — Correcting errors in data. For e.g. — If some entries for a customer’s age are missing, filling them with the average age.
Normalization — Scaling to a standard range, to improve efficiency. For e.g. — Converting test scores ranging from 0 to 100 into a 0 to 1 scale.
Encoding: Converting categorical data into numerical values. For e.g. — Converting “color” feature with categories “red,” “green,” and “blue” into numerical values 0, 1, and 2, respectively.
Data Labeling
Assigning meaningful tags/categories to raw data so that machine learning models can learn from it.
Further it comes with several challenges: low data quality due to inconsistencies, inherent bias, scaling issues and so on.
AI Observability
Refers to the ability to monitor, understand, and manage the performance and behavior of AI systems. Helps in identifying problems early, and improving overall model performance.
Challenges seen :
Data Drift: Changes in input data over time impacting model performance, making it hard to detect and diagnose issues.
Data Integrity: Ensuring that the data used and generated by AI systems remains accurate, consistent, and reliable throughout its life-cycle.
Bias & discrimination: Occurs when decisions favors certain groups or individuals based on characteristics like race, gender, or age.
Open Source AI
Refers to artificial intelligence software whose source code is freely available for anyone to view, modify, and distribute. Promotes collaboration and transparency. Fosters innovation and makes AI technologies accessible to everyone.
History of AI
The “dawn of AI” refers to the early stages and foundational developments of artificial intelligence. AI born as a distinct field in the summer of 1956 during a seminal conference at Dartmouth College [New Hampshire].
Proposal of Dartmouth Conference
Conference brought together leading researchers, including John McCarthy, Marvin Minsky, Herbert Simon, Allen Newell. These big four “founders” of AI had created AI labs: Minsky at MIT, McCarthy at Stanford, and Herbert Simon and Allen Newell at Carnegie Mellon.
They penned a proposal that “every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”
Outcome of this conference marked the formal beginning of artificial intelligence as a distinct academic discipline.
Logic Theorist
Allen Newell and Herbert Simon — introduced the Logic Theorist (considered by many to be the first AI program).
Notice the repeating pattern of hype and setbacks in the field of AI. Summer brought in AI breakthroughs along with increased funding from government and investors for research and commercial use. Winter, however, brought a slowdown and setbacks, leading to diminished funding due to slow AI growth.
First AI Summer : mid 1950s to mid 1970s
Marked the official birth of AI — Dartmouth Conference.
AI research labs were established at top universities.
John McCarthy developed the first AI programming language, LISP [LISt Processing].
Joseph Weizenbaum developed ELIZA — earliest examples of natural language processing.
First AI Winter : mid 1970s to mid 1980s
Slowdown in AI research and development.
Decline in funding and interest due to unmet expectations — limitations of computing power and the complexity of AI problems.
Second AI Summer : mid 1980s to late 1980s
Renewed enthusiasm and progress in artificial intelligence research.
The Japanese Fifth Generation Computer Systems (FGCS) project aimed to develop advanced computing technology in the 1980s.
Second AI Winter: late 1980s to late 1990s
High cost of maintaining expert systems and the end of the Cold War led to a decline in interest and funding.
Cold War : Prolonged geopolitical conflict between the United States & its allies and the Soviet Union & its allies.
Renaissance and Era of Big Data: late 1990s — 2011
Renewed AI interest and significant advancements due to the growth of data and computing power.
In May 1997, IBM’s Deep Blue made history by defeating the reigning world chess champion, Garry Kasparov, in a six-game match — showcasing the capabilities of AI in strategic games.
Emergence of the internet led to a data explosion — marking the beginning of the “big data” era and bringing big data into the spotlight.
Google introduced key technologies like MapReduce and Bigtable
Advancements in computational power and machine learning — significant improvements in AI capabilities, such as recommendation algorithms in shopping and voice assistants in smartphones.
AI Boom : 2011 to present
Breakthrough in Deep Learning particularly with neural networks like GPT and BERT — AI’s ability to understand and generate human-like text, recognize images, and perform complex tasks.
Surge in investment from both private and public sectors, fueling research and development in AI.
Advancements in AI — Autonomous vehicles, personal digital assistants, advanced robotics.
Google DeepMind’s AlphaGo defeated world champion Go player Lee Sedol.
OpenAI’s release of GPT-3 revolutionized text generation with its advanced, language models’s capabilities.
Rise of generative models like DALL-E and ChatGPT demonstrated AI’s ability to create images, text, and other content.
Foundations : 1960s-1980s
Peter Naur coined the term “Data Science”.
Data science is about analyzing data to discover useful information.
Most data handling was manual
Concepts of data mining and analytics were still in their early stages of development.
Age of databases : 1980s-1990s
Widespread adoption of relational database management systems (RDBMS) — Oracle, IBM’s DB2, and Microsoft SQL Server
Structured Query Language (SQL) for querying data
Development of graphical user interface (GUI) tools made database management more accessible
Advent of Internet: 1990s — 2000s
Rapid growth of the World Wide Web, making the internet accessible to the general public
Emergence of “big data” due to massive increase in data generated from digital sources
Data mining” emerged as a key concept — process of uncovering patterns and insights within large datasets.
Rise of data science (2000s-2010s)
William S. Cleveland’s 2001 paper “Data Science: An Action Plan for Expanding the Technical Areas of the Field of Statistics” helped establish data science as a distinct field integrating statistics, computer science, and domain expertise.
Launch of Apache Hadoop (2006) and Apache Spark (2010) provided scalable frameworks for processing and analyzing vast amounts of data efficiently.
Current Trends (2010s — present)
Advancements in Deep Learning
Rise of Automated Machine Learning (AutoML) simplifying the process of building and deploying machine learning models
Growth of Data Science in Business Intelligence — Tableau and Power BI
Data scientists are crucial in various industries
Explosion of IoT and social media data led to the growth of real-time analytics and advanced data processing.
Modern drivers of AI and data science
Cloud Computing — Scalable and on-demand resources for data storage, processing, and analysis.
Mobile technology & social media — Generate vast amounts of data, driving new insights and applications in real-time.
IoT devices — Generate continuous data streams from network of interconnected devices — that communicate and exchange data with each other over the internet.
Privacy enhancing technologies (PETs) — Tools and methods designed to protect personal data and ensure privacy while allowing data to be used and shared securely — such as federated learning and differential privacy.
Blockchain — Digital ledger that securely records transactions across a network of computers, ensuring transparency and preventing tampering — immutable record of transactions.
Computer vision — Enables computers to interpret and understand images and videos — enables more efficient and interactive human-machine interactions.
AR (Augmented Reality) overlays digital information onto the real world — for instance snapchat filters when you take a selfie and add some filters like dog ears or sunglasses.
VR (Virtual Reality) creates an entirely immersive digital environment — for instance simulating the feeling of being on a roller coaster by immersing you in a virtual environment while you sit in a stationary seat.
Metaverse interconnected virtual universe where people can interact, work, and play through digital avatars and immersive environments — for instance Facebook Horizon — a place where friends can create avatars, explore virtual spaces, and hang out together, just like meeting at a park or cafe.
References & some additional reading
https://iapp.org/train/privacy-training/OCT-AIGP/
https://www.serverless.com/aws-lambda
https://www.ibm.com/topics/hpc
https://dualitytech.com/blog/what-is-a-trusted-execution-environment-tee/
https://www.codica.com/blog/ai-tech-stack-complete-guide/
https://platform.openai.com/docs/guides/fine-tuning
Artificial Intelligence: A Guide for Thinking Humans — Melanie Mitchell
https://www.forbes.com/sites/bernardmarr/2022/03/21/a-short-history-of-the-metaverse/
Explore my other articles on AIGP on Medium, and keep an eye out for new ones I’ll be publishing
Preparing for IAPP’s AIGP [Artificial Intelligence Governance Professional] Certification
AIGP — Domain I: Understanding the Foundations of Artificial Intelligence — Part A
AIGP — Domain I: Understanding the Foundations of Artificial Intelligence — Part B
