Representation of an urban landscape with various symbols illustrating the digital / AI evolution (DALL-E, Aug 2024)

A brief history of AI (Artificial Intelligence)

AI milestones, guiding distinctions / concepts, and application examples

Table of Contents

1. Introduction
2. Historical development of AI approaches
3. The conceptualization of AI (artificial intelligence): from what- to how-questions
4. Guiding distinctions in the domain of AI
↳ a. Guiding distinction 1: Weak vs. strong AI
↳ b. Guiding distinction 2: Connectionist (also: sub-symbolic) vs. symbolic AI paradigms
↳ c. Guiding distinction 3: Machine learning vs. deep learning in the connectionist AI paradigm
↳ d. Guiding distinction 4: Large Language Models (LLMs) vs. Small Language Models (SLMs) in the context of deep learning
↳ e. Guiding differentiation 5: Generative AI vs. predictive AI based on deep learning mechanisms, but also going beyond this
↳ f. Guiding distinction 6: Mono- vs. multimodal generative and predictive AI
↳ g. Further AI variants
↳ h. Hybrid solutions
4. Future prospects for AI
5. Summary

1. Introduction

The hype surrounding (generative) AI may have died down a little, especially after the panic-like stock market quake in early August 2024, which caused the market capitalization of the so-called Magnificent Seven (Alphabet, Amazon, Apple, Meta, Microsoft, Nvidia and Tesla) to shrink by many billions of dollars.

Nevertheless, it can be assumed that the AI train will continue to move forward in the coming years.
However, all organizations must ask themselves whether and how the use of AI is worthwhile with regard to specific use cases.

In this new Medium publication by WAITS Software- und Prozessberatungsgesellsch. mbH, you will therefore find a comprehensive guide to help you implement AI projects in your organization.

Before we go into medias res, however, it is worth creating a map of AI in this starter post, which briefly outlines the historical milestones as well as the guiding distinctions / concepts and some exemplary applications.

2. Historical development of AI approaches

A timelinie of notable artificial intelligence systems, in: Max Roser (2022), “The brief history of artificial intelligence: the world has changed fast — what might be next?”, URL: https://ourworldindata.org/brief-history-of-ai' [retrieved Aug 2024].

The history of artificial intelligence (AI) began in the 1950s and has experienced several ups and downs since then. Here are some of the most important milestones:

• 1956: The term artificial intelligence was coined by John McCarthy at the Dartmouth Conference. However, McCarthy’s term was initially primarily an academic marketing tool to generate interest and funding for a new field of research.
• 1966: The development of the first chatbot ELIZA by Joseph Weizenbaum.
• 1970s: The first AI winter.
  The high expectations for AI could not be met, which led to a decline in funding and interest. The reasons for this included limited computing power, inefficient algorithms and the failure to meet the fundamental challenges of AI research.
• 1980s: The renewed upswing through expert systems that used specific knowledge and rules for certain tasks.
  These systems were subsequently able to deliver useful results in narrowly defined domains, such as medicine, and led to a renewed interest in AI research.
• 1987: The second AI winter
  Disappointed expectations and technological limitations again led to a decline in AI research. The high costs and unreliability of expert systems, combined with economic factors such as the collapse of the market for LISP machines, contributed to this stagnation phase.
• 1997: IBM’s Deep Blue chess computer defeated world chess champion Garry Kasparov.
• 2000s: The increasing availability of large amounts of data (big data) and improved computing power, primarily due to the parallel processing of graphics processing units (GPUs), led to a new upswing in AI research.
• 2011: IBM’s Watson won the quiz show Jeopardy! because it could understand human language and answer complex questions.
• 2012: Breakthrough in the field of deep learning with AlexNet, a Convolutional Neural Network (CNN) that won the ImageNet competition.
• 2016: AlphaGo from DeepMind defeated the legendary Go player Lee Sedol (an 18-time world champion who was considered the greatest Go player of that decade) 4:1 (see AlphaGo — The Movie | Full award-winning documentary).
• 2017: The AI version AlphaGo Zero no longer practiced with thousands of human games to learn how to play Go like previous AlphaGo versions. Instead, it learned how to play Go by playing millions of games against itself within three days — without any human help.
  This way, it annihilated the AlphaGo version which had previously won against Lee Sedol in 100 games 100:0!
• 2020: Introduction of GPT-3, a powerful Large Language Model (LLM) with 175 billion parameters.
• 2021: Publication of DALL-E, a model for generating images from text descriptions.
• 2022: Presentation of ChatGPT, a language model based on GPT-3, which spread rapidly and demonstrated to the general public that AIs based on deep learning mechanisms could now conduct conversations that were hardly any different from human dialogs in multiple natural languages (English, German, French, etc.).
• 2023: Release of ChatGPT-4.0, an enhanced GPT version with over 1 trillion (!) parameters. This has greater capabilities and, above all, greater accuracy than the GPT-3 version.
• 2024: Release of LLama 3.x, a series of scalable and efficient open source language models optimized for various applications, by Meta.
• Early August 2024:
  A correction in high-tech stocks in the NASDAQ 100, influenced by the crash of the Magnificent Seven, indicates a more realistic assessment of the challenges and costs of generative AI.
  This dip in share prices does not necessarily herald the general end of the AI hype. However, it does point to an increasingly nuanced view of predictive / generative AI technologies and their risks and economic impact. See, e.g.:
  - The State of Generative AI in Global Business: 2024 Benchmark Report.
  - The Big Picture: 2024 Generative AI Industry Outlook.

3. The conceptualization of AI (artificial intelligence): from what- to how-questions

As with many other (scientific) terms, the question What is AI? leads to a multitude of definitions / interpretations.
This should come as no surprise to anyone. After all, the diversity of AI approaches reflects the heterogeneity of the concept of human intelligence itself.

AI therefore encompasses a wide range of technologies that emulate various aspects of human intelligence (recognizing objects, processsing natural language, drawing conclusions, etc.). These range from simple, rule-based systems to complex machine learning models (keywords: machine learning/deep learning) that can learn from experience and adapt.

And how does the business community benefit from AI?
AI not only offers the business world the opportunity to improve or automate existing processes, but also the chance to innovate and transform existing business models.

Anyway, if an essentialist questioning technique, i.e. What is AI (really?), fails in this context, then it makes sense to switch to how-questions in the sense of the sociological systems theory of Niklas Luhmann & Co.

In other words, how, i.e. with which guiding distinctions, is artificial intelligence conceptualized in each case?
This is the topic of section 4.

Image illustrating the diverse guiding distinctions in artificial intelligence and AI’s impact on the future of human society (Bing Image Creator, Aug 2024)

4. Guiding distinctions in the domain of AI

a. Guiding distinction 1: Weak vs. strong AI

• Weak AI, also known as narrow AI, specializes in domain-specific tasks such as the voice assistants Siri and Alexa or playing chess.
  These systems do not have (self-)awareness and are limited to their programmed tasks. They may therefore be excellent at playing chess or Go, but they cannot automatically also make a cup of coffee or tea.
• Strong AI: This hypothetical form of AI could perform any intellectual task that a human being can do and would also have (self-) conscious-ness. So far, however, this type of AI remains a theoretical or philosophical concept.
  And it is also unclear whether the currently emerging machine intelligence is not characterized by the fact that it can perform certain behaviors (recognizing images, identifying sounds, processing languages, etc.) in the same way as humans, but without having to have (self-)consciousness.

Timeline of AI-generated images from 2014 to 2022, showing progressively more realistic human faces and scenes, in: Max Roser (2022), “The brief history of artificial intelligence: the world has changed fast — what might be next?”, URL: https://ourworldindata.org/brief-history-of-ai' [retrieved Aug 2024]

In this sense, a bifurcation in the evolution of intelligence could be assumed:
- Intelligence with (self-)consciousness in humans and, rudimentarily, in higher primates.
- Intelligence without (self-)consciousness in AI implementations and, rudimentarily, in simpler animals.

A chart that highlights the exponential growth in AI training computation and its correlation with advancements in AI capabilities, in: Max Roser (2022), “The brief history of artificial intelligence: the world has changed fast — what might be next?”, URL: https://ourworldindata.org/brief-history-of-ai' [retrieved Aug 2024]

Note for readers of German:

• Perhaps the consciousness problem of AIs will simply be solved at some point by the fact that the intransparency of the ever-increasing internal complexity of multimodal AIs (especially if human emotions and thought processes can be determined probabilistically in the future) suggests that consciousness should be attributed, even if it cannot be proven.
• In short, the question of consciousness would be answered via complexity-based attributions.
• Reading recommendation in this context:
  W. Vogd / J. Hart, (2023), Das Bewusstsein der Maschinen — die Mechanik des Bewusstseins.

b. Guiding distinction 2: Connectionist (also: sub-symbolic) vs. symbolic AI paradigms

• Connectionist (sub-symbolic) AI: This approach, also known as neural networks or deep learning, models the functioning of the human brain using artificial neurons and their connections.
  Examples of this are models such as the aforementioned GPT-4 and the advanced language processing model BERT (Bidirectional Encoder Representations from Transformers), so that previous and subsequent co-texts can now be taken into account in word processing.
• Symbolic AI: This currently less popular approach uses explicit rules and logic to represent knowledge and reasoning. Symbolic AI systems therefore often work with formal languages and symbols to develop solutions to problems.
  Examples of this type of AI include rule-based systems and early expert systems.

c. Guiding distinction 3: Machine learning vs. deep learning in the connectionist AI paradigm

• The machine learning (ML) approach uses algorithms that can learn from data and recognize patterns without being explicitly programmed. Examples of ML are decision trees and support vector machines.
• Deep learning, or DL for short, is a subcategory of machine learning that uses multiple neural networks to recognize complex patterns in big data.
  Examples of DL are Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN).

d. Guiding distinction 4: Large Language Models (LLMs) vs. Small Language Models (SLMs) in the context of deep learning

• Large Language Models (LLMs) are models such as the mentioned GPT-4 and BERT with billions of parameters that require considerable computing power. Although they are versatile, they are often also expensive and less accessible.
• Small Language Models (SLMs), on the other hand, are smaller language models such as DistilBERT and Microsoft’s Phi-3, which use fewer resources and are thus more cost-effective.
  Accordingly, they are also extremely attractive for smaller companies.

e. Guiding differentiation 5: Generative AI vs. predictive AI based on deep learning mechanisms, but also going beyond this

• Generative AI: This type of AI generates new content such as text and images, as is the case with OpenAI’s already repeatedly mentioned ChatGPT and DALL-E.
• Predictive AI: This AI variant predicts future events based on historical data. Examples of this type of AI are predictive text or security threat identification.

f. Guiding distinction 6: Mono- vs. multimodal generative and predictive AI

• Monomodal generative AI : Systems such as GPT-3 and RoBERTa process only one type of data, e.g. text, to generate new content.
• Monomodal predictive AI: A predictive text system only uses text data to predict future text input.

An illustration of a monomodal predictive AI system (DALL-E, Aug 2024)

• Multimodal generative AI: Models such as the aforementioned GPT-4 and DALL-E combine different data types such as text and images to generate new content.
• Multimodal predictive AI: Such AI systems use several types of data sources simultaneously, e.g. text, images and audio, to enable more precise predictions.
  For instance:
  - Given an instructional video with simultaneous text and video data, the next utterance in a video can be predicted by means a of Co-attentional Multimodal Video Transformer.
  - Other real-world examples relate to the prediction of disease progression using various media sources (voice recordings, images, text data, etc.), predictive maintenance in manufacturing processes, the prediction of consumer trends, demand forecasting, etc.
• In addition, it is also possible to mix the generative and predictive capabilities of mono- / multimodal AI. Siemens, for example, has enhanced its predictive maintenance solution Senseye with generative AI, which significantly improves its user-friendliness.

g. Further AI variants

In addition to the AI types already mentioned, there are also more specific approaches such as:

• Reinforcement learning (RL), which learns through reward and punishment, such as with AlphaGo.
• Transfer learning, which, as in the case of BERT, applies a pre-trained model to a new task.
• Federated Learning, which (like Google’s version) enables the training of models without central data storage.
• There is also Explainable AI (XAI) such as LIME, which is designed to explain what machine learning classifiers (or models) are doing.
• Edge AI should also be mentioned in this context. This AI variant is used on devices at the edge of the network, see, for example, TensorFlow Lite. This means that the AI calculations should be performed on the device (vehicle, camera, etc.) on which the data is generated.

h. Hybrid solutions

Hybrid solutions combine symbolic with connectionist systems to utilize the advantages of both approaches. That is, these systems can handle simple and repeatable tasks using rule-based algorithms, while machine learning is used for more complex and variable tasks.

Here are some examples of such hybridizations:

• Cognex VisionPro ViDi
  This solution combines rule-based vision with deep learning to tackle complex inspection tasks.
  More specifically, rule-based algorithms are used to solve simple and repeatable tasks, while deep learning is used to analyze complex patterns and variations.
• Watson Knowledge Studio (IBM)
  This product is specifically designed to recognize user-defined entities and relationships in unstructured text data. It enables the creation of rule-based models based on linguistic patterns and user-defined rules.
  Knowledge Studio can then also be combined with machine learning to create hybrid models that utilize both rule-based and machine learning aspects.

4. Future prospects for AI

The development of artificial intelligence is progressing at a breathtaking pace for us humans, which makes it difficult to predict the future.

• However, one trend that is already underway should be mentioned here: the development of agents in the field of generative AI that go beyond pure text generation.
  These agents are designed to perform tasks autonomously and manage complex interactions.
  For further details, see the recent report by McKinsey, which sheds light on this development and describes how such agents could be used in various industries to automate processes and tap into new value creation potential.
• We can also assume that Edge AI, for example, will become more widespread because it enables devices to process data locally via AI, reducing latency and improving real-time decision-making capabilities.
• Furthermore, it is very likely that the above-mentioned Small Language Models (SLMs) will prevail because, depending on the intended use, recourse to LLMs is not only costly, but also tends to be overkill.
• As far as the development of general artificial intelligence (AGI) is concerned, there is currently much speculation, but few reliable predictions:
  - Possible manifestations of AGI could include non-domain-specific systems that are able to learn autonomously and adapt to new tasks without explicit programming.
  These systems would not only process information, but also develop common sense, contextual understanding and emotional intelligence so that they can perform any intellectual task that a human can perform equally or better.
  However, the realization of AGI would pose significant ethical, social and technical challenges, including the need to take safety precautions and minimize potential negative impacts.
  - Forecasts regarding the occurrence of this type of AI vary between less than 10 years and breakthroughs in the next 20 to 50 years.
• Whether a technological singularity will then occur, in which AGI triggers an irreversible technical-evolutionary dynamic without equal, cannot be ruled out. But at the moment this is pure speculation or (still) belongs in the realm of science fiction.

5. Summary

In this intro to artificial intelligence, we first listed some key milestones from the first name-and-claim by John McCarthy in 1956 to the stock market crash of the Magnificent Seven in early August 2024.

Instead of the essentialist question What is AI (really)?, the focus was then shifted to how-questions. In other words, which guiding distinctions are used to conceptualize the phenomenon of AI?
In this context, we discussed the following key distinctions:

• Weak vs. strong AI: Weak AI is specialized, strong AI is hypothetical and comprehensive.
• Connectionist vs. symbolic AI: Neural networks vs. rule-based systems.
• Machine learning (ML) vs. deep learning (DL) as a specific ML variant: Algorithms learning from data vs. complex pattern recognition through layers of neural networks.
• Large Language Models (LLMs) vs. Small Language Models (SLMs): Large language models with high computing power vs. smaller, more cost-effective models.
• Generative vs. predictive AI: Creation of new content vs. predictions based on historical data.
• Monomodal vs. multimodal (generative and / or predictive) AI: Processing of one data type (mainly text) vs. several data types (text, image, audio, etc.) simultaneously.

Finally, we discussed the future prospects of AI. That is:

• The development of autonomous agents in the field of generative AI.
• The spread of Edge AI for local data processing.
• The increasing use of Small Language Models (SLMs) compared to LLMs due to their cost efficiency.
• And, last but not least, we also talked briefly about the emergence of general artificial intelligence (AGI) and its possible consequences.

With this basic knowledge of AI, we can now turn our attention to the guidelines for implementing AI projects in organizations. However, this is the topic of our subsequent AI-related posts in this new publication on Medium.

As always, thanks for your attention!

Author for WAITS Software und Prozessberatungsgesellschaft mbH, Cologne, Germany: Peter Bormann— August 2024.