Before we dive deeper… What’s generative AI already?

3rd #AI4BetterScience series blog

The first couple of #AI4BetterScience issues showed the relevance of exploring the potential impact of generative artificial intelligence (AI) in science. It is time to continue our journey towards more conceptual ideas and practical advice. However, before doing so, it is essential to ensure everyone is comfortable with AI, generative AI, and our future focus, large language models (LLMs). Below is a short and accessible introduction that integrates some nuances. You will also find a glossary at the very end. Have a look!

Artificial Intelligence: A growing family!

Let’s start from the beginning: what is artificial intelligence? It refers to the capacity of a computer or a machine to perform tasks that would typically require human intelligence. This includes learning from experiences, understanding and processing language, solving complex problems, and recognising patterns. AI operates through large datasets, sophisticated algorithms, and computing power. It works by analysing vast amounts of data and identifying trends and patterns that are not immediately obvious. The core appeal of AI lies in its versatility and efficiency, enhancing human capabilities and automating specific tasks.

Over the years, AI has become integral to various industries, transforming sectors like healthcare, finance, and automobiles through intelligent automation and predictive analytics. Its application ranges from developing self-driving cars and virtual assistants to more complex systems like disease prediction tools and automated financial advisors. For instance, in healthcare, AI can analyse data from thousands of medical records to identify potential outbreaks of diseases before they occur, thereby helping in preventive healthcare. Similarly, in customer service, AI-powered chatbots can handle thousands of inquiries simultaneously, providing quick responses tailored to each user and showcasing AI’s ability to perform repetitive tasks and adapt to new challenges dynamically.

It sounds like magic as if the machine has acquired spontaneous capacities and gained intelligence. However, generations of scientists and engineers have studied and developed AI in various fields, such as computer science, mathematics, and cognitive science. They conceptualise and experiment with different model architectures and training strategies. It is a fast-moving field, bringing regularly surprising discoveries, which make it exciting and promising. However, we are more often than not far away from the uncontrollable technology the media likes to portray. A thrilling example of this difference in perception is the term “intelligent” machines, which was chosen at the early conceptualisation time. Field professionals would prefer the term “learning” machines, even if it is a regular debate. Below, you can see the sub-main domains of artificial intelligence, which appear with the progression of the field, with generative AI arriving the most recently.

Generating new data

Now we have dealt with the landscape of artificial intelligence, let’s zoom in on generative AI. Its power lies in its use of deep learning models, specifically neural networks that mimic the structure of the human brain. These networks are trained using large amounts of data and adjust their internal parameters based on the feedback received through training. The sophistication of these models enables them to identify and replicate complex patterns. Therefore, after training on a vast corpus of existing material, generative AI models can generate outputs indistinguishable from human-generated content and produce original creations ranging from text and images to music and synthetic data. For example, writing poems, news articles, and even code in the text domain. In visual arts, it can generate images that mimic the style of famous painters or create entirely new visual expressions. It makes generative AI a powerful tool for innovation across various fields. For instance, product design can quickly generate multiple prototypes, speeding up the development process and allowing designers to explore more creative solutions.

At this point, it is fundamental to distinguish generative AI from discriminative AI. They represent two distinct approaches to machine learning, each with specific objectives and methods. Generative AI generates new data instances or content that mimics the training data. It learns the underlying distribution of a dataset to produce outputs that can be remarkably similar to the original examples. In contrast, discriminative AI focuses on distinguishing between different categories or labels within a dataset. It does not generate new data but instead learns the decision boundaries between classes. For instance, while the generative model could create new images of cats that look real, a discriminative model would identify whether a given image contains a cat.

Fundamentally, discriminative AIs classify or distinguish (e.g., predict that a specific email is spam), then engage a decision (e.g., place the potential spam email in the junk folder) or support the decision-maker (e.g., alter the user about the potential spam and let them make the decision). They are challenged with bias and fairness in decision-making, as it directly impacts outcomes based on the learned decision boundaries. On the other hand, generative AI is particularly prone to “hallucination,” where the model generates false or misleading information that appears plausible. This is a significant issue because these models, such as those generating text or images, can produce convincing but entirely fabricated outputs, which can mislead users or propagate misinformation. Ethically, generative AI introduces unique concerns, especially around the originality and ownership of generated content, raising questions about intellectual property rights and the potential for copyright infringement. Furthermore, the ability of generative AI to create realistic simulations of human beings or events can lead to privacy violations or misuse in creating deceptive media, commonly known as “deepfakes”.

Remember my name: Large Language Model

Now you know the basics about generative AI, let’s talk about Large Language Models (LLMs), which are generative AI specialising in understanding and generating human-like text. These models, such as OpenAI’s GPT (Generative Pre-trained Transformer), are trained on diverse internet text to learn the nuances of human language, from grammar to style and even context. The training involves feeding these models with terabytes of written content, allowing them to learn vast information and skills without direct human supervision. Once trained, LLMs can perform tasks such as translating languages, writing creative fiction, and even coding, demonstrating a broad understanding of human languages.

One of the critical features of LLMs is their ability to generate coherent and contextually relevant responses in a conversational format. This makes them particularly useful for applications such as virtual assistants, customer service bots, and interactive educational tools. The technology behind LLMs involves layers of neural networks — specifically transformer models — that excel in handling data sequences. These transformers analyse the text input and generate responses based on the probability of one word following another, enabling them to create responses that are not only relevant but also remarkably human-like. This capability revolutionises how we interact with machines, making digital interactions more natural and intuitive.

An active community is exploring how to improve these models, and significant progress has been made since the launch of ChatGPT 3.5. The current trend aims to create multi-agent LLMs where multi-specialised models could interact to solve complex problems. Another promising trend is the development of multimodality capacity, where a model could integrate different data types, such as images, text, video, sounds, etc. These are the most promising generative AI models for science, and we will focus on them further in future issues!

There is a lot more to say about history (e.g., AI winters), different specialities (e.g., Natural Language Processing), and architecture (e.g., Generative Adversarial Networks or GAN). However, this is the fundamental information we need to pursue our journey about #AI4BetterScience!

Thank you for reading. Please comment below or on LinkedIn/Twitter to share your experience, constructive feedback on this blog, and suggestions for future issues.

Next time, we will discuss a central concept regarding the cooperative relationship between a large language model and humans: Me, What, and How. See you next week!

Thank you for reading!

Keeping going with the following issue: A Human-AI Knowledge Interaction Model for Trustworthy Generative AI
Or read the previous issue: The problematic grey area in the current guideline for Generative AI in science

Glossary

Artificial Intelligence (AI) encompasses systems or machines that mimic human intelligence to perform tasks and can iteratively improve themselves based on the information they collect. This includes learning, reasoning, problem-solving, perception, and language understanding abilities.

Deep Learning is a subset of machine learning that employs multi-layered neural networks to analyse various factors of large amounts of data. Deep learning models are particularly good at recognising patterns and making intelligent decisions based on large datasets.

Neural Networks are a foundational technology in AI. They are algorithms designed to recognise patterns and perform tasks related to human perception, such as vision and speech. Neural networks consist of layers of nodes, or “neurons,” that can learn to perform tasks by considering examples, generally without being programmed with task-specific rules.

Generative AI is a branch of AI that focuses on creating new content — whether text, images or even music — that resembles human-generated content. It uses deep learning techniques to understand and replicate the nuances of the input data it has been trained on, allowing it to produce novel creations indistinguishable from genuine articles.

Discriminative AI, unlike generative AI, identifies differentiating features between categories or labels within a dataset. It does not create new data but focuses on analysing given data to classify it accurately or make predictions based on observed boundaries between different classes.

Large Language Models (LLMs) are advanced generative AI models designed to understand and generate text that closely mimics human language. Trained on vast amounts of text data, LLMs can perform tasks like translation and content creation and even engage in conversation by predicting text sequences in a contextually relevant manner.

Transformer Models are a type of neural network architecture that has revolutionised how machines handle data sequences, such as text or time series. Transformers use mechanisms called attention to weigh the influence of different words on each other and are highly effective at modelling complex patterns in data.

Smart Automation refers to using artificial intelligence to automate complex processes that traditionally require human cognitive functions. This can range from automating routine tasks like data entry to more complex activities like decision-making in unpredictable environments.

Predictive Analytics involves analysing historical data through statistical algorithms and machine learning techniques to predict future events. Used extensively in finance, healthcare, and retail industries, predictive analytics helps make more informed decisions by anticipating likely outcomes.

Virtual Assistants are AI-powered applications that understand spoken or typed commands and perform tasks for a user. These include various services, from simple task handling like setting reminders to more complex functions like providing personalised recommendations based on user preferences.

Ethical AI addresses the moral concerns in AI development and deployment, such as fairness, transparency, accountability, and privacy. It involves designing AI systems that adhere to broadly accepted ethical standards and encourage positive societal impact while minimising adverse effects.