Generative AI for Beginners: Part 8 — Challenges and Limitations in Generative AI
This blog is part of the series Generative AI for Beginners, where we are learning basics of Generative AI, one simple step at a time.
To make it easy to grasp, I have divided the entire series in small parts. Each blog requires maximum 15–20 minutes to learn. After finishing the series, you will get a clear idea on fundamentals of Generative AI and its various aspects.
Part 2 — Understanding Machine Learning
Part 3 — Deep Learning: The Fundamental Pillar of Generative AI Advancement
Part 4 — Introduction to Generative AI
Part 5 — What is Large Language Model (LLM)?
Part 6 — Prompt Engineering: The Art of Communicating with AI
Part 7 — Ethical Considerations in Generative AI
Part 8 — Challenges and Limitations in Generative AI [current blog]
This is the 8th and final blog in this series where we will learn about Challenges & Limitations in Generative AI.
Let’s quickly recap what we have learnt so far!
- With a simple analogy and example, we learnt what AI is.
- We learnt capabilities of AI and how it is changing our day-to-day life.
- We looked into different types of AI with examples.
- We also understood how AI is different from human intelligence.
- With a simple analogy and example we learnt what machine learning is.
- We got a clear idea on supervised, unsupervised learning and reinforcement learning.
- We learnt how ML is different from AI.
- We looked into real-life examples and applications of ML
- We learnt how deep learning is inspired from human brain.
- We understood how artificial neural network works.
- We learnt how deep learning is used to solve complicated problems.
- With a simple analogy and example, we learnt what Generative AI is.
- We understood how Generative AI is different from AI.
- We looked into real-life examples and applications of generative AI.
- We learnt what language model and large languga model is.
- We understood how/why AI systems use large language model.
- We looked into some popular large language models.
- We learnt what prompt and prompt engineering is.
- We understood how to write clear and effective prompts
Ethical Considerations in Generative AI
- We explored ethical challenges in generative AI and how to deal with them.
Let’s start the topic challenges and limitations in generative AI.
Challenges and Limitations in generative AI
So far, we have learnt how generative AI and associated technologies like LLM, NLP, etc. are changing the entire world. On a high-level Generative AI seems to be an absolutely game changer which can generate any text, image, video, audio, music etc. like we human do and perhaps replace many types of jobs that we human pursue today. However, in reality, there are still some major challenges and limitations are there which needs to be solved before generative AI becomes part of our daily life.
Let’s have a close look into some of these challenges and limitations.
Lack of real creativity — Not able to think out of the box
We human are amazing at creativity. Its not just about creating a new music or write a poetry or a new painting, its also about altogether start a new genre of an art. For example, Pablo Picasso amazed the world by his new style of painting by breaking down objects into geometric shapes and presenting multiple perspectives simultaneously. Similarly, Kool Herc, known as “fathers of hip hop”, innovated a new genre of music by mixing different techniques.
Generative AI is still has a long way to go before it reaches the true level of creativity as we human have. Although generative AI has capability to generate new ideas, music, poetries, stories, it still goes by the defined rules and genres. It cannot altogether start a new type of story telling techniques or start a new genre of music.
Let’s simplify this with an example. If we train a generative model to solve Rubik’s cube, it can generate different ways to solve the cube. However, it may not propose the idea to unassembled the Rubik’s cube into individual pieces and reassemble it again.
Intensive resource requirements for training generative AI models
One of the significant limitations of generative AI is intensive resource requirements for training the generative AI models. Generative AI models, especially large-scale ones, for example, OpenAI’s GPT, require extensive hardware and energy consumption. Let’s break this down:
Training Time
Generative AI models are trained on huge datasets, often consisting of millions or even billions of examples. The training process can take weeks or even months, depending on the complexity of the model and the available computational resources.
Hardware Requirements
Training and running generative AI models require substantial computational resources. It includes high-performance CPUs or GPUs, memories, and other hardware components. These resources are often expensive to acquire and maintain. It’s one of the reasons that only big organizations are able to build the generative AI models who can afford huge cost and other resources.
Energy Consumption
Training generative AI models consumes a significant amount of energy. This also leads to environmental concerns, particularly the carbon emissions.
Ethical Considerations and Challenges
This is probably the most critical challenge we are facing with generative AI. Due to lack of regulations, there are many ways generative AI can be misused. Copyright and data theft Issues, plagiarism, distribution of harmful content, deepfakes, identity theft are some of the major ethical concern we have. In our previous blog Ethical Considerations in Generative AI, we have covered this in detail.
Challenges with Training Data
The training data plays a crucial role in generative AI. Any generative model depends highly on quality and quantity of training data. There are several challenges associated with training data, such as:
Quality of Data
The quality of training data directly impacts the performance of Generative AI models. If the training data is noisy, incomplete, biased, or of poor quality, it can lead to the generation of inaccurate or undesirable outputs by the AI model.
Quantity of Data
Generative AI models require large amounts of diverse training data to learn effectively and generalize well to new scenarios. However, obtaining and curating large datasets can be challenging and resource-intensive, particularly for niche or specialized domains.
Data Bias
Training data may contain biases present in the real-world data it was collected from. If not properly addressed, these biases can be perpetuated and amplified by Generative AI models, leading to unfair or discriminatory outcomes in generated content.
Data Privacy and Security
Accessing and utilizing large datasets for training Generative AI models can raise concerns about data privacy and security. Ensuring compliance with data protection regulations and safeguarding sensitive information is crucial to prevent unauthorized access or misuse of data.
Technical Challenges
The architectures of generative AI models are often very complex, comprising millions or even billions of parameters. Just to give you some examples, OpenAI’s GPT-4 has more than 1 trillion parameters. Even GPT-3 has around 175 billion parameters. Google’s BERT has 110 million parameters. Handling this technical complexity requires high expertise in machine learning algorithms and programming skills.
Quality control and Human Intervention
The content generated by Generative AI may contain inaccurate results. In industries where the impact of these generated content is affecting direct human life, for example, healthcare, legal services, it’s important to have a proper quality control and human review in place.
For example, if a generative AI tool in healthcare generate a wrong report based on the medical data of patient, the recommended treatment should not be given to the patient. We still need qualified doctors to review the generated report and provide their confirmation.
Summary
With this final blog, we have completed the blog series — Generative AI for Beginners. Hope you have a crystal clear understanding of generative AI and all its related concepts.
If you have any question or feedback, please let me know in comment!
