Transforming Learning with LLM

QANDA’s Approach to Utilizing LLMs in Education

Recently, the field of artificial intelligence has seen a surge in attention towards “Large Language Models (LLMs).” Although artificial intelligence has already been a highly regarded technology, the recent popularity of LLMs seems to overshadow the attention AI had received until now. Considering that LLMs are a subset of artificial intelligence, it’s not an overstatement to say that AI is once again gathering expectations due to the rapid emergence of LLMs. In this blog, we will provide an overview of LLMs and share how we view LLMs.

LLMs are gigantic language models that learn from millions of sentences and words, demonstrating a human-like ability to understand and generate language. Services built on powerful LLMs, such as OpenAI’s ChatGPT and Google’s PaLM, are based on training with massive text data collected from the internet. Through the learning process of such large datasets, models acquire a wide range of linguistic knowledge, including grammar, vocabulary, and common sense, resulting in a dramatic improvement in natural language understanding and generation.

Over the past decade, the AI field has seen numerous studies, leading to significant performance improvements in almost all areas, not just in natural language processing. Yet, LLM technology has been like a powerful wave crashing onto the market. What makes LLM stand out?

1. Innovation in natural language processing: Among numerous AI research efforts, LLMs have made noticeable progress in the field of natural language processing. The use of large pre-trained models has shown high performance in a variety of Natural Language Understanding tasks (text generation, text completion, machine translation, abstraction, question answering, etc.). This innovative achievement has recognized LLMs as a core technology in various NLP fields.
2. Structural scalability: LLM models based on the Transformers architecture can scale in size. Previous studies focused on finding task-specific architectures, and efforts were even made in applying AI for research (Neural Architecture Search). In contrast, recent LLMs trained on large datasets with a vast number of learnable parameters have become scalable. This scalability allows them to capture complex language patterns and generate human-like text, making them highly versatile for various tasks.
3. Generalization capability: Recent LLM models have shown remarkable generalization capabilities. They perform exceptionally well even on tasks that were not directly trained for. This capability allows for a broad application of AI without the need for complex development. The emergence of prompt engineering, a field specializing in this, reflects LLMs’ ability to understand user requirements very well, sparking discussions on whether “LLM represents general intelligence.” Moreover, this adaptability is leading to attempts to integrate LLMs into various industries, including healthcare, finance, entertainment, and education.
4. Accessibility: OpenAI, the developer of GPT, has made these models highly accessible to developers and researchers. By releasing pre-trained versions of models like GPT-3 and providing APIs that allow for further training, developers can easily integrate these models into their applications. The public API for ChatGPT has made it even easier for various businesses to utilize this technology. Considering the complexity of serving such large models, the improvement in accessibility has played a crucial role in the popularity of LLMs. Unlike OpenAI and Google, which do not disclose actual parameters and source codes, Meta plays a significant role in the open-source community. Meta has released the parameters and the operational code of its high-performance LLM model, Llama-2, as open-source, leading to various research based on Llama-2.

The features listed above are causing LLM-based changes to sprout around us. Encountering these attempts makes us anticipate the innovative changes and the impact that LLMs and derivative AI technologies will bring to industries and society at large. However, like any technology, LLMs also face various challenges. Among them, the phenomenon of hallucination has been highlighted. LLM’s “hallucination phenomenon” refers to the model generating or understanding information or context that doesn’t actually exist. This occurs during the process of generating or reasoning based on random patterns or information learned from training data, rather than data provided by users. Hallucination in LLMs can appear in situations like:

1. Errors in causality: LLM considers previous sentences, words, or dialogue history provided by the user to grasp the context. However, the text generation process includes probability-based sampling, which can lead to the model establishing inaccurate causal relationships and generating unrelated information.
2. Exaggeration of information: If the model learns a pattern that appears frequently in training data, it may exaggerate or overly emphasize information, leading to hallucinations. In severe cases, this can result in the provision of incorrect information to users.

The reason why the hallucination phenomenon in LLMs poses a significant problem is due to the highly diverse patterns in which users utilize LLMs. Among these usage patterns, the most critical is ‘replacing search.’ Users have started to ‘search for knowledge’ from LLMs, going beyond merely conversing with them or directing them to perform specific tasks. According to multiple articles, in the United States, the active user count of Chegg, used by students to search for learning content, and Stack Overflow, used by developers for troubleshooting, has significantly decreased following the introduction of ChatGPT and Copilot. This indicates that many users are asking LLMs for information that has definitive answers. The hallucination phenomenon occurring in such situations is critical. LLMs provided plausible answers to users’ questions, and although users were satisfied, in reality, it meant that users acquired incorrect knowledge, which is arguably worse than if the answer had not been provided at all.

To mitigate these disadvantages, Microsoft introduced Bing Chat, which combines a search engine with LLM. Similarly, Google also unveiled a service that combines search with generative AI through Search Labs. These services go beyond merely retrieving information contained within LLMs; they utilize conventional search engines to summarize searched results and deliver them “with sources” to users. This allows users to view more detailed information or verify the source of the information through additional actions if they have doubts. By equipping LLMs with the powerful tool of a “search engine,” the benefits gained are clear.

How can QANDA utilize LLMs?

QANDA also has a vast database of problems/solutions and successfully operates a search service using this data worldwide. Naturally, this has led to a significant interest in researching and developing ways to utilize LLMs.

In fact, even before the advent of LLMs, various AI models were already being used in QANDA services. Vision-based AI models were used to automatically detect problem areas, filter out non-problem images, or read math problems containing equations through OCR. These models automatically solved equations read by OCR, understood the context of problems, classified them, analyzed users’ learning patterns, and recommended content accordingly. AI was also crucial in detecting duplicate items in the database to refine it. As shown in the examples, AI was mainly used in a pipeline of discriminative models. However, LLMs are very powerful generative models, meaning that QANDA’s AI can now do a much wider variety of tasks. Instead of automating pipelines through classification and recognition as before, LLMs can now process the database and generate new information to deliver to users.

In this situation, we view the use of LLMs in three main ways:

1. Data processing: QANDA has various types of data. Most notably, there are problems and solutions, as well as students’ questions and teachers’ answers. Data on grade-level curriculums and mathematical concepts also exist. These data are composed in their own formats, hence lacking uniformity. LLMs are exceptionally effective at understanding these data, organizing them, and converting them into a unified format. The first data applied with LLMs were problem explanations. LLMs were used to transform explanations into the most efficient form for learners, and this service was launched under the name QANDA bot solutions.
2. Search-based solutions: LLMs have shown weaknesses in solving math problems, failing to correctly identify concepts needed for problem-solving or even making mistakes in simple arithmetic operations. This leads to hallucination issues. Observing “search engine combined with LLM services” released by Microsoft and Google naturally led to the idea of integrating QANDA’s search service with LLMs to address these hallucination issues. This rapidly progressed to research and development, leading to QANDA’s AI tutor service.
3. Subject-specific LLMs: Although combining search with LLMs can mitigate hallucination phenomena, LLMs inherently have limitations in problem-solving due to their characteristics. To overcome this, QANDA decided that it needs to train a subject-specific LLM tailored to the math domain. Currently, research is underway to train an expert LLM in the math domain using the accumulated data and AI models and to apply this to the service.

Learn More About Us

We, Team QANDA, believe that quality education is a right for every student, not a privilege. We are dedicated to delivering personalized educational resources to students worldwide, making learning accessible, engaging, and impactful for all.

Discover more about our journey on

• Our website
• Forbes interview article
• Company brief

📖Check out Cramify, the ultimate study tool to ace your exams!

#Cramify #QANDA #GenAI