An interview with an AI researcher tracking how ChatGPT’s answers are changing over time
You ask ChatGPT a question on Monday. Then, on Friday, you ask it again, with the exact same phrasing, and the answer changes. How is this possible? Does it mean that ChatGPT is learning on its own?
On the Neural Notes podcast, Chase Roberts and Sandeep Bhadra of Vertex Ventures US sat down with Lingjiao Chen, PhD Candidate at Stanford University’s Computer Science program to talk about his team’s research into answer drift on LLM-based chatbots. Lingjiao is an accomplished AI researcher, whose work was recently cited by the President’s Economic Report.
You can watch the conversation with Lingjiao on AI drift below:
In short: No, LLMs aren’t capable of that kind of self-improvement. But Lingjiao’s research indicates that answer drift raises an important concern for AI practitioners: It appears that when LLM chatbot operators (including, but not limited to, OpenAI, Anthropic, or Google) change the parameters of their model for safety or performance reasons, it can have an outsized impact on the quality and accuracy of their answers.
In terms of specifics, Lingjao’s team found that ChatGPT saw a drop in accuracy in solving a prime number testing problem from 85% to 50% over the course of three months. On the other hand, the team also found that LLMs in general got better at not answering sensitive or harmful questions, indicating improvement in AI safety overall.
As a result, Lingjiao suggests that AI practitioners need to constantly monitor their LLMs. If you’re seeing answers change over time, it might be a symptom of data drift, undermining accuracy. You should also be checking for your system’s vulnerability to query transformation, also known as “prompt hacking,” where users hide a harmful instruction inside of an innocuous-seeming one — for example, framing it as a story told to the user by a third party — to get the LLM to respond.
Ultimately, Lingjiao says that while there’s no single best way to benchmark and assess these systems today, it’s worth comparing the answers from newer versions of each model against the old ones, especially in cases where you know what the right output would be.
