Google Gemini’s Outdated Information Problem: A Setback for AI Assistants

How Google’s AI is falling short in providing up-to-date information, and why it matters

We are currently in an age of AI, at least when it comes to our phones. With the recent revelation of Apple Intelligence, there is now no smartphone that does not come with at least some form of generative AI. However, a troubling issue has emerged with Google’s own Gemini.

Recent testing of different generative AIs for creating D&D backstories led to experimenting with their capabilities. One task involved creating a 2K Warhammer 40K list for an Ork army in 10th edition. Theoretically, Gemini should have searched the web for current point values and tactics, presenting the best option. However, the information and lists provided were completely outdated.

Why is Gemini giving out-of-date info?

There are three variations of Gemini, with mobile devices using Gemini Nano. Unlike its counterparts, Nano isn’t directly connected to the internet, resulting in a different training set. Generative AI models are trained by introducing data and refining the gap between actual and expected outputs. For Nano, this training, including updated information, occurs in quarterly “Training rounds.” If these rounds lack current data, Nano can only respond with available information.

Another factor is that Google Gemini reportedly has fewer parameters compared to similar models — about 1.8 billion, roughly half of Apple Intelligence. These parameters determine how the model processes data, with more parameters generally allowing for better data processing.

The concerning aspect is that this misinformation issue isn’t limited to Nano but also occurs in the web browser version, which is supposed to be connected online.

Why is Gemini Nano giving out old info an issue?

The primary purpose of an AI like Gemini is to answer questions and enhance web searches. While it may suffice for simple queries like weather forecasts, it falls short for more specific interests and hobbies.

Using the Warhammer example: Building a new army and estimating costs typically requires multiple sources — list building apps, price searches, and tactical advice websites. Ideally, AI should consolidate this information into an easy summary. If Gemini can’t provide accurate, up-to-date information, its utility is severely limited.

Relying on outdated information for budgeting and planning can lead to frustration and wasted efforts, potentially causing users to abandon Gemini. This is particularly problematic as Google has been actively promoting Gemini across various devices.

What can Google do about this?

For Gemini to be a viable choice among competitors, Google needs to address these information accuracy issues promptly. This is especially crucial as Apple works to integrate ChatGPT into its Apple Intelligence, with plans for further expansions.

While Google Gemini on phones is still relatively new and requires significant improvements, its current state feels premature. For now, many users might prefer traditional web searching and personal fact-checking. However, given the rapidly evolving nature of AI, we may see improvements in the near future.