How Well Does ChatGPT Really Remember Our Conversations?

Conversations are dynamic, influenced by multiple factors including tone, background knowledge, and notably, context. It’s the context that often proves to be a hurdle in the realm of artificial intelligence, especially when conversing with AI models like ChatGPT.

What is Context?

Imagine two individuals immersed in a discussion. A third person enters, and without knowing the beginning or current topic, they feel lost. This missing backdrop, the essence of the conversation they jumped into, is context.

In the digital world, context can be thought of as the series of previous inputs and outputs that shape a current conversation. The GPT series, standing for “Generative Pre-trained Transformer,” uses this principle to create relevant responses. Through its underlying architecture, GPT processes input tokens, like words or characters, in relation to each other rather than individually, facilitating a contextual understanding.

Can GPT Hold a Fully Contextual Conversation?

The straightforward answer is both yes and no. While GPT can produce text that feels contextually appropriate, there are nuances to consider.

Here’s a simplified workflow of how ChatGPT appears to understand context:

1. User enters a prompt into ChatGPT.
2. This prompt is transmitted to the OpenAI API for the chosen model (GPT-3 or GPT-4).
3. The API returns a contextual response.
4. User continues the conversation or refines their prompt.
5. The process is repeated.

Regular users might observe that extended interactions can lead to responses that seem out of context or fuzzy. This isn’t because ChatGPT forgets, but rather due to the token limit it operates within.

Each model version can process only a certain number of tokens:

• GPT 3.5 turbo: 4096 tokens
• GPT 4: 8192 tokens
• GPT 4 (32k): 32768 tokens

As conversations grow and more tokens accumulate, if they exceed the model’s limit, the earliest tokens start getting removed, making the AI lose the ‘beginning’ of the conversation. Thus, to maintain a coherent conversation, users should ideally keep interactions within the token limits.

Just remember, GPT models don’t “remember” in the way humans do, but they use the previous parts of a conversation to generate relevant responses during a session. Once the session ends, they don’t retain that information.

Conclusion

When using tools like ChatGPT or the OpenAI API, understanding the concept of tokens and context can make a significant difference in the quality of the conversation. As we harness the power of such advanced models, it’s crucial to be aware of their strengths and limitations to get the best results.