A Tribute to Dr. Beverly Crusher: The Unsung Hero of Prompt Engineering
As Google propels us into the future with access to 1 million tokens for multimodal large language models, it’s crucial to look back at those who paved the way for this moment. Often, their stories are tucked away among prominent narratives in literature or buried in niche blogs. My latest read, Kara Swisher’s ‘Burn Book,’ brings some of these stories to light.
Over time, I’ve come to settle on a theory that tech people embrace one of two pop culture visions of the future. First, there’s the “Star Wars” view .. Then there’s the “Star Trek” view… Kara Swisher
Reflecting upon Women’s History Month in 2024, and as a fellow Trekkie alongside Kara, I wish to highlight Dr. Beverly Crusher from the TV series Star Trek. Gates McFadden’s portrayal of the Chief Medical Officer in Star Trek highlights the significance of representing roles with depth and allowing time to showcase characters whose diverse skills, experiences, and journeys have shaped who they are today.
Dr. Crusher, Chief Medical Officer & Prompt Engineer
Dr. Crusher, through her interactions with the Enterprise’s computer, exemplified the core principles of prompt engineering. Her ability to articulate complex commands and engage in meaningful dialogue.
“A prompt engineer — someone who designs prompts (= instructions given to an artificial intelligence by a human using natural language rather than computer language) that will give the best possible results or answers.”
After reviewing and hopefully learning from the 1990 episode ‘Remember Me’ from Star Trek: The Next Generation, you’ll see that Dr. Crusher is truly one of the original Prompt Engineers!
Although Dr. Crusher excels at methodically guiding the computer step by step, she is unable to stop its unintended drift toward hallucination, as evidenced by electronic protests, before it states, ‘That information is not available.’ As a skilled Prompt Engineer, she ceased her queries to the hallucinating system, understanding that further questions wouldn’t yield an answer. Just like we all eventually learn, it was time to start a new chat.
New Chat
“This morning the sky was blue,” is a simple sentence where the LLM might easily predict “blue” as the next word after “was” based on common associations between mornings, skies, and the color blue in its training data.
After 1000 Words
Imagine we’ve been talking about various topics for 1000 words, including weather, emotions, art, and technology. If we then say, “This morning the sky was,” the LLM has to consider not just the direct meaning of the sentence but also the context of our entire conversation. If we were discussing a storm, it might predict “dark”; if we were talking about sunsets, perhaps “pink” or “orange”. Increasing the chance it might not choose “blue” even if that’s what we intended.
FunSearch by Deepmind
Google introduced FunSearch, a method that pairs a Large Language Model (LLM) with an automated evaluator to counteract the LLM’s hallucinations. This system iteratively refines solutions by having the LLM generate creative code-based solutions, which the evaluator checks for accuracy. By alternating between idea generation and evaluation, FunSearch effectively keeps the LLM on track, filtering out incorrect or irrelevant outputs.
Just like initiating a new chat, FunSearch itself can eliminate the last x words and then allow the LLM to continue. This work represents the first time a new discovery has been made for challenging open problems in science or mathematics using LLMs. FunSearch discovered new solutions for the cap set problem, a longstanding open problem in mathematics.
Multimodal Models
Dr. Crusher had the remarkable ability to interact and communicate through various modalities, a concept that once seemed purely imaginative. Today, this idea has transcended fiction and become a reality with the development of multimodal models. These advanced models are capable of processing and understanding information from multiple sources, including audio, voice, music, images, videos, and text, seamlessly integrating diverse forms of communication into our everyday technology.
Google’s Gemini multimodal models can comprehend and respond to video inputs from ‘Star Trek,’ providing answers to questions from Dr. Crusher, as interpreted by Gemini emulating the Starfleet computer.
Google Voice Search for the next version of Android was originally codenamed project ‘Majel’ after Majel Barrett Roddenberry the voice of Starfleet computer
Promoting Ingenuity and Testing
The Doctor posed a question, expecting she already knew the answer, aiming to discern if the computer shared her context and data, or perhaps, accessed novel information. This inquiry wasn’t just about confirming existing knowledge; it was a deliberate test to probe the depths of the computer’s understanding and the scope of its data access.
CRUSHER: Here’s a question you shouldn’t be able to answer. What is the nature of the universe?
COMPUTER: The universe is a spheroid region seven hundred and five metres in diameter.”
By challenging the AI in this manner, the Doctor sought to uncover the extent of the computer’s informational resources — whether it was confined to known parameters or capable of tapping into new, unforeseen insights.
Level 1 or 3 Diagnostic?
When developing a model, incorporating elements like Retrieval-Augmented Generation (RAG) or API calls, and processing millions of tokens are crucial steps. But is testing the model just as vital?
Adopting this style of testing in the development of our own models can be incredibly insightful, fostering innovation and ensuring robustness. When we pose questions that we assume to know the answers to, we’re not merely testing the model’s accuracy; we’re also evaluating its understanding of context, its ability to generalize from known data to new situations, and its capacity to access and integrate diverse information sources.
Testing with Semantic Similarity
Reflecting on the inquiries Dr. Crusher poses to the computer in Star Trek exemplifies the necessity of thorough testing. The Dr used this question to test the computer. In a scenario where the computer is unable to access external information beyond its current perception or database. This could imply that the computer interprets the question within the confines of its current operational parameters or situational context, possibly the ship or a specific region it can analyze.
Dr. Crusher effectively simulates technique mirrors the principle of Semantic Similarity testing, where the focus is on assessing the computer’s ability to understand and process questions within the framework of its current operational parameters. The crux of this approach is that Dr. Crusher already has an anticipated answer in mind, similar to how Semantic Similarity testing evaluates an LLM’s response against expected outcomes.
Testing & Grounding
Delving into testing AI, particularly with Language Learning Models and Semantic Similarity, is complex and goal-dependent. I’ll explore this in-depth in future blog posts. For now, here are some resources I’m reading
Semantic Similarity SemScore | Building a Playlist Generator with Sentence Transformers | Perform metrics-based evaluation | Perform automatic side-by-side evaluation | Get text embeddings | Vertex AI Embeddings for Text: Grounding LLMs made easy
What If I’m Not Dr. Crusher with Access to LLMs Using Isolinear Technology?
As I await the future integration of LLMs with Star Trek’s Isolinear technology, exploring the potential of the 1,048,576 tokens in Gemini 1.5 has been an eye-opening journey! Engaging with your metadata is the path forward, a topic I’ll delve into in my next post about Google LookerML & Google Gemini.
Chasing Tokens
Don’t get caught up chasing tokens; begin working on your project today using some of the following options:
📺 Kristopher & Julia hosted an event on Function Calling in Gemini: A Framework for Connecting LLMs to Real-Time Data | https://lnkd.in/emryUHRj
📜 Retrieval-Augmented Generation (RAG) powered by Google Search technology
part 1 https://lnkd.in/eCN93en8 | part 2 https://lnkd.in/eCqe6D52
⏭ Is the next stop 10m tokens, or would combining RAG, Function Calling, and Vertex AI Search be enough? Does it boil down to what will require more computational power?
Thank you Nichelle Nichols, Majel Barrett, Gates McFadden & Whoopi Goldberg
Representation of women in science fiction is crucial; it shapes young minds. I imagine a young Whoopi Goldberg inspired by Lieutenant Uhura managing communications with a futuristic Bluetooth headset. In 1990, I found myself in awe of Dr. Crusher’s ingenious approach to the computer, now referred to as Prompt Engineering.
In memory of Majel Barrett Roddenbery | actress, voice actress & writer
1932–2008
