Revolutionising Design Thinking Workshops and Engagements in WOG with “Eureka!”: A Mini-App Utilising Large Language Models (LLMs).
Introduction: LaunchPad Overview
LaunchPad, introduced by the Data Science and Artificial Intelligence Division (DSAID), is an AI innovation and experimentation platform designed to kickstart agencies’ AI transformation journey. It offers various in-house solutions to help public officers experience the latest features in generative AI and test the capabilities of different large language models (LLMs).
LaunchPad’s Playground, within the LaunchPad platform, serves as a hub for users to access a wide range of LLMs. Here, users can experiment with suggested prompt templates, allowing them to explore the potential and limitations of these models before applying them to their specific use case. This playground environment provides a safe and supportive space for discovery, enabling users to unlock the full potential of language models. An important point to note is that LaunchPad only allows input of data with classification up to Restricted/Sensitive Normal.
Additionally, LaunchPad features an Ideathon platform, where individuals or teams across the Whole-of-Government can come together to propose ideas, identify problems, and brainstorm innovative solutions. Ideathons, conducted within the framework of Design Thinking Workshops, heavily focuses on problem statements, fostering a collaborative and creative environment for idea generation. Through LaunchPad’s Ideathon and Playground, public officers can rapidly prototype and vote for impactful projects, facilitating the implementation of innovative AI initiatives aligned with agency needs. For instance, the Testimonial Generator, now called Appraiser, was a preliminary idea prototyped by the LaunchPad team, and is now launched in beta version.
2. The Objectives of our Mini-App: “Eureka!”
We wanted to enhance the efficiency of the design thinking process and enable users to engage regardless of their location. We aim to create a solution that allows individuals, including non-design thinkers, to self-serve and participate in the design thinking process, even in hybrid workshop settings. By doing so, we can empower a diverse range of users to contribute to innovative solutions. As we explored the potential of LaunchPad’s Playground, we asked ourselves, “How might we leverage its Playground in the ideation process and help agencies discover ideas to their use cases/problem statements, ultimately uncovering new potential use cases where agencies could tap on DSAI solutions and increase government-wide engagement levels?”.
During our research, we came across an enlightening article by Giuseppe Scalamogna, titled “New ChatGPT Prompt Engineering Technique: Program Simulation”. In this article, Scalamogna introduced a fascinating prompt engineering technique that allows ChatGPT to simulate a mini app through a well written prompt.
With this idea in mind, the team from DSAID Engagement Workgroup developed “Eureka!”, a mini app that is built upon a prompt template. The app aids users in the design thinking process for innovative solutions. It provides valuable suggestions on how DSAI solutions can be incorporated, helping users explore the potential of integrating DSAI tools into their solutions. With “Eureka!”, government officers now can engage in design thinking at any time and from any location!
“Eureka!” has also helped us to achieve the following:
i. Guides non-Design Thinkers, speeding up the design thinking process for officers.
ii. Enhances DSAID’s engagement within WOG, uncovering new use cases and highlighting the potential DSAI tools integration.
iii. Streamlines design thinking workshops, reducing the need for extensive manpower.
3. The Process in Creating “Eureka!”
In our real-life journey to understand and address user’s design thinking problems, we discovered the importance of tailoring solutions to their unique challenges. Through engagement with users from diverse backgrounds, we realised the flexibility and creativity were key in adopting different thinking methodologies for their specific use cases. Aligning the generated content with each user’s problem statement became crucial for successful outcomes.
As we delved deeper into the capabilities of LLMs, we encountered the challenge of limited context awareness in LLMs. Without proper context, the generated content often relied on the model’s general knowledge and hypothesis. To address this, we realised the need to establish the right context, allowing the generated content to converge and align more closely with the user’s use case. This personalised approach bridged the gap between the user’s needs and the generated content, enhancing the effectiveness of the design thinking process. As such, “Eureka!” was designed with the following key considerations in mind:
i. Broad Contextualisation: To provide general contextual knowledge, making it applicable to all users, regardless of their profiles.
ii. Seamless User Experience: The program should adhere to the design thinking process, providing clear instructions for a smooth user journey while stimulating critical thinking.
iii. Engaging Ideathon: Detailed, concise, and professional idea generation, helping users to effectively submit their concepts.
4. Challenges Faced in Developing “Eureka!”
One of the main challenges stemmed from the nature of LLMs which we use, as it resembles a black box. These LLMs acted as mysterious agents, with their internal workings and output being “invisible” to both end-users and developers. This lack of transparency posed a significant challenge during troubleshooting, as it made it difficult to identify the exact cause of problems and address them effectively.
In contrast to software development, where traditional programming languages provide developers with control and the ability to program applications using computer languages like Java, C, C++, Python, prompt engineering relied on a high-level programming language: English. Prompt templates are developed with high-level instructions and contexts, allowing LLMs to exhibit a mind of their own. However, the lack of control in LLMs, as compared to traditional programming, has introduced the possibility of inconsistent behaviours across different models and timeframes, which remains a concern as I write this article.
The challenges encountered during the development resulted in the use of a problem-solving technique commonly used in mathematics, the Guess-And-Check approach. This involved diligently verifying the behavior and output after making changes to the prompt template. However, the constrained allocation of tokens and messages for LLMs frequently resulted in exceeding the daily limit, thereby impeding our progress in further development.
5. Tips to Maximise “Eureka!” Output and Effectiveness
Unlocking the full potential of “Eureka!” involves more than just leveraging LLMs. Users must also adopt effective strategies to optimise outputs and ensure effectiveness. In this section, we will share techniques to help you fully utilise “Eureka!”, whether your goal is improving output quality or streamlining user interaction.
Tip 1: No one-size-fits-all solution; tweak the prompt templates to your needs.
Feel free to tailor the prompt template to fit your unique needs and goals. There’s no one-size-fits-all solution. Use the template as a starting point for your ideation process and feel free to diverge from it to let your ideas flow.
Tip 2: Treat “Eureka!” as your Design Thinking Coach.
Engage “Eureka!” as a conversation partner and engage in a dialogue. If the generated content doesn’t meet your expectations, it might indicate a need for more contextual information or precise input. Don’t hesitate to ask follow-up questions or seek clarifications to enhance content output. The more “Eureka!” knows about you, the more effectively it can assist. Here is what you can do when the solutions provided by “Eureka!” doesn’t meet your expectations!
Tip 3: Be precise, detailed, and unleash the power of LLMs with quality responses.
The more detailed your responses to “Eureka!” are, the better it understands your ideas and limits its imagination. This makes the generated content from LLMs more relevant to you!
When comparing both poor and good input, it is evident that LLMs offer more detailed generated content regarding the specific applications of computer vision in solutions. This is in contrast to simply stating the problems it can solve and the benefits it provides, which tends to be superficial.
6. Key Learning: Prompt Templates are akin to Project Scope Documents
A key learning during the development of “Eureka!” is the significance of prompt templates. These templates are akin to project scope documents — they set clear expectations and guidelines for the LLMs. A well-crafted prompt template will ensure the quality of output from LLMs, like how a well-defined scope document lay the foundations for the success of a project. As such, investment of time and effort in crafting a detailed and precise prompt template is essential, as it establishes a strong foundation for effective utilisation of LLMs.
7. Conclusion: Harnessing the Power of LLMs for Design Thinking through “Eureka!”
The arrival of ‘Eureka!’ presents a small step forward in infusing LLMs into design thinking workshops within WOG. It sets the stage for officers coming together across agencies to engage actively in ideation and problem-solving, thereby enhancing traditional boundaries. Through user testing, we have also discovered that prioritising users’ sense of autonomy is crucial. By allowing users to freely interact with the tool without being forced to ideate with it, we create an inclusive environment where individuals can confidently express their own ideas. This approach ensures that the design thinking process is user-centered and empowers participants to contribute their unique perspectives.
However, the journey towards the development of “Eureka!” came with its fair share of complexities. The challenges in navigating the “black box” nature of LLMs resulted in various hurdles. Yet, this aligns with the essence of design thinking process that “Eureka!” seeks to encourage — to seek innovative solutions to address complex issues.
We envision “Eureka!” to continually evolve and foster greater levels of engagement and innovation within the WOG, enhancing the quality and efficiency of public services. “Eureka!” is a testament of the potential of LLMs, and a reminder that although the technology is powerful, user collaboration is still required for true success.
