Reducing Stochasticity in Corporate Applications of Language Models Through Semantic Programming
In this article, I explore the challenges of integrating Language Model (LM)-based prompts into mature corporate workflows that must adhere to stringent compliance requirements. These challenges include navigating corporate compliance demands and managing the inherent randomness (stochasticity) of model outputs. To address these issues, we have developed SIKE (Secure Intelligent Knowledge Engine), a language model orchestration platform equipped with a suite of corporate-friendly tools. These tools are designed for developing and seamlessly integrating prompt-based workflows in compliance with regulatory standards.
To contextualise the role of SIKE, it’s helpful to compare it to an operating system. Just as an operating system manages processes, memory, file systems, devices, and user interfaces around a central kernel — linking the CPU and RAM to other system components — SIKE operates similarly. It integrates a collection of modules around a foundation model, such as GPT-4, Gemini, or Claude, to streamline the development of Language Model (LM) based workflows.
Key design features of SIKE include:
- Conditional execution of prompt sequences,
- Modular knowledge bases linked to file systems, databases, and APIs,
- Permissions-based access to prompts and knowledge bases,
- Effective cost management,
- Web API access for executing workflows.
The Problem
When a Language Model executes a statement, it implicitly performs two key functions. First, it selects the context of the statement; second, it defines the steps necessary for execution.
Defining the context accurately is paramount, and the most effective method to ensure precision is by explicitly incorporating all the relevant information needed for execution. This is where Retrieval-Augmented Generation (RAG) plays a crucial role in addressing these challenges.
Our view is that RAG is not only important for augmenting a model, but also for the specifying context even the model has already been trained with the relevant data.
Most natural language statements are ambiguous enough to have multiple potential solutions. Decomposing the statement into sub-statements facilitates a clearer and more targeted path of execution. Failing to break down the statement forces the Language Model to independently determine the execution path, introducing an element of randomness. This issue is effectively mitigated by using PromptFlows to daisy-chain prompts, which guides the model through a structured execution process.
Crafting reliable prompts is still quite a dark art full of mysticism and wizardry. Our research aligns with much of the existing literature, which suggests that to mitigate model stochasticity, it is crucial to break down complex tasks and precisely define the execution context. For this purpose, sequential sub-prompt execution is essential when addressing more complex problems.
Visual Representation
When discussing the issue of context and execution ambiguity in natural language processing using a 2-dimensional plane, we visualize the challenges faced by Language Models (LMs) in understanding and responding to statements. Imagine a plane where:
- The bottom axis (X-axis) represents the range of possible contexts a statement could refer to. This dimension spans the variety of interpretations or meanings that any given statement might have, depending on its phrasing, the content, and even the external knowledge or data available.
- The height axis (Y-axis) represents the possible execution paths that the solution to the statement could take. This dimension captures the range of actions or responses the LM might generate based on its understanding of the context.
In this visualisation, the entire area of the plane symbolises the total solution space within which the LM operates. Each point on the plane would represent a unique combination of a specific interpretation of context and a corresponding execution path.
In scenarios with high stochasticity, or randomness, the plane is vast and widely varied, indicating a broad solution space. This is due to the LM’s potential to generate multiple, diverse outputs for a single input, especially when that input lacks specificity or is broadly applicable.
To reduce stochasticity we aim to guide the LM by specifying both the context and the execution steps more clearly. This process involves:
- Specifying Execution Paths (Constraining the Y-axis): By clearly defining what actions or outputs are expected or by outlining the steps the LM should consider in its response, we limit the possible execution paths. This is akin to reducing the height of the plane, focusing the LM on fewer, more precise execution trajectories.
- Narrowing the Context (Constraining the X-axis): By providing more detailed, specific, or contextual information in the input or prompt, we limit the range of contexts the LM might interpret. This can be visualised as reducing the width of the plane, focusing the possible contexts to a narrower, more relevant subset.
By constricting both the X and Y dimensions, we effectively shrink the total solution space of the plane. This reduction not only makes the LM’s outputs more predictable and relevant but also improves the effectiveness and efficiency of the model, particularly in structured or critical applications such as in legal, medical, or technical domains where precision is crucial. This strategy essentially guides the LM to function within a more defined and controlled operational space, enhancing both the quality and applicability of its outputs.
Examples
- Ambiguous prompt: “Write about the interface” — This prompt is very ambiguous because “interface” can refer to many different things depending on the context. It could mean a user interface, a programming interface, a physical connection between devices, or even a metaphorical interface in communication
- Better prompt: “Write a detailed analysis of the user interface design principles used in the latest version of the iOS mobile operating system, focusing on accessibility features and improvements compared to the previous version.” — This version is much more specific, outlining the exact type of interface (user interface for iOS), the aspect to focus on (design principles and accessibility features), and a comparative element (improvements from the previous version), which gives clear guidance on what the response should include.
- Sequential decomposed prompt:
- Introduction to iOS UI Design Principles:
- What are the fundamental user interface design principles that iOS adheres to?
- How do these principles manifest in the latest version of iOS?
- Overview of the Latest iOS Version:
- What is the latest version of iOS?
- What are the key features and updates introduced in this version?
- Accessibility Features in the Latest iOS Version:
- What accessibility features are available in the latest iOS version?
- How do these features enhance the usability of the operating system for users with disabilities?
- Comparison of Accessibility Features:
- What were the accessibility features in the previous version of iOS?
- How have the accessibility features improved or changed in the latest version?
- Detailed Analysis of Specific UI Changes:
- Identify specific user interface changes in the latest iOS version that impact accessibility.
- Discuss how these changes adhere to or deviate from established UI design principles.
- User Feedback and Impact:
- What has been the user response to the new UI and accessibility changes?
- How have these changes affected the overall user experience, particularly for users requiring accessibility accommodations?
- Conclusion and Future Directions:
- Summarize the effectiveness of the current UI and accessibility features.
- What future improvements or changes could be made to enhance accessibility and user experience in iOS?The natural question after seeing this example is, how do I efficiently execute this prompt sequence in a secure way?
PromptFlows and Semantic Programming
PromptFlows, a YAML-based Domain Specific Language (DSL), serves as the programming mechanism for SIKE, facilitating the design and execution of prompt-based workflows.
Executing natural language commands through language models (LMs) has proven exceptionally effective for complex reasoning tasks. However, this approach often falls short in rigid execution pipelines due to its inherent randomness.
Combining natural language with a strict, deterministic software-based language can significantly enhance effectiveness.
PromptFlows empowers prompt engineers to define exact execution contexts, thereby ensuring reliable processing of natural language prompts. Below is an example PromptFlow YAML script:
- history: 5
- context: $personal$
- thread: 1
- model: gpt-4
- prompt: first
rag:
- module: $personal$
description: |
1. "Personal" is a RAG tool.
2. Use the "Personal" RAG tool to search for any topic or context related to football.
3. If there are more than one subject, create several queries, one query per subject.
4. For example if the query is about A and/or B then first ask about A then B.
- module: crypto_outlooks_2024_2
description: |
1. "Crypto Outlooks 2024" is a RAG tool.
2. Use the "Crypto Outlooks 2024" tool to search for any topic or context related to crypto in 2024.
3. If there are more than one subject, create several queries, one query per subject.
4. For example if the query is about A and/or B then first ask about A then B.
tags: [a16z, Gemini]
temperature: 0
instructions: |
Extract the relevant topics / subjects from the command below and use the tools for relevant information needed to execute the command.
If you don't find a topic or context being specified in the command then search for "general descriptions" in EVERY SINGLE one of the relevant RAG tools separately.
Make sure you use every single RAG tool when the topic is general.
Use multiple tools when relevant.
If there are multiple topics / subjects in the command, use the tools with each topic / subject separately.
ONLY answer based off information available in the tools or the chat history. If the answer is not in the tools say you don't know.
DO NOT refer to the tools name in the response, if the tools is useful then execute it and use the output.
ONLY respond in simple markdown text.
Check that the response only pertains to information found in the tools.
---
what is this about?
- prompt:
action: Translate
language: Swedish
- prompt: last
action: Format
format: MarkdownThe YAML script outlines a sequence of actions within a given context and thread, linked to specific tool modules for information retrieval and augmentation. The script sets up a scenario where the model (GPT-4) refers to the last five interactions within this thread to ensure continuity and relevance in responses.
The primary action involves using two specialized tools designed for gathering and generating responses based on predefined knowledge bases, tagged as “$personal$” and “crypto_outlooks_2024_2”. The “$personal$” module is tailored for inquiries related to football, whereas the “crypto_outlooks_2024_2” module focuses on cryptocurrency forecasts for the year 2024, specifically highlighting vectors tagged with “a16z” and “gemini”.
The instructions dictate that the model should extract relevant topics or subjects from user commands and apply these tools appropriately to fetch or generate the required information. If a command pertains to a general topic, all relevant RAG tools should be utilized separately to provide comprehensive coverage.
The process concludes with additional commands to translate the output into Swedish and format the result in Markdown, ensuring that the final presentation is both linguistically and visually appropriate.
PromptFlows is designed to facilitate the sequential processing of tasks through what is known as “daisy chaining” of sub-prompts. This method allows the outputs of earlier sub-prompts to be dynamically incorporated into subsequent ones, creating a streamlined flow of data and operations within a single session.
- context: $personal$
- thread: 1
- model: gpt-4
- prompt: first
temperature: 0
instructions: what is one plus one?
- prompt: second
temperature: 0
instructions: multiply $first$ by 2
- prompt: third
temperature: 0
instructions: divide $second$ by $first$In this specific example, the process involves three sub-prompts within a defined context and thread:
1. First Sub-Prompt: This initial prompt calculates a simple arithmetic operation — asking “what is one plus one?” The answer to this question (which is 2) sets the baseline for the next operation. This sub-prompt’s output is identified by its name ‘first’, which can be referenced in subsequent prompts.
2. Second Sub-Prompt: The next step takes the result from the ‘first’ sub-prompt, referenced as `$first$`, and uses it as input for a multiplication operation. Specifically, it multiplies the output of the first sub-prompt (2) by 2. The result of this operation is 4, and this output is now labeled as ‘second’.
3. Third Sub-Prompt: The final step involves dividing the result of the ‘second’ sub-prompt by the result of the ‘first’ sub-prompt, effectively using `$second$` divided by `$first$`. This calculates 4 divided by 2, resulting in 2.
This chaining mechanism demonstrates how outputs from previous prompts can be seamlessly used as inputs in following prompts, enabling complex sequences of tasks to be handled efficiently in a modular and error-reducing manner. Each step builds upon the previous, illustrating a powerful tool for conducting sequential data processing and decision-making tasks within an interactive model environment.
Knodules
SIKE enables users to modularly define knowledge bases, termed ‘Knodules,’ which are tailored with specific access rights in line with corporate compliance requirements and IAM privileges. These Knodules form the foundation for all information access within SIKE.
Each Knodule contains information in the form of vector representations, facilitating its integration into prompts through Retrieval-Augmented Generation (RAG) models. Additionally, each vector is tagged with custom labels to enhance search precision during RAG queries.
The combination of Knodule design architecture and PromptFlows allows prompt engineers to create rich, dynamic, and detailed descriptions of prompt contexts. This enables an in-depth execution process tailored to effectively solve complex tasks.
Conclusion
The integration of Language Models (LMs) into corporate workflows, while fraught with challenges, presents significant opportunities for enhancing human efficiency and compliance adherence.
SIKE, as a sophisticated orchestration platform, delivers the potential of semantic programming to refine the application of LMs by reducing stochasticity and improving output reliability. Through tools such as PromptFlows and Knodules, SIKE not only allows developers to addresses the inherent ambiguities in natural language processing but also pioneers a structured approach that can be scaled across various industries.
We have shown that constraining the solution space through precise context definition and execution specification, SIKE ensures that LMs can operate within a more predictable and effective framework. This strategic containment of variability is crucial for corporations that operate under strict compliance and accuracy requirements. As we continue to evolve these technologies, the goal will be to further refine these models to meet the specific needs of different sectors, ultimately leading to broader adoption and more robust, dependable systems.
