Building an Enterprise Ready Chatbot with OpenAI Assistants (GPTs) — Part I

Build a low cost experimental architecture to protect your data when using OpenAI Assistants.

Created using Fotor.com

See Part II for details on how to build a custom GPT using a custom API.

TL;DR

• OpenAI Assistants (GPTs) are data leaky, so protect your data.
• OpenAI wants your data, well, the good stuff. Conversation data is more valuable than your raw data. Your raw data is pants. Conversation data is of a much higher quality.
• The users belong to OpenAI. They are not your users. You are providing a service to OpenAI.
• There’s a conflict of interest with regards to the data between all the partners in a chatbot. OpenAI, the creater and the User. This has not been resolved.
• Build a low cost experimental architecture to protect your data when using OpenAI Assistants.
• How to get access to the full conversation

Key Concepts for Building an Efficient and Secure Chatbot:

1. The Data Pull: Understand who wants your data and why
2. Data Leakage Prevention: Given that OpenAI Assistants are leaky, it’s crucial to hide sensitive data behind the API to prevent any unintended data exposure.
3. API Optimisation: Utilise OpenAI Actions to route API calls to your API aggregator. This approach helps in limiting the data returned, thereby reducing token usage. Additionally, customising the API to return only necessary data can further enhance efficiency.
4. Security: Restrict access to specific APIs, ensuring that only authorised requests are processed. This step is crucial for maintaining the chatbot’s security and integrity.
5. Monitoring: Regularly monitor logs, traffic, and requests. This oversight is vital for identifying potential issues and optimising performance.
6. Learning Engine: Building that knowledge about your customers and their use of your chatbot. It’s not what you think it is.
7. Data Compression (to be tested): We’re exploring the possibility of compressing traffic to streamline communication between the chatbot and servers. This aspect is still under testing for its effectiveness and impact.

The Data Pull

There are three crucial users in the data ecosystem of a chatbot assistant: the chatbot provider (OpenAI), its OpenAI chatbot users, and the chatbot creator.

‘Privacy’ is a priority area where all three entities intersect. This is a mutual commitment to ensuring privacy, pointing to OpenAI’s focus on maintaining user confidentiality while fostering meaningful conversations.

This creates a conflict for the creator, as they do not have access to the conversation and are unable to ‘Learn’ from this, putting the creator at a disadvantage. Why would you provide your proprietary information to the platform. What are the benefits to the creator?

This highlights a tension or challenge in balancing the data requirements of OpenAI, users, and creators. This needs a reevaluation of how data needs are addressed to satisfy all parties without compromising individual objectives or privacy standards.

Data Leakage Prevention

OpenAI Assistants are leaky, it’s crucial to hide sensitive data behind the API to prevent any unintended data exposure. It’s easy for anyone to get a copy of your prompt and source data held within OpenAI. So you should use obfuscation techniques to hide the data and prompt from the user.

By implementing OpenAI Actions, it’s possible to ‘hide’ some of the prompt behind and API, and by using an API Aggregator, it’s possible to ‘hide’ the API from the users.

API Optimisation

OpenAI custom actions allow access to external APIs. Actions allow GPTs to integrate external data or interact with the real-world. Connectibf GPTs to databases, plug them into emails, or make them your shopping assistant. For example, you could integrate a travel listings database, connect a user’s email inbox, or facilitate e-commerce orders.

By using an API aggregator instead of connecting directly to your APIs you can help to limit the data returned, thereby reducing token usage. Additionally, customising the API aggregator to return only necessary data can further enhance efficiency.

For example, if you are using an API aggregator that returns a list of all users, you could use OpenAI Actions to route the call to your API aggregator and only return the data for the users that you are interested in. This would reduce the amount of data that is returned and, therefore, the number of tokens that are used.

Similarly, you could customise the API aggregator to return only the data that you need. For example, if you are only interested in the user’s name and email address, you could customise the API to only return that data. This would further reduce the amount of data that is returned and, therefore, the number of tokens that are used.

You could also use an API aggregator to combine data from weather APIs, traffic APIs, and news APIs to provide the data required by the chatbot.

Overall, customising your APIs can help to reduce the amount of data that is returned and, therefore, the number of tokens that are used. This can lead to improved efficiency and cost savings.

Security

Don’t let OpenAI eat you for breakfast, lunch and dinner. In this experimental architecture we use obfuscation and separation to protect your data.

Monitoring

Monitoring is enabled at all layers within the architecture, some of this is automated and some can be implemented in code in your API Aggregator.

CloudFlare Analytics

Replit Analytics

Learning Engine

More on this in another post. It’s not what you are thinking. See here for some insight.

Experimental Architecture Pattern

This diagram illustrates a complex, multi-tiered approach to data handling and security.

At the foundation of this experimental architecture pattern is the prioritisation of protecting your data and your infrastructure services.

There are reportedly over 100 million users of OpenAI and a popular chatbot could overwhelm your current infrastructure, and we know many are hacking chatbots to get access to your prompts and data sources.

This architecture may support any future developments by OpenAI, as suggested by their patent applications suggesting that OpenAI is gearing up for more advanced AI models that will be capable of pulling in and utilising this secure, high-quality data.

Security (Cloudflare)

Central to the security mechanism is Cloudflare, which acts as a robust defensive layer, providing security features such as DDoS protection, firewall services, and rate limiting.

By using the Cloudflare DNS, we can also hide the IP addresses of our API services, protecting them from potential DDoS attacks.

This ensures that data is not only secure from external threats but also controlled in terms of access and flow.

Cloudflare provides us with the following capabilities:

• IP rules to only allow access via OpenAI to the API services
• Rate limiting to protect the data APIs and data sources
• DDoS protection
• DNS for API services
• Enforce TLS and HTTPS
• Caching
• Monitoring, logging and analytics
• Potential to expand the service in the future to use Cloudflare services, such as, Workers, R2, etc.

API Aggregator (Replit)

This architecture involves an API layer provided by Replit, a collaborative browser-based coding platform. This API serves as a bridge between OpenAI and a variety of data sources, providing integration of external data into OpenAI’s ecosystem. The APIs facilitate the flow of data from multiple sources, allowing OpenAI to leverage a vast pool of information while maintaining data integrity and security.

This allows you to create a specific API for the data needs of the chatbot, only returning the data needed for the query.

Therefore;

• Combine APIs where required to provide the required information. This reduces the need of the chatbot to make multiple API calls using actions.
• Implement strategies to reduce the amount of tokens returned to OpenAI. Only return the data needed for the query. Reducing token count and latency
• Reduce latency using caching
• Provides the opportunity to leverage different prompting techniques and algorithms that are not currently available via OpenAI.
• Compress traffic? (to be tested)

LLM (OpenAI)

This architect currently supports OpenAI, but this could easily be swaped out for another LLM. LLM capabilities are evolving at lightning pace and we need to ensure can we support any future developments.

Data API

These are your existing APIs to access your data and services

Data Source

These are you current source of data.

Conclusion

Building an enterprise-ready chatbot with OpenAI requires a multifaceted approach. By focusing on efficient data handling, security, and monitoring, businesses can leverage the full potential of AI while ensuring data privacy and operational efficiency. Stay tuned for more updates and insights as we continue to explore the frontiers of AI in business applications.

Implementation of this architecture to follow. Stay tuned, including how to get access to the full conversation.

Now check out Part 2