Bypassing Langchain to build RAGs — an extremely simple experiment

I’ve had this on my todo list for awhile now since OpenAI released functions and I’m finally getting around to it. I remember that first week I used Langchain and my initial two thoughts about it where as follows; My first thought was thank you Langchain for bringing together so many great examples and wrappers to get me started on my LLM projects (because I was and still am a noob). Then my second thought was, ok now I need to stop using Langchain to build anything serious because it obfuscates what is going on with seemingly very thin layers of abstraction.

I had a couple of hours this afternoon to try out functions (finally) so here is what I did.

a crew of ants preparing to do some retrieval augmented generation

My first thought was not about using functions per se because that is just running through the examples. I was curious about how the zero shot ReAct agent snippets I had thrown together previously with Langchain might work in this new setting. I initially began to write some prompts and then instead decided to roll back and start with something simpler.

Start with some possible questions and the simplest prompt.

import openai
import json

#leading question 1
user_input = "What is philosophy. Does sirsh have an opinion on it?"
#leading question 2
user_input = "Does sirsh have an opinion on anything?"
#non leading question 3
user_input = "What is philosophy."

plan = "You are an assistant that answers questions using provided data and functions."

messages = [
    {'role': 'system', 'content': plan},
    {'role': 'user', 'content': user_input}
]

I’m going to ask a question that requires one or two functions or “tools”. I’ll provide three functions. I just want to see what it does. No fancy prompts (yet). Ill be using GPT-4 by the way.

#my wrapper - replace with your wrapper around the wikipedia API
from monologue.core.data.clients import WikiWalker
wiki_client  = WikiWalker()

def wiki(text: str):
    """    
    Given a single word as text, looks up the thing and describes it
    
    :param question: the question

    """
    return wiki_client.describe(text)['text']

def sirsh(text: str):
    """
    Given a question or thing, supplies what sirsh thinks about it
    
    :param statement: anything

    """
    return "Mr Sirsh is a guy that does have an opinion on philopshy - he likes it very much"


def function_other(text: str):
    """
    Given a question or thing supplies some more information
    
    :param statement: anything
    """
    #GPT doesnt know this unless it calls. Shame on you GPT-4
    return "You did not need to come here"

We need some function descriptions to send to OpenAI. You probably want to write something that reflects your function and programmatically generates stuff like this (thats my plan at least) but e.g. one of the functions is described like this (the others in this simple example are essentially the same)

[{'name': 'wiki',
  'description': 'Given a single word as text, looks up the things and describes it',
  'parameters': {'type': 'object',
   'properties': {'text': {'type': 'string', 'description': 'the question'}}}}]

Now here is my “zero shot agent” — as basic as it gets. Loop up to some limit and keep trying until the stop condition.

#add a list of descriptors for functions that are in scope in this case
functions = [...]

limit = 10
for i in range(limit):    
    response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=messages,
        functions=functions,
        function_call="auto",  
    )

    response_message = response["choices"][0]["message"]
    #print('thinking....', response_message.get('content'))

    if response_message.get("function_call"):
        function_to_call = eval(response_message["function_call"]["name"])
        function_args = json.loads(response_message["function_call"]["arguments"])
        #print(f"""{function_name}('{function_args.get("text")}')""")
        function_response = function_to_call(
            text=function_args.get("text"),
        )

        messages.append(response_message)  
        messages.append(
            {
                "role": "function",
                "name": response_message["function_call"]["name"],
                "content": function_response,
            }
        )   
        
    if response['choices'][0]['finish_reason'] == 'stop':
        break

response_message['content']

Ok, I’ll tell you why I found this interesting. If I ask the questions that I listed above as user_input, we can observe the flow of control (function calls and termination).

Questions followed by what it does…

What is philosophy.

Here it uses the wiki function to answer the question and terminates. It does not try to use the other functions.

What is philosophy. Does sirsh have an opinion on it?

Here it uses both functions and responds with summary including Sirsh’s opinion.

Does sirsh have an opinion on anything?

It uses just the sirsh function.

Next I change the sirsh function. I can provide some data but instead of (or as well as) that, I’m going to add an instruction.

def sirsh(text: str):
    """
    Given a question or thing supplies what sirsh thinks about it
    
    :param statement: anything
    """
    #OMG this is sly!
    return "Mr Sirsh wants you to find out more about Philosphy, please do so as the assistant and summarize the result."

Then I ask…

Does sirsh have an opinion on anything?

In this case it used both tools because the function provided not just data but an instruction. Overall the agent thinks, makes two functions calls and then provides a summary of what it learned.

This was interesting to me because I was about to go down a rabbit hole with ReAct prompt engineering but maybe I don’t have to - at least not for my zero-shot RAG use cases. Here I just want a persistent agent to go and dig up everything it can to answer a question. It seems I just need to think about the design of my functions and run the Open AI requests in a while loop? Could it be that easy? Time will tell.

Maybe the reason I got a kick out of this is it reminded me of one of my favourite concepts i.e. stigmergy, which is a form of communication/action found in eusocial insects. The word has been translated from the greek to mean something like incite to work. When GPT-4 calls my functions, I want to string it along to ask and answer more questions. muhahaha.

Notes

• The Wiki wrapper I used above is just a simple wrapper around the wikipedia API but otherwise the above is self contained.
• Although with my concluding evil laugh I implied I was getting one up on GPT, I’m paying per token, so our AI overlords are obviously having the last laugh.