Blogging with AI Agents
As a conjurer of AI agents and agentic systems, I delve into using agents for everything. I have made agents do everything from creating PowerPoint to video animations from images, gifs from videos, Word and Excel documents, and more. I even have agents who can research topics, reorganize and summarize the research, create a post, and post it on Twitter and Reddit.
Given the right tools, agents can automate many sophisticated tasks. While I will admit they don’t always get it right, they are more than capable helpers. I wanted to demonstrate this by creating an agent that could automatically create a blog based on my writing, so I made a blog helper agent that does the following:
1. Extract all the images, text, and code from a source Word document. In most cases, this is a chapter from one of my books.
2. Compose a blog post given a rigid template.
3. Generate a Word document for the post, including images and code.
I then take the contents of the Word document, review it, and post it on Medium or wherever. To demonstrate the output of this agent I submit below a generated blog post from Chapter 8 from my book AI Agents In Action. This post is written in the style of Ernest Hemingway, which upon reflection may not be a good style for a technical article.
The blog within the blog (AI generated from source)
Artificial Intelligence is growing by leaps and bounds, like a young sapling straining towards the sun. One of its most exciting branches is the development of agentic systems — agents infused with memory and knowledge that can interact more naturally and effectively with us. Let’s delve into how retrieval plays a crucial role in making these systems tick.
Introduction
In our ever-evolving world of AI, the ability to retrieve information is foundational. Whether it’s about drawing from vast databases or pulling snippets of past interactions, retrieval shapes the context and enhances the capability of AI systems. This journey will cover retrieval across semantic search, document indexing, and the intricacies of retrieval-augmented generation (RAG) workflows.
Retrieval is more than just fetching data; it’s about understanding and contextualizing information. The synergy between memory and retrieval mechanisms enables AI to simulate human-like patterns of understanding and response. By delving deep into how retrieval works, we can better appreciate the artificial minds we are constructing.
In this blog, we’ll explore the critical aspects of retrieval in AI applications. From document indexing to vector databases, every step in the process plays a crucial role in making these systems smarter and more efficient. With examples and code snippets, we’ll walk through the practical applications of these concepts, giving you a comprehensive understanding of its significance.
Basics of Retrieval Augmented Generation (RAG)
The RAG mechanism swings into action when you ask a question about a sophisticated AI system. It takes your query and dives into preloaded documents, transforms them into context chunks, embeds them into vectors, and stores them in a vector database. When you query again, it compares the query with these stored vectors and pulls the most relevant chunks to form an answer.
RAG, or Retrieval Augmented Generation, combines the strengths of retrieval systems with generation models. The key here is context. It can generate more accurate and meaningful responses by providing more relevant and detailed context to the language model. This process involves several steps, from document loading to querying an embedded vector database.
The effectiveness of RAG lies in its ability to handle vast amounts of unstructured data and provide coherent and contextually accurate answers. This makes it particularly useful for applications such as document chat systems and question-and-answer systems where precise and relevant responses are critical.
import plotly.graph_objects as go
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
documents = [
"The sky is blue and beautiful.",
"Love this blue and beautiful sky!",
"The quick brown fox jumps over the lazy dog.",
"A king's breakfast has sausages, ham, bacon, eggs, toast, and beans",
"I love green eggs, ham, sausages and bacon!",
"The brown fox is quick and the blue dog is lazy!",
"The sky is very blue and the sky is very beautiful today",
"The dog is lazy but the brown fox is quick!"
]
vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(documents)Delving into Semantic Search and Document Indexing
Document indexing is like creating a detailed map of a rugged terrain. It transforms information into a structured format, making it easier to retrieve. Semantic search, on the other hand, goes beyond the words to grasp the meaning, making your search more intuitive and accurate.
Semantic search relies on understanding the context and semantics of the words in a document rather than just matching keywords. This makes it extraordinarily potent, especially in complex queries where meaning precedes exact word matches. Techniques like vector similarity searches can compare the semantic content of documents, leading to more accurate and relevant results.
The process begins with transforming a document into a semantic vector, a numerical representation that captures the meaning of the text. Various algorithms, such as TF-IDF, are used to create these vectors. Understanding how these vectors work and how they can be searched is essential for building an efficient retrieval system.
cosine_similarities = cosine_similarity(X)
while True:
selected_document_index = input(f"Enter a document number (0-{len(documents)-1}) or 'exit' to quit: ").strip()
if selected_document_index.lower() == 'exit':
break
if not selected_document_index.isdigit() or not 0 <= int(selected_document_index) < len(documents):
print("Invalid input. Please enter a valid document number.")
continue
selected_document_index = int(selected_document_index)
selected_document_similarities = cosine_similarities[selected_document_index]
Vector Databases and Similarity Search
Once vectorized, these document vectors are stored in a vector database. This storage allows fast similarity searches, enabling the AI to fetch relevant contexts swiftly. The ability to perform efficient similarity searches ensures the AI remains responsive and relevant.
Vector databases are pivotal in managing and querying large volumes of vectorized data. They enable the efficient retrieval of similar vectors using various similarity measures, such as cosine similarity. This efficiency is vital for real-time applications, where prompt and accurate data retrieval is critical.
This process involves several layers, starting from loading the data, transforming it into vectors, and storing these vectors in a database optimized for similarity search. Each step in the process is crucial for maintaining the fidelity and speed of the retrieval system.
Constructing RAG with LangChain
LangChain offers powerful tools to implement RAG workflows. It abstracts retrieval processes, making them accessible and efficient. Using LangChain, we can build robust, scalable retrieval systems that enhance the AI’s knowledge and memory capabilities.
One of LangChain’s critical aspects is its ability to handle various data sources and vector stores seamlessly. It provides comprehensive tools for data loading, vectorization, and querying, making the development process straightforward and efficient. This flexibility is essential for developing scalable and versatile AI systems.
Creating a RAG workflow with LangChain involves several steps. First, documents are loaded and split into manageable chunks. These chunks are then transformed into vectors, typically using models like OpenAI embeddings. Finally, these vectors are stored in a database, ready to be queried for similarity search when needed.
from langchain.document_loaders import UnstructuredHTMLLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
loader = UnstructuredHTMLLoader("sample_documents/mother_goose.html")
data = loader.load()
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=100,
chunk_overlap=25,
length_function=len,
add_start_index=True,
)
documents = text_splitter.split_documents(data)
Implementing Memory in Agentic Systems
Agentic memory incorporates various forms — sensory, short-term, and long-term — to enrich interactions. Each type of memory enhances the AI’s capability to contextualize responses based on past interactions, making conversations more natural and meaningful.
Sensory memory in AI processes immediate inputs like text, images, and audio, providing rapid but temporary contextual information. Short-term or working memory holds a limited amount of recent input, allowing the system to maintain context over the course of an interaction. Long-term memory stores more durable, relevant semantic and episodic memories which can be recalled over extended periods.
Memory retrieval in AI systems follows a similar pattern to RAG but focuses more on the dynamic and evolving nature of accumulated information. By effectively categorizing and managing these different forms of memory, AI systems can simulate a more nuanced understanding, similar to human cognitive processes.
Conclusion
The journey through retrieval in agentic systems reveals a rich landscape where memory and knowledge intertwine. Each step enhances the capability and depth of AI interactions, from semantic search to advanced memory implementations. As we continue to explore these terrains, the potential for more intelligent and intuitive systems burgeons, promising a future where AI doesn’t just respond but understands.
My Conclusion
Development of the blog agent took about a day using the GPT Assistants Playground and most of that was spent generating code to extract content from Word documents correctly. The agent needs more work but overall I was generally happy with the results. If you like or dislike the output please let me know.
The tools
All the tools used to create the inner blog are available from my project GPT Assistants Playground. https://github.com/cxbxmxcx/GPTAssistantsPlayground
I am continually adding new tools/actions to the Playground all the time.
