Why I’m building my own AI Agent library
Last month, I started diving deep into the world of AI Agents. While exploring this field, I had a revelation: I was going to work on AI Agents from now on. In fact I was so enthralled with AI Agents that I did two things. First, I wrote a blog post about it, where I shared my enthusiasm for the field. Second, I began developing my own AI Agent library in Python.
Lately, I’ve been pondering on the second point. Why develop my own AI Agent library when there are plenty of available options out there?
After some reflection, I believe I am now able to articulate the reasons why I’m building my own AI Agent library. This post will cover the 3 main reasons why I’ve gone down this route.
There are no standards yet
There are countless libraries and AI Agent frameworks out there. The most popular ones include BabyAGI, CrewAI, LangChain (including LangGraph), and AutoGPT. Yet, there is no standard on how agents should be developed. The field is quite new and there is no “leader” who sets the pace. Therefore, more innovation and attempts to implement agents should be encouraged.
Among the libraries and frameworks I cited, LangChain is probably the most well known out there. But it also has its fair share of problems and detractors. It feels bloated: when installing LangChain you also install a lot of other dependencies. Additionally, developers have also complained about its slowness when running in production. I’ve also read, and experienced the frustration of going through the myriad of abstraction layers that must be peeled away to understand what the code does under the hood. Finally, others have highlighted security concerns with using LangChain.
Despite the negative sentiment in some developer circles, I still intend to experiment with LangChain in the future and draw my own conclusions. From my perspective, I believe it could be a useful library for quickly prototyping agentic systems.
Since there no standards, I believe the best approach is to build my own AI Agent library to familiarise myself with the field, incorporate the best parts from other libraries and frameworks as I see fit, and keep experimenting to determine what works and what doesn’t.
I want to learn from scratch
I like to understand concepts using a first-principles approach as much as possible. In this context it makes sense to build my own AI Agent library from scratch.
Building it from scratch means going beyond the layers of abstractions erected by other libraries and frameworks, to get to the nexus of agentic code and workflows.
Understanding the inner workings of agentic systems includes gaining a solid grasp of these elements:
- effective prompting
- capabilities of underlying LLMs being used
- planning
- tooling
- memory
- reflection
The library I’m building may only be in its infancy, yet I’ve learned a good deal about the above elements. And there’s still much to learn. I simply don’t believe I would have gained as much insights into these agentic system components had I opted for an already existing library or framework.
It’s fun
I wrote it in my previous blog post, and I write it again here: I haven’t had this much fun in years. After the disappointment of EnVsion and a first quarter of 2024 that felt somewhat directionless, I’m glad to have found a new sense of direction.
The momentum around agents, and the general excitement in this field is contagious. Personally, it reminds me of the time when I was fascinated with self-driving cars. I ran with this passion at the time and completed Udacity’s self-driving car nanodegree. I feel I should have doubled down on this semi-quixotic interest at the time and join an interesting company in the autonomous vehicle space. I didn’t for various reasons: I had just bought a property in the UK, the most promising self-driving car companies were US based, and my partner and I were planning to start a family.
I am dead set on not missing out on the AI Agent party. I’m here to play, and I’m here to stay.
Building this AI Agent library, and learning about AI Agents in general, feels like play to me. So I can play in hard mode, despite having a demanding full-time job and parental responsibilities. Moreover, I don’t expect my library to become the gold standard for building agentic systems, bit it makes me proud to contribute a little something to the open source community.
Closing words
While I am building my own library for now, I don’t exclude adopting a more popular/standard/powerful third party one in the future. This could make sense once I decide to build a product with AI Agents and realise that my library is too limited for the use cases I have in mind.
And since speed is crucial in startups it may not be wise to get bogged down tinkering with a library when there are already good ones out there can help deliver value to the customer early and fast.
I’m also unlikely to build my own eval framework. I am not sure about monitoring, orchestration either. I reckon it could make more sense for bigger companies or those selling to enterprise to build build their own evals or at least make sure that data/prompts stays in their systems.
This is a young space, and I’m excited to be among the early explorers skirting at the edges of this movement. And I hope some of my excitement trickled down towards you. You can find out more about the AI Agent library I’m building by clicking on this Github link.
If you’re on Twitter/X then do follow me to read my daily thoughts on AI Agents.
