A Personal Evaluation of Using Vibe Coding Tools
We are in the 6th industrial revolution. Like it or not, this is what I’m calling this transition we are currently in. AI is making changes to so many workflows, and it’s obvious that everyone will be having this integrated in their lives in one way or another.
Human Agent vs AI Agent
So here are my two cents on this Vibe Coding Thing that directly impacts us Software Engineers. Now that I think of it, I’ve been vibe coding before the AI boom happened. Most often, I had my Human Agents write code for me. I explained to them my expectations and they wrote some pieces of code after a few hours. Then I reviewed them and either approved them or asked them to rewrite. Sometimes I told them how the architecture of the application should be, sometimes I didn’t. Sometimes they exceeded my expectations and sometimes they just did straight out dumb shit that to me looked dumb. (Obviously, they were too young to know why it was dumb.) Anyway, so my Human Agents (We call them junior engineers or interns, or trainees) were slow. At least compared to AI they areas slow as a snail.
Then they came. The AI Agents. And now I can tell the agents to write code for me. And honestly
- They are Fast.
- They are cheap. (The throughput is astronomical compared to some junior)
- They know the basics better than any junior.
Anyway. These AI agents are 100% Better than any junior Human Agent.
There’s hardly any difference in the prompt I used to give my juniors vs what I now have to type or voice message to some AI.
My entire career revolves around Unity and C#. I’ve always wanted to dabble in some web development. But heck, I ain’t gonna spend time understanding how JavaScript works. I think at least 2 new frameworks got released while I’m writing this article. But hey, now I don’t need to do any of that. AI can learn for me. They can know how the framework works, and I can simply learn the basics of the framework and put my engineering skills to work.
Everyone started coding with ChatGPT. I did too. I told GPT about what class I want to write, what methods it should have, and what properties it should have. It wrote me my class. Basically, I used it as my typewriter. AI is exactly that, a glorified typewriter or maybe an evolution of autocomplete.
Then one day I was assigned to arrange a Learning event called Learnathon. Now this article isn’t about that so ain’t gonna waste my time. But if you are interested do check this website. However, this is how I was introduced with bolt.new . Basically what I wanted to do is build some kind of onboarding platform for the participants as quickly as possible using some AI. After some good old google searching I found this tool called bolt.new which took in some prompt and built entire apps or so it promised. To my disbelif I actually saw it deliver. Or at least it did deliver what I needed at the moment. It didn’t have the built in Supabase integration, so I had to use Neon DB for the backend. Fun fact is my knowledge about react was as good as a fresher nah even lower, that of a grad student.
But there it was my beautifully looking platform that I actually used to onboard the participants. And no it wasn’t just some form. It onboarded them to GitHub Organization , Discord , provided them with a JetBrains coupon, assigned a mentor, created GitHub repo for them and assigned the mentor along with their team members to the repo. And finally threw a confetti animation in the end.
And all of that with just prompting to AI. If I were to get one of my web devs to built it it’d take at least 10 workdays. Even then it’d have 10 bugs. The app didn’t have random bugs. And no, I’m not joking It was used to onboard 500 people.
And then I realized. WOW! I can now get rid of my web developers.
Because you know I only needed them to know about the framework. I can now delegate that stuff to AI.
I played around with Bolt for weeks. Got it’s subscription and was amazed at what it did. Now point to be noted. Since I’m noob in web this was amazing for me. Because I couldn’t do that. This would be different for a seasoned vet. For example, I think GenAI for Game is still dogshit. It probably is amazing for a noob.
Anyway, back on the story. And then one day on youtube I see this video where they are saying this lovable.dev is great. So, I think lets give it a shot. And I soon realized wow this guys know what they are doing. Code is being committed to GitHub, and they have this supabase integration that helps me write backend without doing any mental gymnastics. Now reminder that the MCP servers were not a thing back then. So, they probably had this in their system when it wasn’t even a thing. But GitHub 2-way sync meant a lot for me. It means I can create whatever garbage I want then if needed I can get one of my web devs (haven’t fired yet. Maybe will need them still) to fix whatever I don’t yet understand. But I do plan to learn this quickly so I can make sure AI isn’t feeding me garbage.
Since JavaScript is wildly different than C# it’s not really easy for me to wrap my head around it in such a short time.
Anyway. To run the Learnathon event smoothly I still needed to develop a few more platforms to automate some of the boring tasks like. I wanted sonar cloud to analyze the participants codebase and create a leaderboard out of it. And Another platform where the participants can finally showcase their work. And mentors can provide judging feedback there. Wasn’t going to maintain excel or google forms for all that.
This is when I decided to give Lovable a shot. Bruh this one made my life even more easy. Specially the way it integrated with supabase. I literally didn’t have to think about the backend. [Well of course as I understood more my involvement became more precise. Guess what AI does feed you garbage when you aren’t looking.]
Then one day lovable had some outage and they offered a weekend of free credits. Holy moly. I spent 2 days to try the thing out at my hearts content. I wasn’t interested in bulding a business out of it. I only wanted to learn how well I can use this tool. And for two days I had full free access. And I did learn a lot. I realized what works and what not. That being said Lovable matured quite a lot since then, so some knowledge is no longer useful. However, I was now confident that lovable can be used to develop products.
And after some time, I did manage to develop an internal tool called Open Pulse Survey.
This is now being used at Brain Station 23 To run Pulse Survey. Now if you are thinking what type of hot pile of garbage this might be, the entire codebase is open sourced you can check it yourself
While I personally am not confident in what I know about web technology. I did audit this project using a human to evaluate the quality of the project. The overall report was rather satisfactory. It mentioned that this AI generated code wasn’t entirely garbage. It could be better but to me. Anything that’s not flagged as a garbage by another human gives it a pass. And of course I had Sonar Cloud and other tools to monitor the quality of the code and stuff like that. If you didn’t have time to take a look at what I built watch this video.
Aside the reporting section this has full-fledged admin panel and configuration. You can take a look at this docs site to get a better picture.
I strongly recommend taking a look at what I was able to build using AI. Because The goal of it is
- Prove my experience in using these tools
- Make a comparison of existing tools that can be used and when to use.
I’ve personally built working 3–5 projects that’s being used internally using vibe coding. And a few static sites. You know what I think I’m going to rebuild our company website using bolt. (I have some tokens left there)
Anyway, lets now dive into the actual topic that I wanted to share.
So With all the vibe coding tools I’ve tried out I realized there are 2 Major Categories of tools that can be used.
- Web container Based Tools: Lovable, Bolt, Replit, Chef. (There are more I guess)
- IDE based tools: Cursor, Tabnine , Windsurf , Junie etc. etc.
Now before we dive any further. Lets just be clear on a few facts. Most of these tools use some base models. So in a way all of these are kind of a glorified wrapper (not really) of GPT , Claude , Gemini models.
However. If you are talking about generating code Claude is the King. Period. Nothing comes close to Claude when generating code.
Gemini is the cheapest model which is probably good enough for 60% of the business cases we have today. They aimed for mass people this AI is cheap and good for mass people.
GPT is Jack of all treads but master of none. So ideally if you are planning your business around them Claude is probably the best choice for quality and Gemini is for financial stability.
Now I’m going to review all the tools as harsh as possible. So don’t get upset if you are part of the company that I thrash.
Web Container based Tools
I’m going to start with Chef (https://chef.convex.dev/)
Now I didn’t know about this at all until my brother introduced it to me. I thought let's give it a shot. It uses other AI models for generating code. Which is something almost all the tools does so that’s that. But what turned me off totally was it was based on convex database. It’s based on no-sql and closed source. why would we ever want to build an AI based application where I get vendor locked? wtf?
So Chef just outright lost the race. Well I guess the wholething is backed by convex. But as a person who sells software as a service a vendor lock in would be the worst-case scenario.
Next, we have Replit (https://replit.com/)
Now they go even one more step further to lock you in. while they do let you download the code their whole service is based off having you build and deploy on their servers. (Although replit is actually quite cool. It does awesome things. Except it tries to lock you in otherwise it can-do things that most other tools can’t. The whole UX is awesome)
So again, since they try to lock you in. It’s a failing mark for me.
And then we have Bolt.new(http://bolt.new/)
This is what introduced me to the whole vibe coding. Bolt uses Claude Behind the scenes. No model switching bullshit. Obviously, I mean since it’s about generating code why bother switching across other models that just simply isn’t good enough.
Bolt is pretty chill. It doesn’t try to lock you in even though they are basically backed by a company that’s about code editor. StackBlitz. You guys are awesome. Now it has integration with Supabase and Github which makes it even more awesome. And to be honest bolt did the supabase integration right. I think nobody did this as well as they did.
Ah Just so you know Supabase a open source alternative to Fire Base. It’s basically firebase but Open Source & with Posgres Database. So yeah for AI related work having a relational database makes more sense than no-sql.
Anyway, Supa base is awesome and right tool for AI based backend.
BUT!
This is at the moment as good as a toy. I built this site with Bolt.new in the hackathon.
https://melodic-boba-2c907a.netlify.app/ (Might delete this link at some point. so don’t be mad if link doesn’t work)
So basically, what I wanted to build is a MyAnimeList for Food. items. Bolt worked pretty well until a point and it outright told me the project grew too big. And refused to develop any further.
BRUH!
This is the reason I kept sharing what I built internally using lovable. That is a codebase of 5–10x of what I built so far using bolt. So, whatever the case bolt has a choking point if you are trying to make a scalable application.
And their token consumption increases as the project grows which is pretty sad as well.
So it’s still a no go for serious development tool. That being said. Bolt is surely on the right track. They just need to figure out a few things. Specially they did welcome this bolt.diy project which means even bolt is free and you can even self host the whole thing including whatever AI model you want (if you can afford it however lmao).
But I think eventually Bolt is on the right way. But right now, it’s more like a toy. That can be used to maybe do some prototyping or develop static sites. yeah, it should be good enough for that. And it does support a lot of tech stacks so you can even make presentations using bolt.
And now we have Lovable (https://lovable.dev/)
From my experience, they are in the Apex of this whole bubble. They got most of the things right. Especially the whole scalability thing. And they don’t measure in tokens instead they charge you based on the times their ai writes code. Yes it only charges when it writes code that makes sense. So if AI breaks something, it’s on them. And bo,y AI does break things often. And they have the GitHub integration and Supabase integration so the basics are covered. The only thing I dislike is that they don’t follow the .env file to store the supabase config, which makes the whole deployment a bit weird. (Not that it’s insecure, it just means if I wanted to have an Anti-Saas platform, it’d be weird for me. Yeah, my product was in fact anti-SaaS.)
Now there’s a catch. Lovable has really focused on doing one thing, and they want to do that good. They only support React.js and no other framework. While all the platform wants to deliver an AI agent that does everything. They focused on delivering one agent that does one thing well. And I personally think that’s why they are at the production level today.
They didn’t want to be the Jack of All trades and master of None.
Overall, Lovable is the only web container-based platform that makes sense to me to use for developing any platform using React and a backend as supabase.
While writing the article, Lovable rolled in a new update where we can try out other models from OpenAI, Google , Anthropic. So I have some more insight about this tool.
As I mentioned previously, Gemini doesn't make sense for these tools so no point ranting about it. It does what it is supposed to do best.
OpenAI and Anthropic out of the box are more or less okay. But it’s not as powerful as Lovables' paid model. There’s a clear quality difference between the Anthropics model and the Lovables model. Maybe the one they let us use is an older version? Not sure, but so far it’s good for doing small tasks. For bigger tasks, I still used the paid model.
IDE-Based Tools
IDE-based tools are for an entirely different audience. It’s for the devs who like to have more control over the whole process. Basically, it’s something that integrates with your favorite IDE or code Editor. Some even come with their own Editors. But most do offer a plugin. And they connect with external models, or their hosted models, based on their business model. Anyway, let's dive into the tools I tried. But before I go any further, I must say.
All of them were VERY SLOW
Not exactly sure why they were slower than webcontainer based ones. The overall code generation time was rather slow. Not like they didn’t work. Those worked as they should; however, I had to wait a significantly longer time before each task.
Let's start with WindSurf (https://windsurf.com/)
I’ve tried it out very recently. It provided a generous amount of free credit for GPT-4.1. I tried out its quality.
While it does work well as a typewriter, as I mentioned previously, it’s slow. Also, the quality of work delivered is kind of acceptable. Since I’ve worked so long with Anthropic’s models and lovable the expectations are a bit higher. But then again, these tools are probably designed for more precision. Because lovable is more like a canon. And it will often break things if you don’t regulate your prompt. While these IDE-based tools are more on the precision side, where the developer feels more comfortable by setting guidelines and rules that the AI must follow.
So, the bottom line for these tools is how good the generated code is. Their base model is pretty useless. Tried that and mostly generated things that didn’t work at all. But the credit system was very nice. It doesn’t charge you per token instead it has this credit system similar to lovable. They only charge you a credit after a set of generations has been done. And I tried out when they were promoting the GPT 4.1 model which basically charged 25% of the actual amount so I got to use it quite a lot without worrying too much.
My overall personal Experience for this is.
- It works in small contexts. The codebase can be large. But if you can point the tool to the right files it needs to check. Then it works well enough.
- Overall credit model is nice it deducts one credit after executing a series of actual tasks.
- This is great for people who like more control over the code generation and what needs to be accepted in the code base and what needs to be rejected.
Let's be honest though. Most users would just accept all changes.
Okay wait. While The GPT-4.1 wasn’t nearly up to the mark. I started using the DeepSeek R1 model, which is FREE. And it’s looking promising. It’s fast. And delivering good stuff. Almost as good as Claude. But how is this Free?
This is looking promising. But wait.. Why is this free? What’s the catch?
Okay, after playing with it for some time, DeepSeek is like a rogue agent. It does things fast, but it’s unreliable. It doesn’t yet seem fully compatible with windsurf. Seems a bit weird though. But in a way, this looks rather promising. If it becomes reliable, then I might consider using this in the long term.
Next up is Junie (https://www.jetbrains.com/junie/)
Fundamentally, these tools are essentially the same. For now, it seems Junie works with Jetbrains Tools only. At least that’s what I felt like. Anyway did give it a try. It uses all the models in the market. As usual using Anthropic’s model is the way. They didn’t provide too much on free so I tried only as much as they provided on free. And to be honest I’m neither impressed nor disappointed. If you are using a IDE based tool , this is just yet another tool that provides glorified code generation with existing AI models that works as good as every other model.
One important thing about them is. They have OFFLINE MODE. now that’s value proposition. Now if you don’t want to send your code to these 3rd party AI hosted services you would like to use the offline mode. The way this works is very simple. You can have LMStudio or Ollama on your network or pc. You hook it up with the IDE. and there you go it works as good as the model you hosted. There are so many other tools like Continue who does the same thing.
Then we have Cursor , Tab nine , etc. etc.
Honestly they are more or less the same in terms of quality and speed. The only thing differentiate them right now is the ecosystem the are connected to and the pricing. But ain’t going to compare pricing at this point. Too soon to compare cost vs value for this.
Conclusion
The way I see it, we’re definitely in a new era. These AI agents are seriously fast and can handle a lot of the grunt work.
Lovable impressed me. It felt like the most mature out of the bunch, especially if you’re into React and using Supabase.
Bolt.new has some serious potential, especially with its Claude integration, but it feels more like a playground right now if you’re aiming for something big. It’s great for quick prototypes and static sites, though.
As for Replit and Chef, the whole vendor lock-in thing just doesn’t sit right with me. And I doubt anyone in the service industry.
Moving over to the IDE-based tools, they’re a different beast altogether. While they give you more control, they just feel a bit slower in my experience. If I had to pick one, I’d say WindSurf stood out a bit, particularly with its credit system and how it handled GPT-4.1. It’s good for when you want more precision and want to guide the AI more closely.
Junie was alright, nothing too groundbreaking, but the offline mode is a cool feature for those concerned about data privacy.
But overall, all the IDE based tools felt pretty similar to each other, so it really comes down to personal preference and maybe which ecosystem you’re already invested in.
For now, I’d be sticking with lovable if I want to build something serious. Things may change in 6 Months because the speed of AI progress is happening.
