(Not an anti-AI post — Copilot is great, I use it all the time.)
Early this year I built some cryptocurrency trading and data gathering tools for personal use and to get more experience in Rust programming. While asking questions in various groupchats, it didn’t take too long to see that lots of other people were demanding similar tools — and would be willing to pay for them. Pretty soon after that realization, I had some API endpoints set up where people could access the data for free and submit trades for a small commission.
I started getting a few customers, a very cool experience since this was the first time people were paying for software I built myself! I started a Telegram channel for feature announcements and support, which worked well at first. But, as my customer base slowly grew, support began taking up more and more of my time. I know this is the case for any SAAS startup, so the increasing support burden was hardly a surprise, and, after all, more customers is a good problem to have! What became irritating was not the quantity, but the quality of the support requests I was receiving.
My API is just a few well documented endpoints. If you can figure out how to send a POST request using any programming language, you should have no problem using it. But that seems to be too high a bar for the new generation of prompt-engineer coders. Since opening my support channel, I have fielded many a “Help! My trading bot is not working!!” support request. More often than not, I will be sent customer code that is mostly fine, but has some error that should be glaringly obvious to anyone who has read the documentation and has some programming ability. Often this takes the form of trying to access an endpoint that does not exist, or read a property off the API response that does not exist. After probing a bit more, my suspicions are usually confirmed — ChatGPT hallucinated that endpoint or property, and the customer I’m talking to has little to no programming knowledge. If they are just trying to build a simple script I’ll help them out, and fix the hallucinations — it’s not much effort and creates a potentially paying customer. Often, though, the customer is envisioning a more complex application, and I just have to tell them, “Sorry, you’re going to have to hire a professional developer for this.” The worst is when a request starts out simple — I help them fix one hallucination — but then that customer wants to build more complex logic, and somehow I’ve set the expectation that I will provide unlimited free support forever. I’ve gotten a number of angry messages from customers who essentially want me to build their whole app for free.
I’m sure these challenges sound familiar to anyone who has run support for a SAAS business, but AI programming tools have exacerbated the problem. Helping a customer solve challenges is often super rewarding, but only when I can remove roadblocks for customers who can do most of the work themselves. When customers offload software engineering to AI because they don’t have the capability themselves, they still need to find a developer to fix the bugs that AI creates. I don’t want to be that developer!
