Efficiency vs Mediocrity: The Double-Edged Sword of Foundation Models

TL;DR: AI tools like ChatGPT can boost developer productivity but risk creating an illusion of competency if misused by those without a solid foundation — be it coding or other areas. The challenge is balancing efficiency with genuine skill development.

Regardless of the team size — be it 5 or 50 — I always recall having at least one junior or mid-level developer who, during code review sessions, wasn’t entirely sure what their code did. Often, I received an honest and appreciated “I got this snippet from StackExchange,” but sometimes, I had to probe further to understand the origins of their code.

Today, it appears we’re rapidly shifting from “I got it from StackExchange” to “I got it from ChatGPT.” While this might seem like a clear upgrade, I firmly believe it’s a double-edged sword — and a particularly dangerous one at that. Here’s why.

The rise of foundational models like GPT 3.5+ in coding presents a critical choice: efficiency or a facade of competency

Firstly, it’s evident that juniors are now “abusing” this tool to such an extent that their progression up the Dunning-Kruger curve towards the peak of “Mount Stupid” is exponentially accelerated. Secondly, I believe this can significantly mislead team leaders — especially in the early stages of team formation — about the true potential of these developers.

Let me illustrate this with an example. Last year, we onboarded a new team member who was initially evaluated as somewhere between a junior and mid-level developer. She was responsible for developing microservices for a third-party app that leveraged our AI-on-edge processing platform. Concurrently, we encountered issues within our DevOps ecosystem for the third-party app and couldn’t quickly rely on our external DevOps team to explore potential Kubernetes solutions for monitoring these issues.

We devised and planned a simple external service to monitor, at a business level, the exact Kubernetes services, pods, and resources that were fluctuating — a kind of quick-and-dirty replacement for Prometheus and Grafana but highly focused on our tasks. I personally evaluated the task at around 40 hours of mid-level Python programming and assigned it to her. She estimated the task would take 1.5 weeks but completed it in just 2 days. The service app was impressive, complete with a UI and documentation. I was both amazed and thrilled.

Then, I began assigning her more “foundational” tasks that were not insular but rather integral parts of an enterprise app. Long story short, within a couple of months, we faced an avalanche of problems: bugs and functional mistakes. During a deep code review — bordering on a “code shaming session” — we discovered the truth: it was ChatGPT all along, even for the initial one-time hit. Although we initiated additional coaching and development mentoring, it took a few more months before we parted ways with her.

I believe coding-focused foundational models have the potential to dramatically change the landscape of software development and perhaps one day democratize it entirely. However, at this point, I also believe that a foundational model used by someone without a solid foundation — particularly in programming, but likely in other areas as well — is dangerous for both the employer and the individual (again, due to the accelerated Dunning-Kruger effect). Mindlessly prompting and using generated code without the necessary skills to understand, review, remodel, and reframe it is more perilous than not knowing how to do the task at all.

Ensuring a balance between foundational knowledge and advanced tools is crucial

Conversely, foundational models in the hands of experts can significantly enhance their productivity, potentially increasing efficiency by at least 50% if used correctly (with different agents for different areas, careful system prompting, etc.).

Ultimately, it all boils down to one question: will we witness the rise of uncontrolled mediocrity or a new age of efficiency?

Conclusion

The rise of foundational models like GPT 3.5+ in coding presents a critical choice: efficiency or a facade of competency. While these tools can significantly enhance productivity for skilled developers, their misuse by those lacking a solid foundation can lead to severe problems. Ensuring a balance between foundational knowledge and advanced tools is crucial. In the right hands, these models can revolutionize development; in the wrong hands, they risk accelerating incompetence.

Ultimately the key lies in leveraging these tools to truly enhance, not just superficially improve, human capability.