Tried to solve todays Daily Challenge problem of Leetcode using ChatGPT (Analysis and comparision)
I had solved the question before trying it on ChatGPT. I just copy-pasted the question. https://leetcode.com/problems/maximum-bags-with-full-capacity-of-rocks/ . This is just an analysis on the above question. I am writing this for the purpose of awareness.
Language Selection
I tried to see how good is it in terms of optimisation aspect of the code it provides. I was surprised by the solution it provided. It gave a python solution without mentioning any language to code. Which means It took decision to give solution in python without asking the user. Not sure on what parameters it selected python. If any ChatGPT developers see this maybe able to explain.
Language Conversion
Now I wanted to try that if I ask the solution in other language like Java, will it convert the given solution or give a completely new solution. Astonishingly it converted the same solution into Java, with same variable names same structure of coding.
You can see the comments are even same. Everything exact except language. It even gave me what was different.
Code
This question can also be solved using min heap. Priority Queues(PQ) in Java helps us implement min heaps. So I tried to ask for a solution using PQ. It gave me this.
Same thing happened, same variable were used, same comments where used.
Analysis
When you solve a question in leetcode you don’t know if your solution is better or the solution provided by the people in solutions tab. As in even if both solutions have same time complexity and space in the above case which will you say is the better solution as a developer. So I asked ChatGPT. This is what I got as a reply.
So it said with PQ it is a better solution as creation takes O(n) for n elements and it takes O(logn) per element for insertion and deletion. As we are targeting to remove only few elements from PQ, until we remove all additional rocks. Which makes PQ approach a better solution even if it takes O(nlogn) time.
But here comes a question, why did it not provide the PQ solution on the first place and provided it after being asked. It took a decision to give solution in Python but couldn’t give the best practical solution.
Analysis (without sorting)
If we ask for a solution for this without sorting, it forgets about the maximum aspect of the question and gives me a greedy solution, which will be correct if difference(capacity[i] — rocks[i]) is already sorted. This is again a wrong assumption made.
This made me sad, as in you can compromise with time/space complexity but you cannot assume a wrong parameter. It didn’t even tell me about the assumption made until I told ChatGPT about it and then it gave me the old solution again.
Conclusion
ChatGPT is a good interactive tool but its solutions are not as practical as you think nor it talks about the assumptions it made while providing the solution to you. So if you are a beginner in software industry you might get quick and easy solutions using ChatGPT but would not know the edge cases you might miss if you use this code in production. They are not bug free.
You need to be an experienced developer to understand the bottlenecks of the solution and assumptions the code is making. This is good for R&D purpose but think before you copy-paste this code in production.
Still there is time to beat humans !!