Exploring ChatGPT’s Potential: Addressing Advanced Programming Challenges Beyond Basic Games

Like many programmers in recent months, I was intrigued by the new version of ChatGPT 4 and its capabilities in the field of programming. Articles on the internet and initial prompts involving simple script converters or database queries showed very optimistic results, but did not provide a definitive answer to the question of whether ChatGPT can produce production-ready code and replace part of the programmer workforce. To do this, it was necessary to find a real challenge that would allow testing GPT under less controlled conditions, with a focus on its ability to work with substantial code.

Unfortunately, in my opinion, it is currently not feasible to use ChatGPT at work, as I understand that this carries a serious risk of data leakage to third parties, which contradicts the security policies of many companies. Therefore, I decided that, for now, the most appropriate use of ChatGPT would be for open source projects.

So, here are the conditions of our task:
I have a small custom animated button project for iOS, written by me 8 years ago in Objective-C. It’s a relatively simple code that describes the animations of individual layers in great detail. The code consists of 500 lines in total.

- (NSArray *)fromMenuToCrossAnimationValues:(CALayer *)line
                                    reverse:(BOOL)reverse
{
    NSMutableArray *values = [NSMutableArray array];
    CGFloat lineMult = line == _topLayer ? 1.0 : -1.0;
    CGFloat yTransform = _middleLayer.position.y-line.position.y;
    
    CATransform3D transform = CATransform3DIdentity;
    [values addObject:[NSValue valueWithCATransform3D:transform]];
    transform = CATransform3DTranslate(transform, 0, yTransform, 0.0);
    [values addObject:[NSValue valueWithCATransform3D:transform]];
    
    transform = CATransform3DRotate(transform, M_PI_4*lineMult, 0.0, 0.0, 1.0);
    [values addObject:[NSValue valueWithCATransform3D:transform]];
    if (reverse) {
        values = [[[values reverseObjectEnumerator] allObjects] mutableCopy];
    }
    return values;
}

In addition to this, I haven’t been writing for iOS for more than 5 years, but I’m not a complete newcomer to programming either.
The goal of our experiment will be to completely rewrite the code in Swift, support SwiftUI and SwiftPackage (I am not familiar with these things). Well, let’s get started.

Day 1

To start, I asked if ChatGPT could convert Objective-C to Swift.

Me:
Do you know objective c language? Can you convert some code on objective c to swift?
Answer:
Yes, I’m familiar with Objective-C, a programming language often used for iOS and macOS development, as well as Swift, Apple’s modern programming language for the same platforms. While I cannot directly overwrite code as an AI, I can help you understand how to convert Objective-C code into Swift code.

Next, I needed to figure out how to send code to ChatGPT since even my small library doesn’t fit within the prompt limit (2048 characters). So, I decided to try this approach:

Me:
I will send you objective c code by parts. Please send me back an example of rewriting it on swift when i tell you — i finished.
Answer:
Sure, please go ahead and send the Objective-C code in parts. Once you tell me you’ve finished, I will help you convert it into Swift code.
Me:
Chunk of code
Answer:
Chunk of code

ChatGPT started translating the code right away, as I sent it, which led to errors in the new code due to the lack of complete context. As a result, I decided to send the code in minimal functional parts, like one method or a structure declaration. It wasn’t as convenient, but it was better than nothing.

After testing the basic hypothesis, I could start creating a new project from scratch.

Me:
How to create new swift library project of ios ui component with Package.swift and example application for tests?
ChatGPT
Creating a new Swift library project for an iOS UI component with a Package.swift file and an example application for testing involves a few more steps compared to a regular Swift package. Follow these steps:
…

The suggested algorithm contained the following errors:

It asked for unnecessary steps. This shows that ChatGPT handles simple tasks well, but when it comes to more complex configurations, it gets lost.
It couldn’t solve the problem of connecting a local package in Xcode, although the solution can be found in a 30-second Google search.
The generated code for the component library did not contain a public signature for the button class, causing the test screen code not to compile.
After overcoming all the listed issues, I proceeded to generate a demonstration screen for the library’s capabilities using SwiftUI. Despite not having worked with this technology before, I achieved a working result in just a few minutes.

In total, the whole process took about two hours.

Day 2

On the second day, I (crossed out ChatGPT) began the actual code conversion. It took me about 20 requests to send the entire library code to ChatGPT. This task was quite tedious and time-consuming, requiring a certain level of attention to ensure nothing was missed.

Unfortunately, the resulting code contained errors and bugs and required human intervention for refinement. For example, ChatGPT would forget to add the class name to static public variables it had previously created.

It made silly mistakes:

[self.layer addSublayer:layer];

was translated into:

layer.addSublayer(layer)

This code leads to a runtime crash due to an infinite loop when adding a layer to itself.

Due to the large number of requests, the 25-message limit was quickly exhausted, and I decided to try fixing some errors using ChatGPT 3.5. However, after working with the older version, it seemed utterly useless for programming tasks.

In total, I spent an hour on this day, most of which was spent copying code and sending it to ChatGPT.

Day 3

I managed to complete the code conversion and compile it. Most of the time, I was trying to implement SwiftUI support for my component, and the hints and generated code from ChatGPT were very helpful in this regard. However, since I was not familiar with this technology, it was challenging to compose the right prompts to achieve the desired result.

As a result, after an hour, I saw the button icons on the screen, but they didn’t function as intended due to bugs in the code.

Last day

On the last day, I dedicated my time to debugging the issues. It wasn’t too difficult and merely involved double-checking the generated code. Here are a few examples of errors:

Original:

[_topLayer addAnimation:topAnimation forKey:tbAnimationKey];
[_bottomLayer addAnimation:bottomAnimation forKey:tbAnimationKey];

Converted code:

topLayer.add(bottomAnimation, forKey: TBAnimatedButton.tbAnimationKey)
bottomLayer.add(bottomAnimation, forKey: TBAnimatedButton.tbAnimationKey)

Original:

times = @[@(0.0),@(0.4), @(0.4), @(1.0)];

Converted code:

let middleTimes = reverse ? [1.0, 0.6, 0.6, 0.0] : [0.0, 0.4, 0.4, 1.0]

After two hours, all bugs were fixed, and I obtained an equivalent of my old library entirely rewritten in Swift.

Conclusion

In total, this project took me six hours of my time. I don’t think I could have completed it faster on my own, as I would have needed to spend just as much time recalling how to write in Swift. However, I can’t say that the process was enjoyable, as it was quite mechanical and dull while still requiring careful checking.

ChatGPT 4 can indeed write code, but using it to solve real-world problems can be challenging due to its limited context and not always user-friendly interface.