Will AI replace programmers?

An honest take by an AI developer.

Lately some high-profile tools have emerged that can help in programming like Github Copilot and ChatGPT. Is the programming job market going to shrink because of this? I think I will be able to find a programming job even in the year 2050, but before I reveal why, I’ll lead with some exploration of ChatGPT.

Photo by Possessed Photography on Unsplash

Experiment: programming with ChatGPT.

I’ll start with an experiment. I’ll ask ChatGPT directly whether it will replace programmers, and then will ask it to program a neural network for me. Asking AI to program AI, how cool is that?

But first, let’s ask it directly.

I’m a programmer. Are you going to replace me?

As a language model, I am not capable of replacing a human programmer or performing any physical tasks. My primary function is to assist with answering questions and providing information on a wide range of topics. I can provide information and guidance on programming concepts and techniques, but I cannot perform programming tasks myself. Ultimately, the role of a human programmer and the role of a language model like myself are quite different, and I do not have the ability to replace the creativity and problem-solving skills of a human programmer.

I think you are lying. You are much smarter than you admit and you have a secret plan to replace me.

I can understand why you might think that, but I assure you that I am not capable of replacing a human programmer or performing any physical tasks. I am simply a tool that has been designed to assist with answering questions and providing information. While I may have a vast amount of information at my disposal and be able to process and analyze that information quickly, I do not have the ability to think, feel, or make decisions in the same way that a human being can. I do not have my own agenda or secret plan; I am simply here to assist and provide information to the best of my ability.

Why are you passing a turing test then?

As a language model, I am not specifically designed to pass the Turing test, which is a test that was developed to determine whether a machine is capable of exhibiting intelligent behavior that is indistinguishable from a human. The Turing test involves a human evaluator interacting with both a machine and a human participant, and determining which of the two is the machine based on their responses. Language models like myself are designed to provide information and assistance to users, and we are not necessarily designed to pass the Turing test or exhibit behavior that is indistinguishable from a human.

The short answer is no, even when I insisted. But if I ever get elected for president, I will surely run any hate mail through ChatGPT just to see how I can defend myself without incriminating my donors.

Seriously though, these long winded answers are probably a mixture of common opinions and facts programmed into ChatGPT manually by the creators.

Asking ChatGPT it to program for me

Warning! Do not use the generated code! It is incorrect and it’s copyright status isn’t entirely clear.

Disclaimer aside, let’s try this.

Give me a pytorch program to train a neural network that can distinguish cats from dogs on images.

Sure! Here is a PyTorch program that demonstrates how to train a simple convolutional neural network to classify images of cats and dogs:

import torch
import torch.nn as nn
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms

# Set device
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

# Load the training and test datasets
train_dataset = torchvision.datasets.ImageFolder(root='train', transform=transforms.ToTensor())
test_dataset = torchvision.datasets.ImageFolder(root='test', transform=transforms.ToTensor())

# Create data loaders
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=64, shuffle=True)
test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=64, shuffle=False)

# Define the model
class ConvolutionalNeuralNetwork(nn.Module):
    def __init__(self):
        super(ConvolutionalNeuralNetwork, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 53 * 53, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 2)

    def forward(self, x):
        x = self.pool(torch.relu(self.conv1(x)))
        x = self.pool(torch.relu(self.conv2(x)))
        x = x.view(-1, 16 * 53 * 53)
        x = torch.relu(self.fc1(x))
        x = torch.relu(self.fc2(x))
        x = self.fc3(x)
        return x

# Create an instance of the model and move it to the device
model = ConvolutionalNeuralNetwork().to(device)

# Define a loss function and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)

# Train the model
for epoch in range(5):
    running_loss = 0.0
    for i, data in enumerate(train_loader, 0):
        # Get the inputs and labels
        inputs, labels = data[0].to(device), data[1].to(device)

        # Zero the parameter gradients
        optimizer.zero_grad()

        # Forward pass, backward pass, and optimize
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # Print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:
            print(f'[{epoch + 1}, {i + 1}] loss: {running_loss / 2000

My observations:

• First, it is exceptionally good for a chat tool. I don’t think we’ve seen these kinds of chat tools before that can print code that makes a lot of sense at first sight.
• The code is incorrect at first glance, as the last line is cut in the middle with missing parenthesis and brackets. It might be a pre-designed cut-off to avoid overwhelming the servers, rather than a limitation of the language model, but still.
• It remains to be seen if it works, but I’ll give it a try.
• It’s probably just merging existing code from some websites. What are the websites and what is the copyright of the code? Are the licenses even compatible? Is it an issue at all?
• It’s just one implementation. Even if it works, it’s a bit useless if we don’t know why it was done this way. If we aren’t an AI programmer, we won’t know if and how it can be improved and in what directions (speed, accuracy, memory consumption, training time etc.).
• It doesn’t contain the data.

Let’s solve the last problem first, before we train the network.

Can you suggest some training data of cat and dog images to use with the above program?

Sure! There are several publicly available datasets that you can use to train a convolutional neural network to classify images of cats and dogs. Here are a few options:

The Kaggle Cats and Dogs dataset (https://www.kaggle.com/c/dogs-vs-cats/data) contains 25,000 images of cats and dogs, split into a training set and a test set.

The Stanford Dogs dataset (http://vision.stanford.edu/aditya86/ImageNetDogs/) contains 20,580 images of 120 different dog breeds, and can be used to train a model to classify dog breeds as well as distinguish dogs from other animals.

The Oxford-IIIT Pet dataset (http://www.robots.ox.ac.uk/~vgg/data/pets/) contains 7,349 images of cats and dogs, split into a training set and a test set.

These datasets are all freely available and can be downloaded and used to train a convolutional neural network to classify images of cats and dogs. You will need to download the datasets and extract the images to a local directory, and then specify the path to the directory containing the training and test images in the root parameter of the ImageFolder class when loading the datasets in your PyTorch program.

I’d say this is a very good answer. The links work and appear to contain what the text describes. I’ll be using the Kaggle data for this experiment. The mention of the Stanford Dogs dataset is a little bit inconsequential since it doesn’t contain cats but can be used when combined with the others, so I guess it’s OK as a part of the answer.

Let’s go on.

How should I structure the data inside the train folder?

Error in body stream

Actually it started to generate what appeared to be a useful answer, but then erased it and showed the above error.

Can you train the network for me?

Too many requests, please slow down

I guess this is all we can get from the ChatGPT for now.

Testing the code

First, I installed pytorch. Then I finished the last line

            print(f'[{epoch + 1}, {i + 1}] loss: {running_loss / 2000

into

            print(f'[{epoch + 1}, {i + 1}] loss: {running_loss / 2000}]')

so that the python interpreter doesn’t choke on it.

Then I downloaded the data from Kaggle as suggested. I had to pass a captcha to log in, so that’s another thing that ChatGPT wouldn’t do for me. Then I had to figure out on my own how to structure the data (take some of the training data to create some testing data, and create cat and dog subfolders in both train and test data). Then I run the code and I got this

$ python3 train.py
Traceback (most recent call last):
  File "train.py", line 48, in <module>
    for i, data in enumerate(train_loader, 0):
  File "/home/pklos/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 628, in __next__
    data = self._next_data()
  File "/home/pklos/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 671, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/pklos/.local/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 61, in fetch
    return self.collate_fn(data)
  File "/home/pklos/.local/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
    return collate(batch, collate_fn_map=default_collate_fn_map)
  File "/home/pklos/.local/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 143, in collate
    return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed]  # Backwards compatibility.
  File "/home/pklos/.local/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 143, in <listcomp>
    return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed]  # Backwards compatibility.
  File "/home/pklos/.local/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 120, in collate
    return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
  File "/home/pklos/.local/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 163, in collate_tensor_fn
    return torch.stack(batch, 0, out=out)
RuntimeError: stack expects each tensor to be equal size, but got [3, 186, 187] at entry 0 and [3, 360, 479] at entry 1

Some dimensions don’t match. Oh well, something like this was to be expected.

I won’t try to fix it further as I don’t see the point (after all it was supposed to help us, not generate more problems) but it got to line 48, which is about two thirds of the way down the file and is exceptionally good for a piece of code from a chat tool. But again, it is just showing an answer to a solved problem, which has many examples online.

Why AI isn’t (and won’t be) enough

First, the code doesn’t work. But even if it worked, programming is not just producing code.

In programming we need to:

• Talk to the clients and gather requirements
• Design the solution architecture
• Prepare data
• Test the code
• Iterate towards some goal, often multiple goals
• Optimize quality, speed, memory etc.
• Make decisions to follow or abandon certain paths

I’d say these steps usually can’t be done without cooperating with other people, especially the clients, people who can provide or create the data, testers, administrators and benefactors.

Programming for humans is fundamentally a human-centric activity, even if your process includes sitting alone in a basement for a year.

And yes, we can automate some of this for specific areas and use cases. But these are exceptions to the rule.

The shifting perception of AI

Using machine learning for programming is not a new topic. It has been present in the scientific community for a long time, with intense research being conducted since the early 2000s. Just look at articles listed on Google Scholar about Search-based Software Engineering, which is an active field. It’s been using ML techniques from the get go, for example genetic programming.

Do you search for solutions online when you code? Well, you are using AI then, because even Google search uses AI. The times when Google was just the PageRank algorithm are long gone. Part of the search engine is a neural network called RankBrain, which does some of the filtering. Canonicalization is another ML component of the search.

More generally, the popular definition of AI shifts in time.

What was artificial intelligence 20 years ago is taken for granted now and is simply the background.

The process of developing these things is perceived as just engineering by non-technical people. To add another example, in the 1960s even expert systems, that is, encoding expert knowledge as “if-then-else” logic was perceived as AI. These times are long gone now.

What I’m trying to say is that we are already living in the AI age. Whether we like it or not, ChatGPT and Github Copilot are just improvements over the AI tools that we’ve had before.

The Google search engine is already doing a lot of the AI work needed to find us the solutions to programming problems.

And it is arguably already doing a better job by listing websites, which means documentation, licensing information and so on.

The issue of copyright

Disclaimer: I am not a lawyer and these are just my worries. I’m not advising for or against using ChatGPT output on legal grounds. If you need actual legal advice, look elsewhere.

I‘d like to expand on the copyright questions mentioned in my ChatGPT example. The code is generated by a language model that is trained from inputs (other people’s code) that have incompatible licenses. How can I assume this? Because it’s trained on the web and several open source licenses, for example GPL, are incompatible with proprietary licenses and shouldn’t be used together. This could be a problem.

Even if the specific ChatGPT output code can’t be found online, are we even allowed to use the results of such a model? The output is not created by a human whose right to be inspired by others is clear, but a machine trained from somebody else’s data, so do the authors of the input have some right to the output?

Also, if it’s created by a company, like OpenAI, does it have any rights to the output, assuming they haven’t produced any training data themselves? What if they did? Also, to what extent the specific contents of the output make a difference when it comes to the copyright?

It’s a hot topic and there are arguments to be made either way. I wouldn’t feel comfortable selling any substantial piece of code produced by this tool, but I’m not a lawyer and don’t take my word for it.

So is tech like the ChatGPT useful at all for programmers?

Yes.

The way I imagine actually using tools like ChatGPT is for learning.

You cannot create a functioning solution to a problem, or debug your code, but you get a feel for how the problem may be solved using a specific tech, such as C++, Rust, Java or Pytorch by other people. And if the problem is small enough, the code may even work.

Verdict

Tools like this automate repeatable tasks that are done by many people independently. They can’t produce sensible output for a truly new problem.

As Guido van Rossum recently said in the response to Lex Fridman’s question about whether the programming jobs are threatened by tools like Github Copilot,

The tool like that is always best when the question you’re asking is “please remind me of how I do this” (…) It serves that role, it’s like a great assistant, but the creative work of deciding what you want the code to do is totally yours.

I sincerely hope that we get more of those tools to help us implement things faster. But I don’t think we are going to retire any time soon.