ChatGPT As A DevOps Assistant— Day 28 of #30daysofAI

8 min readAug 30, 2023

Today’s exercise walks through how to use ChatGPT as a DevOps assistant to build, deploy, and monitor a service in Google Cloud. As I discovered today, no previous experience is needed with containers or Google Cloud services for deployment/orchestration/monitoring when using ChatGPT in this fashion. Continue reading below for the step by step I followed to get this done in less than an hour.

AI Deploying to the Borg, by Stable Diffusion

For those missing context, this is related to a blog series called 30 Days of AI, where I challenged myself to spend the next 30 days learning more about AI.

Background

Earlier this week I used ChatGPT to help me quickly create an interactive Discord chatbot wired up to OpenAI APIs. Today I decided to turn this into a service that will run in the cloud 24/7. I decided to try out Google Cloud as my cloud hosting solution — since they give $300 in free credits to new users.

Prerequisites

Before getting started with trying to deploy this service, here are a few things that should be done first:

Make sure your service can run locally. Or for those wanting to follow along with my example, follow along from my previous post that created the original CatBot python bot
Sign up for a free Google Cloud account
Install Docker (we’ll use this in a later step)
For those working in Windows (like myself), make sure you have WSL2 installed and working (aka Linux/Ubuntu on Windows)
Install the Google Cloud CLI in your development environment
After installing the Google Cloud CLI, make sure to login to your account using this command:

./google-cloud-sdk/bin/gcloud auth login

Deploying The Bot

First Instincts

My first instinct in figuring out how to do deploy the bot in Google Cloud was to ask ChatGPT everything. I asked this question in the same conversation that I previously used to create CatBot in order to give more context. Here’s the prompt I tried:

I want to deploy this bot as a service in google cloud. Tell me the steps on how to do that

ChatGPT gave some useful information, but suggested the path of running the bot directly on a VM in Google Compute Engine.

Deploying With Containers Instead

Instead of using a raw VM on Google Compute Engine to deploy the bot I’d prefer to use a managed service. This is because managing raw VMs is a bit more difficult and requires more hands-on DevOps which can be time consuming. Another path for deploying services typically is to create a virtual container for the application (typically using Docker), and then deploy that container into a managed service in the cloud. I followed up with ChatGPT to ask how this can be accomplished with Google Cloud:

How can I deploy to a managed service instead of a raw vm in google cloud?

ChatGPT gave step by step instructions on how to do this, starting from creating the container up until running the container in Cloud Run (Google Cloud’s managed platform for running container applications).

Creating My Dockerfile

From the ChatGPT steps above, I first created a Dockerfile in my python project directory:

# Use an official Python runtime as the parent image
FROM python:3.9-slim

# Set the working directory
WORKDIR /app

# Copy the current directory contents into the container
COPY . /app

# Install required packages
RUN pip install -r requirements.txt

# Run your bot script when the container starts
CMD ["python3", "catbot.py"]

After this was done I was able to use Docker in WSL to build an image for my application:

docker build -t discord_bot .

Note: I actually ran into some issues with Docker not being recognized in WSL. If you see the error ”The command ‘docker’ could not be found in this WSL 2 distro.”, then follow the mitigation steps under the Docker docs followed by restarting WSL and Docker. After doing this I was able to use docker from commandline in WSL

Create A Google Cloud Project

Google Cloud has a concept called projects which allows easy management of permissions and services into a single logical project. For this exercise I created a new project in my account called “CatBot” in the Google Cloud Console:

Push The Container To Google Cloud

As part of the previous instructions from ChatGPT I was given instructions on how to get the container into Google Cloud:

Unfortunately when I tried to do this I received a few errors from Docker. I was able to get help from ChatGPT to unblock using the following prompts:

I got this error when trying to do docker push:
An image does not exist locally with the tag: gcr.io/catbot/discord_bot

And following again:

I got another error trying to do docker push. How do I fix this?
unknown: Service ‘containerregistry.googleapis.com’ is not enabled for consumer ‘project:catbot-123’.

Tagging Container Image And Pushing

As per the above ChatGPT fix responses, I followed the below steps to get the container deployed:

docker tag discord_bot gcr.io/catbot-<yourid>/discord_bot
docker push gcr.io/catbot-<yourid>/discord_bot

Note: the id from your Google Cloud project should replace the <yourid> tag above if following along. This is a unique id created for each project in Google Cloud, so if you don’t know this check out your projects in the console:

If you end up seeing this error from docker (unknown: Service ‘containerregistry.googleapis.com’ is not enabled for consumer ‘project:catbot-1234’.), make sure you have enabled the Container Registry service for your project.

Deploying The Container

Earlier in the prompt flow ChatGPT already gave some instructions on how to deploy the container to Google Cloud Run (Google’s platform for running managed container applications). Here’s a quick recap of the response and steps:

Visit Google Cloud Run in the Google Cloud web console
Create service and choose the container image that we just pushed
Pay attention to the regions and pricing tiers. I had to look this up, but Tier 1 pricing is cheaper
Choose settings related to your service requirements. Since CatBot is a Discord bot that doesn’t take traffic I made sure to set up Cloud Run to run a single instance with internal traffic only:

Monitoring

Once the Cloud Run service is created it may take a few minutes to spin up. After that you should be able to see the bot start up and connect to Discord. Cloud Run has a streaming logs feature where you will also be able to see console logs spit out in close to real time:

Better Monitoring

I asked ChatGPT how I could improve monitoring of this service:

How do I add monitoring to my cloud run service to make sure my discord bot python stays running?

The response for this was pretty generic, and it assumed that my service would be taking traffic:

I asked ChatGPT with a more specific prompt to get better results:

Since my discord bot is a python script that doesn’t take traffic, how do I monitor that it’s running in cloud run?

ChatGPT gave some better ideas that mainly consisted of checking the logs (which I described above already), adding a heartbeat into the code, and doing some automatic error handling with Cloud Error Reporting

Adding Monitoring Metrics

For most remotely deployed services, emitting metrics directly from the service code or platform is a great way to improve monitoring. ChatGPT suggested to add something like a heartbeat into the chatbot, so I asked for some more details on how to do this:

How do I emit a metric from my discord bot python to google cloud monitoring and set an alert on it?

ChatGPT gave some useful info on how to get started with Google Cloud Monitoring, which included a Python code snippet that emits a sample metric.

Final Thoughts

I was able to start from zero context of how to deploy and run my chatbot as a service in the cloud and accomplish my mission. Asking ChatGPT targeted questions allowed me to work through setting up a build process for my service into a container, creating a service in Google Cloud Run, and doing some basic monitoring of the service health.

As with most topics there were some intermittent hiccups with some of ChatGPT’s responses. However, continuing on with follow-up prompts allowed me to work through all the problems I ran into and get my chatbot deployed successfully.

I would certainly recommend use of ChatGPT for simple DevOps scenarios like deploying and monitoring a simple service like my chatbot. I encourage developers that have little experience with cloud services (or Google Cloud) to give this a shot since all of the tools I used today are free to use (with Google Cloud free for up to $300 of usage).

<< EOM >>

Have feedback or questions about using ChatGPT as a DevOps assistant? Drop me a line in the comments and let me know what you think.