Schedule PostgreSQL Backups with IBM Cloud Code Engine

Learn how to build a container image from source code and use the image to schedule your PostgreSQL backups using IBM Cloud Code Engine.

Following the instructions in this post, you can schedule the backup to run hourly, daily, weekly, monthly, yearly or even every minute, if you want.

Before jumping into the technical how-to instructions, let’s understand a bit about IBM Cloud Databases for PostgreSQL and IBM Cloud Code Engine with quick introductions below

What is IBM Cloud Databases for PostgreSQL?

IBM Cloud Databases for PostgreSQL is a serverless cloud database service that is fully integrated into the IBM Cloud environment. This offering lets users access and use a cloud database system without purchasing and setting up their own hardware, installing their own database software or managing the database themselves.

IBM Cloud Databases for PostgreSQL requires no software, infrastructure, network or OS administration. IBM continuously provides fully automated updates to the service, such as security patches and minor version upgrades. A database instance is deployed by default as highly available across multiple data centers in an IBM Cloud Multi-Zone region with synchronous replication. Customers need only connect to a single database endpoint and IBM automatically manages the failover between Availability Zones.

IBM Cloud Databases for PostgreSQL provides the ability to horizontally scale the PostgreSQL instance with Read Replicas in region or cross-regionally. IBM Cloud Databases for PostgreSQL Read Replicas can be easily transformed into fully functioning IBM Cloud Databases for PostgreSQL instances, an especially useful feature for online cross-regional disaster recovery strategies.

What is IBM Cloud Code Engine?

IBM Cloud Code Engine is a fully managed, serverless platform that runs your containerized workloads, including web apps, microservices, event-driven functions or batch jobs. Code Engine even builds container images for you from your source code. Because these workloads are all hosted within the same Kubernetes infrastructure, all of them can seamlessly work together. The Code Engine experience is designed so that you can focus on writing code and not on the infrastructure that is needed to host it:

Architecture Diagram

Before you begin

• Create an IBM Cloud API key.
• Create an IBM Cloud Code Engine project.
• Create an IBM Cloud Databases for PostgreSQL service instance. Copy and Save the CRN (deployment ID) from the service Overview page for quick reference.
• Create an IBM Cloud Container Registry namespace.

Create a job from source code

You can create your job from source code. Find out what advantages are available when you build your image with Code Engine.

A job runs one or more instances of your executable code. Unlike applications, which include an HTTP Server to handle incoming requests, jobs are designed to run one time and exit.

IBM Cloud Code Engine can automatically push images to Container Registry namespaces in your account and even create a namespace for you. To push images to a different Container Registry account or to a private Docker Hub account, see Accessing container registries.

1. Open the Code Engine console.
2. Under Start from source code, enter https://github.com/VidyasagarMSC/automate-postgresql-backup as the source URL and click Start creating.
3. Select Job.
4. Enter a name for the job or leave the default. Use a name for your job that is unique within the project.
5. Select the project you created from the list of available projects. You can also create a new one. Note that you must have a selected project to create a job.
6. Select Source code under Choose the code to run.
7. Click Specify build details:

• Check the source repository https://github.com/VidyasagarMSC/automate-postgresql-backup. Click Next.
• Select Dockerfile as the strategy for your build and resources for your build. Click Next. For more information about build options, see Planning your build.
• Provide registry information about where to store the image of your build output. Select a container registry location, such as IBM Registry Dallas.
• Select an existing Registry access secret or create a new one. If you are building your image to a Container Registry instance that is in your account, you can select Code Engine-managed secret and let Code Engine create and manage the secret for you.
• Select a namespace, name and a tag for your image. Click Done.

8. Add environment variables by clicking Add under the Environment Variables (optional) section. Select Literal value:

• Environment variable name: IBM_CLOUD_API_KEY. Provide the API key under Value and click Done
• Repeat the steps and add POSTGRES_DEPLOYMENT_ID. Use the CRN (deployment ID) from the PostgreSQL overview page. Click Done

9. Click Create.

Create an event subscription

In distributed environments, you’ll often want your applications or jobs to react to messages (events) that are generated from other components, which are usually called event producers. With Code Engine, your applications or jobs can receive events of interest by subscribing to event producers. Event information is received as POST HTTP requests for applications and as environment variables for jobs.

The cron event producer is based on cron and generates an event at regular intervals. You will use a cron event producer when an action needs to be taken at well-defined intervals or at specific times.

1. Once the image build is successful, click on the project name in the navigation menu.
2. Click on Event subscriptions and then Create.
3. Select Periodic timer, provide a name and click Next.
4. Under cron expression, add */60 * * * * to schedule backup every hour and click Next. You can schedule the backup to run hourly, daily, weekly, monthly, yearly or even every minute, if you want:

Code Engine create an event subscription

5. Skip custom event dataand click Next.

6. Under the Event consumer page:

Select Component type: Job

Name: <Name of the Job created earlier using the source code>

7. Click Next.

8. Check the Summary and click Create.

Check the backup status

You can check the backup status either on the PostgreSQL service page or under the Code Engine job runs tab:

1. Navigate to the resource list and under Services, click on the name of the PostgreSQL service.
2. Click on Backups and restore to see the automatic and on-demand backups under the Available backups section.
3. You can also check the Job run that initiates the job to see the status:

• Navigate to the Code Engine project page.
• Click on the name of the project and then click Jobs.
• Click on the Job name > Job runs > Check the status in the table:

What’s next?

• Instead of using environment variables as literal values, learn how to bind Cloud services to an application and a job: Tutorial: Text Analysis with IBM Cloud Code Engine
• Image Classification with IBM Cloud Code Engine and TensorFlow

Conclusion

Following the steps in this post, you learned how to create a container image directly from the source code on a Git repository, push the image to a private container registry and then create a cron job from the container image to automate the PostgreSQL backups using IBM Cloud Code Engine.

Along with the IBM Cloud console, you can also use the IBM Cloud CLI with Code Engine and cloud-databases plugins to achieve what’s shown above. Remember, PostgreSQL deployments come with free backup storage equal to the service total disk space. If your backup storage usage is greater than total disk space, each gigabyte is charged. Backups are compressed, so even if you use on-demand backups, most deployments will not exceed the allotted credit.

If you have any queries, feel free to reach out to me on Twitter or on LinkedIn.

Originally published at https://www.ibm.com on March 31, 2022.