CI/CD for NodeJS on OCI without Kubernetes

11 min readFeb 19, 2024

Just recently I published an Azure DevOps example on Oracle DevRel GitHub how to do CI/CD for Oracle Kubernetes Engine (OKE) using Azure DevOps tasks. Fun stuff!

However, not everyone wants to build and run on Kubernetes and wants to use plain infrastructure VMs instead. Let’s have a look in this article how to do that for a NodeJS application in OCI using OCI DevOps service.

VM based CI/CD for a sample NodeJS application can be achieved by the following steps:

Create a compartment
Create a VM in VCN private subnet
Connect to VM using OCI Bastion to set it up for OCI DevOps
Open VM firewall for application VCN traffic in port 5000 and setup Run Command agent (ocarun) for OCI DevOps
Setup OCI DevOps policies
Create the OCI DevOps project CI/CD
Create OCI Load Balancer to serve requests from Internet

I’d say this can be won’t be taking no more than 60 mins of your time to complete. Let’s go!

Compartment

Start this example by creating creating a compartment for the application setup. Select Identity & Security/Compartments from the Cloud UI menu and hit the “Create compartment” button. Give the compartment a name e.g. “NodeJS”.

We’ll be using this compartment for all the next steps in this article.

VCN and subnet creation

A VCN for our purpose can be created very fast with OCI Cloud UI using the VCN Wizard with “Create VCN with Internet Connectivity”. To do this select Networking/Virtual Cloud Networks from the Cloud UI menu and hit the “Start VCN Wizard” button.

It is perfectly fine to use the default settings by just going thru the Wizard steps and finally hitting the “Create” button. The only change I would do is to give the VCN a more meaningful name like “public-vcn”.

Create the VM

Next, let’s create the VM to deploy our NodeJS application later on it. Choose Compute/Instances from the Cloud UI menu and hit the “Create instance” button.

For this purpose I like to use the Oracle Cloud Developer OL8 VM image that contains all the software required in this example preinstalled. Choose this from the list of available Oracle Linux Images. Give the VM a name “NodeJS” for example.

List of available Oracle Linux VM images

Select the private subnet to be used for the VM creation. Other defaults will do just fine. You don’t need to assign a ssh key since we’ll be using OCI Bastion service instead to connect to the VM in next steps.

Going thru the instance creation steps will then create the VM with only a private address e.g. 10.0.1.241

Access VM using OCI Bastion

Select Bastion from the Identity & Security menu to and hit the “Create bastion” button.

For the CIDR block allowlist “0.0.0.0/0” can be used and select the VCN public subnet for the Bastion creation.

After the Bastion is created add a new session to it. It will run a couple of hours after it’s creation. Set Username as “opc”, select previously created compute VM instance “NodeJS” and for the SSH key use the “Generate SSH key pair” option and download the private key to your local machine.

Once the session is running copy the SSH command using the three dots from the right and access the VM instance from your localhost:

ssh -i key.txt -o ProxyCommand="ssh -i key.txt -W %h:%p -p 22 ocid1.bastionsession.oc1.eu-frankfurt-1.amaaaaaauevftmqazfh3dusw.....k2bzvgq@host.bastion.eu-frankfurt-1.oci.oraclecloud.com" -p 22 opc@10.0.1.241
[opc@NodeJS ~]$

Setup VM firewall and ocarun for OCI DevOps

Now that we are in the VM, let’s adjust the firewall for our application that will use port 5000 for it’s network traffic:

sudo firewall-cmd --add-port=5000/tcp --permanent
sudo firewall-cmd --reload

The next thing to setup on is enable the Run Command agent “ocarun” on the VM. This allows OCI DevOps to send files over to the VM like the NodeJS app artifact once built and run a shell script to start it. Follow this guide to enable it.

Once set you should see the Run Command agent poller running successfully on the VM:

[opc@test ~]tail -f  /var/log/oracle-cloud-agent/plugins/runcommand/runcommand.log
2024/02/19 07:54:06.993981 poll.go:534: poll command status: 200; size: 0 took: 39ms; with opc-request-id:d78f7f037ad222bb2edb3f00c2cce6e4/FC4DA88D43C6E7440D318AAB67649EF0/6EB12B72579A3E6D9FF1C42918EA838E; 
2024/02/19 07:54:06.994036 scheduler.go:131: SimpleScheduler scheduling to run PollCommand after 3m14.738600535s
2024/02/19 07:57:21.735618 base_client.go:87: fetched metadata from http://169.254.169.254/opc/v2/instance/ , status 200 OK
2024/02/19 07:57:21.735806 jitter.go:24: jittering for 34.578707035s
2024/02/19 07:57:56.348323 poll.go:534: poll command status: 200; size: 0 took: 33ms; with opc-request-id:ccbffbec2d010cfad459289c85c75f4a/0DA3CF244C479A678547E56A9BCF58AB/5186126A732E9585E6DAAC1178CE3483; 
2024/02/19 07:57:56.348370 scheduler.go:131: SimpleScheduler scheduling to run PollCommand after 3m25.384373069s
2024/02/19 08:01:21.736543 base_client.go:87: fetched metadata from http://169.254.169.254/opc/v2/instance/ , status 200 OK
2024/02/19 08:01:21.736764 jitter.go:24: jittering for 56.61708866s
2024/02/19 08:02:18.455766 poll.go:534: poll command status: 200; size: 0 took: 102ms; with opc-request-id:750849df0ad93a79ea2e4987a5014b48/4542BF4AB2BC2C687C98C65B6357FDE2/C82A94593D14F3DCEC7AD72C64CC4072; 
2024/02/19 08:02:18.455814 scheduler.go:131: SimpleScheduler scheduling to run PollCommand after 3m3.277968304s

Setup OCI DevOps policies

Depending on the tenancy structure setting up the OCI DevOps policies can be a bit tricky. Follow this guide to setup the policies for OCI DevOps.

Besides the pipelines execution the policies should allow the OCI Artifact Registry access and Run Command agent execution e.g.

Allow dynamic-group <YOUR DYBNAMIC GROUP HERE> to manage generic-artifacts in compartment NodeJS
Allow dynamic-group <YOUR DYBNAMIC GROUP HERE> to manage instance-agent-command-execution-family in compartment NodeJS

Create the OCI DevOps project CI/CD

Now that all the OCI environment preparations have been done let’s get to the fun stuff that is setting up the CI/CD pipelines for our NodeJS application. The steps to do this are as follows.

Create a new Topic for the DevOps project by selecting Developer Services/Application Integration/Notifications from Cloud UI menu and hitting the “Create topic” button. Please note this might need an additional policy to be created as advised on the create page.
Create a new OCI DevOps project by selecting Developer Services/DevOps/Projects from the Cloud UI menu and then hit the “Create DevOps project” button.
Give the project a name e.g. “NodeJS on VM” and set up the project notifications by selecting the the topic created in the step 1.
Once the DevOps project has been created navigate to it and create the project GIT repo with a name “NodeJS” for example by selecting “Code Repositories” from the menu on the left. Then navigate to the created repo and hit the “Clone” button. Copy the “Clone with ssh” command.

Before you can actually clone the repo locally using ssh it needs to be setup the local env as instructed on the repo page:

I’m using my OCI user apikey for this that I use also for the oci cli:

Host devops.scmservice.*.oci.oraclecloud.com
User oracleidentitycloudservice/mika.rinne@oracle.com@<MY TENANCY NAME>
IdentityFile /Users/MRINNE/.oci/oci_api_key.pem

After doing this you should be able to clone the repo locally and copy the files from this GitHub source repo and push them to the OCI DevOps repo:

Example source files pushed to OCI DevOps repo

Alternatively you can fork the GitHub source repo under you GitHub user and then use that as a source repo for the OCI DevOps Build pipeline by using OCI DevOps repo mirroring.

5. Create a new Artifact Registry for our NodeJS application artifact (a zip file) by selecting Developer Services/Containers & Artifacts/Artifact Registry from the Cloud UI menu and hitting “Create repository” button.

Registry can be created as immutable that means existing files cannot be overwritten; instead a file with a new version needs to be created. Name it “nodejs” for example.

Then create a new artifact in the DevOps project by selecting “Artifacts” from the menu on the left and hit “Add artifact” button. Have the following settings for it:

name: NodeJS
type: General artifact
source: Artifact Registry and select the registry created previously
location: Set custom location
Path: nodejs.zip
Version: ${buildId}
Allow parameterization: checked

OCI DevOps NodeJS artifact in the OCI Artifact Registry

6. Select “Build Pipelines” from the menu on the left and hit the “Create build pipeline” button to create a new pipeline called “NodeJS” for example. Then using the pipeline canvas add three stages onto the pipeline flow (click the + sign to add each):

“Managed Build” stage with the build_spec.yaml file
“Deliver artifacts” stage with the following setting for the NodeJS artifact:

Build config/result artifact name “nodejs” is important as it points to the variable “OutputArtifacts” name in the build_spec.yaml

“Trigger deployment” stage — this can be added once the deployment pipeline has been created in the step 9.

7. Then create a new artifact in the DevOps project by selecting “Artifacts” from the menu on the left and hit “Add artifact” button. Have the following settings for it:

name: deploy_spec.yaml
type: Instance group deployment configuration
source: Inline
value: Paste the contents of the deploy_spec.yaml file into it
Allow parameterization: checked

OCI DevOps deploy_spec.yaml inline artifact

8. Create a new Environment by selecting “Environments” from the menu on the left and hitting “Create environment” button.

Choose type “Instance Group”, give it name e.g. “test” and manually add a instance from the list on VMs; select the VM instance “NodeJS” that was created in the beginning of this article.

9. Select “Deployment Pipelines” from the menu on the left and hit the “Create pipeline” button to create a new pipeline called “deploy NodeJS” for example. Then using the pipeline canvas add the deployment stage onto the pipeline flow (click the + sign to add it) with the following settings:

name: deploy NodeJS
type: Deploy Instance Group: Rolling
environment: test (the one that was created in the previous step)
deployment configuration: select artifact “deploy_spec.yaml”
select artifact: NodeJS (of the type “General Artifact”)
rollout policy: Rollout by count “1”

(Other settings can be left untouched)

Deployment Pipeline deploy stage (in Edit mode)

10. Now can go back to edit the build pipeline and add the “Trigger deployment” stage to it as described in step 6.

11. Create a Trigger that will kick of the CI/CD whenever there is git commit & push to the repo by selecting “Triggers” from the menu left and hit “Create trigger” button. Use the following settings for it:

name: commit and push
source: OCI code repository and select the repo of this DevOps project
add action: Select the build pipeline of this DevOps project and check the option event “Push” for it

Testing the CI/CD

To test the build and deploy pipelines in action either start the build pipeline manually by selecting the build pipeline to edit it and hitting the “Start manual run” button from top right corner or commit a change to source/server.js file locally and then pushing it the repo using git push.

The build pipeline will then run and once completed look like this:

Now let’s look at the build steps in build_spec.yaml:

version: 0.1             
component: build
timeoutInSeconds: 180
shell: bash
failImmediatelyOnError: true
env:
  exportedVariables:
    - buildId
steps:
  - type: Command
    name: "create build-id for the artifact"
    command: |
      buildId=`echo ${OCI_BUILD_RUN_ID} | rev | cut -c 1-6 | rev`
      echo "Build ID: $buildId"
  - type: Command
    name: "build and run NodeJS app, create zip artifact if runs ok"
    command: |
      cd source
      npm install
      npm start &
      sleep 2
      res=$(curl -s localhost:5000)
      echo ">>>> $res"
      cd ..      
      zip -q -r nodejs.zip source
outputArtifacts:
  - name: nodejs
    type: BINARY
    location: ./nodejs.zip

What this does is:

Generate an exported var “buildID” from DevOps ${OCI_BUILD_RUN_ID}
Install and run the NodeJS app from source directory
Test it with cURL and if no errors zip the source directory and store it as an artifact to the Artifact Registry with the generated “buildID” as artifact version
If there is an error testing the NodeJS app fail immediately
If no errors continue the CI/CD by triggering the Deployment pipeline

Once the deployment pipeline has been triggered and also executed it will look like this:

Now let’s look at the deployment steps in deploy_spec.yaml:

version: 1.0
component: deployment
runAs: ocarun
env: 
  variables: 
    version: ${buildId}
files: 
  - source: /nodejs.zip
    destination: /tmp
steps:
  - stepType: Command
    name: kill NodeJS
    command: "pkill -f 'node /home/opc/source/server.js'"
    timeoutInSeconds: 60
    runAs: root
  - stepType: Command
    name: copy NodeJS
    command: "cp /tmp/nodejs.zip /home/opc/nodejs.zip"
    timeoutInSeconds: 60
    runAs: root
  - stepType: Command
    name: unzip
    command: "unzip -o nodejs.zip -d /home/opc"
    timeoutInSeconds: 60
    runAs: root
  - stepType: Command
    name: start NodeJS
    command: "nohup node /home/opc/source/server.js &"
    timeoutInSeconds: 60
    runAs: root

This will copy the nodejs.zip artifact with “buildId” version over to the VM (defined in the DevOps environment) and then run the shell commands to (re)start the NodeJS app under /home/opc directory on it.

After execution the steps above should be visible in the end of the pipeline log:

...
2024-02-19T08:38:25.000Z
[info] Step unzip succeeded   
2024-02-19T08:38:25.000Z
[info] executing step 5 of 5, name:start NodeJS   
2024-02-19T08:38:25.000Z
[info] executing command step for id:ocid1.devopsdeployment.oc1.eu-frankfurt-1.amaaaa......4jkl57k7rn47g2a, name:start NodeJS   
2024-02-19T08:38:25.000Z
[info] The command to be run is /usr/bin/sudo -n -u root -E nohup node /home/opc/source/server.js &  
2024-02-19T08:38:25.000Z
[info] using bash as the step level shell for step 4   
2024-02-19T08:38:25.000Z
[info] Step start NodeJS succeeded   
2024-02-19T08:38:32.198Z
Deployment executed successfully for instance Id: ocid1.instance.oc1.eu-frankfurt-1.antheljsuevft.....ywkrudk5ze3frq   
2024-02-19T08:38:48.808Z
All stages complete.

In the Cloud UI the deployment pipelines executed will appear in the for Compute Instance Run Command log:

We can use the OCI Bastion to login to the VM and test the NodeJS app locally with cURL:

ssh -i key.txt -o ProxyCommand="ssh -i key.txt -W %h:%p -p 22 ocid1.bastionsession.oc1.eu-frankfurt-1.amaaaaaauevftmqaz.....6mbk2bzvgq@host.bastion.eu-frankfurt-1.oci.oraclecloud.com" -p 22 opc@10.0.1.241
[opc@NodeJS ~]$ curl localhost:5000
Hello from NodeJS on OCI <3
[opc@test ~]$

Create OCI Load Balancer to serve requests from Internet

We have now reached the final step in this article that is to open the traffic from Internet to our NodeJS appliction from Internet.

To do this select Networking/Load Balancers, then select “Load Balancer” from the menu left and hit the “Create Load Balancer” button.

Add the following load balancer details (other settings can be kept as default):

name: NodeJS
visibility: public
VCN: select the “public-vcn” that was created for the VM in the beginning of this article for the NodeJS app
subnet: select the VCN public subnet (regional)
add backends: select the NodeJS app VM and set port 5000 for it
specify health check policy: port 5000
traffic type: HTTP

VCN subnet security lists:

Load Balancer advanced options can be used to automatically to modify the public subnet security list for the HTTP port 80. Alternatively the rule for port 80 can be added manually with a source CIDR of “0.0.0.0/0”.

The port 5000 will be automatically added to the private subnet security list to allow traffic flow from the load balancer to the NodeJS app running on the port 5000. Here the load balancer public subnet CIDR e.g. 10.0.0.0/24 is used as the security rule source address.

Make sure these rules exist or otherwise the traffic won’t flow properly thru.

After the load balancer creation the backend status should turn green after a while: