Find Security Vulnerabilities in Container Images using IBM Cloud Container Registry and Vulnerability Advisor
Introduction
The concept of DevOps (Development, Operations) has been a cornerstone in software development and IT operations for multiple years. This concept has since evolved into DevSecOps (Development, Security, Operations), and teams now see the necessity of embedding continuous security in development and operations workflows. Since containers are one of the cornerstones of cloud computing, container image security is now a major focus areas when improving security posture in a cloud environment.
Many organizations are great at implementing image and code scanning at the beginning of the lifecycle when the source code is compiled and built into an image. However, this is not good enough because it assumes no vulnerabilities are found in the image afterwards. Imagine a scenario where a container image is only scanned when it is built. What happens when a new CVE is found for a package dependency in the container’s base image the next day? If the image is only scanned when built, the vulnerability might be found days (or even weeks!) later. For those familiar with IT operations, the example scenario falls into the Day 2 operations category.
IBM provides the IBM Cloud Container Registry (ICR) as a solution to make continuous image scanning and continuous security easier in a DevSecOps focused world. ICR is tightly integrated with Vulnerability Advisor (VA) which scans images for security vulnerabilities. ICR is also one of IBM Cloud’s many Financial Services (FS) Validated services, which means it leverages the industry’s highest levels of encryption certification, provides preventive and compensatory controls for financial services regulatory workloads, multi-architecture support and proactive, and automated security. To learn more about IBM Cloud for Financial Services, visit the website here.
This article will go through the general architecture of ICR and then go through steps for pushing an image and viewing its image vulnerability report.
Architecture Overview
From the IBM Cloud documentation page:
ICR is a multi-tenant, highly available, scalable, and encrypted private image registry that is hosted and managed by IBM.
Similar to Docker Hub, quay.io, etc. ICR houses container images that can be deployed to platforms on cloud and on-prem. Connections can be made through both public and private URIs depending on the environment. ICR runs backend services in IBM Kubernetes Service (IKS) clusters and leverages IBM Cloud Object Storage to store container images.
There are 3 different scenarios where ICR scans container images with VA:
- When a new image is pushed to the registry
- When a new security notice is released for a package that is installed in the image
- When a tagged image is housed in the registry, it is scanned every week
ICR exposes a registry endpoint for different major regions along with a global registry. The coming steps in the article will use the global registry with an endpoint of icr.io . All of the endpoints for the different registry endpoints are listed here.
Prerequisites
- IBM Cloud Account
- IBM Cloud CLI
- Docker Desktop
Setup IBM Cloud CLI and Login
To push a container image to ICR, you will need to install a plugin to the vanilla IBM Cloud CLI using the following command:
ibmcloud plugin install container-serviceNow login to the CLI and container registry using the following two commands:
ibmcloud login --sso
ibmcloud cr login
To push an image in the next step there needs to be a custom namespace created in the account ICR. Use the following command to create a namespace in ICR:
# In the example below:
# namespace = rs-cribmcloud cr namespace-add rs-cr
Push an Image to ICR
To showcase Vulnerability Advisor integration with ICR this section will focus on pushing a vulnerable image to the account. For the widest availability the chosen image will be an official alpine node.js image found here. The specific name and tag are node:14.16.0-alpine .
Before pushing the image to ICR, the image needs to be re-tagged to match the registry endpoint. Use the following command to tag the image:
# New tag should be of the form
# icr.io/<namespace>/<repository>:<tag>
#
# In the example below:
# namespace = rs-cr
# repository = node
# tag = latestdocker tag node:14.16.0-alpine icr.io/rs-cr/node:latest
Use the following command to push the image to ICR:
docker push icr.io/rs-cr/node:latest
The image is now successfully housed in ICR!
View Security Vulnerabilities through UI
Once the image is pushed to ICR it will automatically be scanned with Vulnerability Advisor. To see the image and corresponding scan results, visit the IBM Cloud Container Registry Dashboard. There will be an Images table with a row for the recently pushed image. There is a Security status column that shows the number of vulnerabilities that VA has detected.
To see more information about the vulnerabilities, click the table entry to view the scan results dashboard. The default menu will be the image details, which shows metadata about the container image such as Docker build version, image creation time, entry command, and more. To see the list of vulnerabilities and more details, click on the issues by type tab on the left of the dashboard.
The issues menu will list the number of vulnerabilities as well as an entry for each CVE in a table format. There is basic information about each CVE in the entry, and clicking the dropdown icon in any row will show more detailed information. An example of the detailed information for CVE-2021–42374 is shown in the image below.
The detailed information gives a summary about the CVE and its associated notice IDs, plus affected packages and resolution steps. All of the information is discovered and returned through the VA integration in ICR.
View Security Vulnerabilities through API
Viewing the security vulnerabilities in a dashboard is great, but the example above is only monitoring one container image. With the popularity of microservices architecture and the amount of applications enterprises have, the number of container images went quickly grow into the hundreds and thousands. Using a dashboard to monitor and remediate that number of container images would be incredibly difficult, even for a large DevSecOps team.
To resolve the issue, VA exposes a REST API for ICR users to programmatically get information about their image security vulnerabilities. This opens the door for automation which can scale easily even as the number of container images grows. View the full Vulnerability Advisor API documentation here. To begin accessing the API, two pieces of authentication are required:
- IBM Cloud Authorization Token
- IBM Cloud Account UUID
To get the authorization token, run the following command:
ibmcloud iam oauth-tokens
To get the cloud account UUID, run the following command:
ibmcloud account show
The cloud account UUID will be found under the name Account ID as shown in the example output above. The two pieces can now be used to authenticate against the VA API and access vulnerability scans through the command line. To do so, run the following cURL command:
# Note: The <auth_token> below includes the 'Bearer' prefixcurl -X 'GET' 'https://icr.io/va/api/v3/report/account' \
-H 'Account: <account_id>' \
-H 'Authorization: <auth_token>'
-H 'Accept: application/json'
A truncated sample response is found in the image above. A more detailed sample response is found in the GIF below. It shows the same data that is seen in the ICR UI in the previous section. It also provides information such as the time when the image was scanned, exemption status (e.g. passing status when the vulnerability is expected), etc.
The API endpoint shown above gets scan results for all of the images found in the registry. The VA API also exposes an endpoint to search for a specific image within the registry. To find the scan result for a specific container image, use the the following command:
# Note: The <auth_token> below includes the 'Bearer' prefix
# The image <name> should be of the following form:
# icr.io/<namespace>/<repository>:<tag>curl -X 'GET' 'https://icr.io/va/api/v3/report/image/<name>' \
-H 'Account: <account_id>' \
-H 'Authorization: <auth_token>'
-H 'Accept: application/json'
The above command will return the same information from the prior command, but will have only one entry (if the image specified exists in the registry).
Summary
This article demonstrated how to create an ICR namespace, upload a vulnerable image, and access security vulnerability scan results through the UI and through API endpoints. It also went through the architecture and context around using ICR to fit into a DevSecOps mindset. View any of the following links for more information about the specific services mentioned in this article:
