Using DevSecOps to Strengthen Security on GCP

Spiros P
Reblaze Blog
Published in
7 min readAug 1, 2019

DevSecOps increases the speed and security at which software is developed and delivered. In today’s fast-moving world, it’s vital to ensuring your application and infrastructure solutions are sustainable, scalable, and secure.

We previously covered what DevSecOps entails and how to solve some of its common challenges. If you aren’t familiar with DevSecOps, it would be helpful to read that article first, before continuing here.

In this article, we will explore how some of the features of Google Cloud Platform (GCP) can be leveraged for DevSecOps, reducing the risk of security incidents that can negatively affect your organization. We will cover identity and access management, audit logging, network control, data encryption options, sole-tenant nodes, virtual machine pipelines, container services, automated scanning, and more.

The Cloud DevSecOps Landscape

As cloud offerings mature, providers are increasingly competing to improve security features in order to attract larger enterprises. Because of this competition, their feature sets are expanding quickly. The three big players (AWS, Azure, and GCP) all take security seriously, but are working in different ways and from different starting points. AWS has a mature patchwork of features that give organizations a lot of power. GCP and Azure are playing catchup with AWS in terms of maturity, but GCP benefits from Google’s many years of experience designing and running its own cloud infrastructure (not least in the container space, where Google played a key part in the inception of containers).

Identity and Access Management

The first foundational area of security to consider when designing your cloud infrastructure is identity and access management (IAM) policy. A solid IAM framework allows your organization to implement the principle of least privilege while retaining control and flexibility. GCP offers four levels of control: Organization, Folder, Project, and Billing Account.

Briefly, Organizations contain Folders and Billing Accounts, while Folders contain Projects. Projects contain all the cloud resources you might expect, such as virtual machines, firewall rules, and so on. You can use this structure to manage permissions at the Organization level so that ordinary users are constrained in what they can do.

For example, you might only allow a very restricted set of users access to particular Folders that you’ve marked as containing “production” Projects. Or you can restrict access across your organization to prohibit the use of products deemed out of bounds by your security function. You can even go down to the level of restricting access to serial ports on virtual machines. The setup of your IAM rules and structure is a critical part of your cloud infrastructure design, and it is worth taking the time to get right.

Logging and Audit: Who Did What When?

In addition to restrictions on resource access with IAM controls, you can monitor GCP usage with the Stackdriver Logging and Stackdriver Monitoring tools. Stackdriver offers functionality similar to Amazon CloudWatch and AWS CloudTrail, which means that you can log and monitor cloud events as they happen. Of particular interest from a DevSecOps perspective is GCP’s Cloud Audit Logs feature, which allows you to determine “who did what, where, and when.” It’s important to note, however, that not all services have audit logging fully available yet, so if this is a critical feature for you, it’s wise to check its availability.

Stackdriver not only offers standard monitoring and audit logging features, but for organizations with a high security bar, it also offers the capability to log the actions Google staff take when servicing your account. Using this feature, you can check which Google staff have accessed which resources, when, and whether that access was expected or appropriate for the support or maintenance task they were performing.

Reactive Controls

You can take this one step further by implementing so-called “reactive controls” via Stackdriver Second-Party Triggers. By creating a Stackdriver logging sink, and linking it to a messaging topic, you can ensure that a Google Cloud function is run on a specific kind of event. You might use this functionality to shut down resources that violate your company policy, or to create a ticket in a third-party system to investigate the use of some rarely-used and sensitive feature.

Network Security

Another facet of cloud access control is perimeter security via the network. In comparison to AWS, GCP has a much simpler approach to network access control due to its design, as each Project in GCP gets its own global network to work with. Using GCP’s firewall rules feature, you can control access at the port and protocol levels to the whole network, individual IP addresses, or tagged groups of machines. You can also limit access by source IP range, allowing the implementation of the principle of least access over the network from within and outside projects.

Data Security

Most organizations have some requirements around the encryption of data, and GCP provides a standard set of options to control the level of encryption required. First, all data is encrypted under 256-bit AES at rest on GCP drives, and master keys are regularly updated by Google. If that does not suffice, then you can use GCP’s Cloud Key Management Service (Cloud KMS) to generate keys that you own, manage, and monitor for usage. You can also implement envelope encryption to have greater control over key creation and revocation. Finally, if you require full control over the creation and management of keys, you can “bring your own key” to the platform. This ensures that Google does not store a copy of the key, thereby removing the possibility that Google can see your data at all.

If data retention is a regulatory requirement, then GCP’s Bucket Locks can be enforced at the organization level to ensure that data is not lost or deleted, either inadvertently or maliciously.

Tenancy

Another layer of protection for your applications at the hardware layer is provided by GCP’s sole-tenant node feature. One possible attack vector on cloud workloads is to target the data center node’s hypervisor. If this component is compromised, then potentially, all the workloads running on that node (or machine) will be compromised.

One way to achieve this is to become a “neighbor” of the target organization’s workload by running a workload on the same node. If the virtual machine (or workload container) can be “escaped,” and the hypervisor accessed through a vulnerability, then all the workloads on the same node are at risk. (Although it’s difficult to find publicized examples of these vulnerabilities being exploited — which isn’t surprising, given the commercial incentives to avoid publicizing such incidents — it’s fairly well known within the industry that such events have occurred.)

Certainly, cloud providers are quick to patch these vulnerabilities. To mitigate the risk, the sole-tenant node feature ensures that only your organization’s workloads run on nodes that your workloads land on, ensuring you won’t have any neighbors you can’t control.

VM Image Pipelines

Any DevOps endeavor involves the use of pipelines, and cloud virtual machine images are no exception. Many organizations prefer not to use the standard images supplied by the cloud provider, and insist on supplying their own hardened images. By building these images within a pipeline using Google Cloud Build, other security benefits can be realized through organizational policy, such as regular cycling of virtual machines. By decommissioning machines regularly (e.g., on a two-week cycle), you can remove the need for regular patching (barring an emergency), reducing your operational costs through lower audit requirements and your threat vectors.

However, such an approach requires a cloud-native architecture and DevOps method for your applications as well, in order to accommodate the regular removal and replacement of individual machines, sometimes without warning.

Containing Your Applications

Another approach to the “patching VMs” challenge is to use container technology to limit the scope of the application to only what it needs to run. Google has a rich pedigree in the container space. For example, it contributed cgroups to the Linux kernel in 2006 to “contain” the resource usage of applications, and, more recently, contributed significant efforts to the open-source libcontainer project.

Google’s major offerings in the container space are Google Kubernetes Engine (GKE) and the much older Google App Engine (GAE). Using these services can remove the burden of managing the security of the operating system and container control plane layer. In GKE, you can further reduce the risk to your business from a nefarious workload by limiting the privileges of the container’s capabilities and access to system calls, and controlling access between containers using network policies.

Scanning, Continuous Integration, and Delivery

You will still be fully responsible for the integrity of your software running on GCP. Here, Google’s Cloud Security Scanner can automate the scanning of vulnerabilities of software running on GAE, GKE and Google Compute Engine (GCE). As with all scanning tools, you should evaluate it to determine what types of vulnerabilities it can report on, as there is a lot of variation in this space. Combining your scanning tool with building your software using Google’s Cloud Build can give you added security through standard DevOps automation techniques. In addition, you can add security to your builds with secrets management options that use KMS to encrypt your key data.

Conclusion

As the adoption of DevOps tooling and techniques grows, the phrase “DevSecOps” is being heard more and more. GCP offers products and features that allow you to set the boundaries for what developers can do, protect the network with firewall rules, constrain what applications can do through containerization, automate your builds, and scan your applications. As with all cloud providers, you will need to pay attention to whether the specifics of the products’ features match your requirements.

Of course, even a full adoption of DevSecOps does not address other security needs, such as providing protection for your web applications and APIs against Internet attackers. For this, you need the flexibility of a full-fledged web security solution such as Reblaze, which supports your CI/CD and DevSecOps practices.

Here’s how to try it for free for 30 days.

--

--