Exploiting Jenkins build authorization

Are you aware of the risks lurking in your default Jenkins configuration?

TL;DR

The default build authorization configuration in Jenkins — controlling the permissions allocated to pipelines — is insecure and is often left unmodified in production environments. To address this issue, you should use the “Authorize Project” and the “Role-Based Authorization Strategy” plugins to define secure build authorization configurations.

Introduction

Jenkins is the most widely used CI/CD automation platform. Its great flexibility and extensibility helps thousands of organizations solve many of their engineering challenges. Having said that, Jenkins is also probably one of the most complex CI/CD automation platforms; While its advanced flexibility and extensibility has many advantages, it also has some significant pitfalls that must be acknowledged and addressed.

One significant pitfall is the platform’s insecure default configurations. The default configurations are designed to allow functionality on initial setup of the system and are meant to be replaced after initial installation, as advised by Jenkins’ documentation. However, these insecure configurations often remain in production environments unbeknownst to DevOps and security engineers.

In this blog we explore the potential dangers arising from one of these default configurations — the build authorization configuration.

What is “build authorization” and why are its default settings problematic?

Running builds require authorization to perform actions in Jenkins, so every build is associated with a user account. This association is called “build authorization.” The permissions are used to perform the various actions that the build needs, such as allocating an agent to run the build or storing build results in the system.

The authorization model in Jenkins consists of permissions that control the access to different resources and actions on various objects in the system. Permissions are verbs, grouped by different system components.

Some notable permissions are:

• Overall/Administer — Allows users to do everything.
• Job/Configure — Allows users to configure jobs.
• Job/Build — Allows users to trigger builds.
• Agent/Build — Allows users to execute builds on an agent.

All other descriptions can be viewed by hovering over the permission names in Jenkins.

There is one more special permission that is not visible in this table, which is called SYSTEM. This is the permission used by Jenkins itself and it can access any resource and perform actions on any object in the system

The most important thing to note in this context is that the Jenkins default settings assign every build to “run as system.” In other words, they assign it to the all-powerful SYSTEM user, meaning any action executed during the build has permission to do whatever it wants.

This can become a major problem when an attacker obtains permissions to a repository linked with a pipeline — by compromising developer credentials, access token, SSH key, or any other method. If the hijacked user account has the ability to modify the Jenkinsfile and, practically, the Jenkins pipeline, the attacker can take advantage of it to run malicious commands in the pipeline by executing a PPE (Poisoned Pipeline Execution) attack. With the default build authorization settings, these malicious commands will run with the almighty SYSTEM user. Some examples of actions that can be performed from within builds are: triggering other builds, creating new jobs and running on different nodes.

How exactly does the default build authorization leave us vulnerable?

To demonstrate why the default configurations are so bad, we’re going to simulate a common situation in which attackers have gained initial access to an organization’s repositories and can push code (either to new or existing branches) that triggers a pipeline defined by a Jenkinsfile in their control — to which they will add malicious commands. As regular users, the attackers in our scenario have only Overall/Read and Job/Read permissions on Jenkins.

When attacking Jenkins through manipulation of the Jenkinsfile, the attack surface that’s available consists of two elements: (1) pipeline steps that can be invoked — dependant on which plugins are installed, and (2) Groovy scripts that can run within the Groovy Sandbox whitelist, assuming it’s turned on as defined in the default configurations. Here we will focus on element #1.

To map all available pipeline steps, the attackers can do multiple things, for example:

• Try to use steps from the Pipeline Step Reference and see if they work. Most likely, the set of plugins suggested for installation by Jenkins during its installation will be available.
• Look for steps used inside the organization’s Jenkinfiles stored in accessible SCM (source code management) repositories.
• Write a script to parse all build outputs that they can read and look for more available steps.

Let’s look at some example attack scenarios:

Attack scenario 1: Running builds on agents

Users with Overall/Read permissions can view all agent names and their labels. Another option for attackers to discover agent labels is to look for them in accessible SCM repositories. Using this knowledge, attackers can execute a Direct-PPE attack and change the agent label in the Jenkinsfile. The attackers can then run builds on any agent they choose — due to the fact that the default Jenkins configuration is to run builds as the SYSTEM user.

The following is an example of the change made to the Jenkinsfile from the diagram above:

By running their build on different agents, attackers can perform multiple actions, such as (a) finding valuable files created by previous jobs running on that agent; (b) extracting sensitive environment variables that were loaded by other builds; (c) gaining extensive network access, which can be used for lateral movement through the organization’s assets; (d) leveraging the agent’s identity, like in the case of an agent running on an AWS EC2 instance which can be used to obtain the permissions of the EC2 instance role in order to perform actions against the AWS account.

Attack scenario 2: Triggering builds

Attackers can exploit the default build authorization permissions to trigger any build job they choose using the build step (and provide it with any parameter they want if it accepts parameters). This step requires the Job/Build permission and can be used in the following way:

The only requirement for this is knowing the target job name, a likely scenario in most environments as global Job/Read permissions are in many cases given to all users. Another means of discovering job names is to check pull requests in the organization’s SCM repositories, which may contain the job name that a request triggers.

An attacker that is able to run build jobs can then carry out malicious deployments using one of the following:

• Deploying malicious code that was previously pushed to the codebase
• Triggering a deployment pipeline with parameters controlled by the attacker that will ultimately change the behavior of the application.

What should we do to protect ourselves?

To protect against attacks such as those described above, two things need to happen. First, we need to make sure that new builds do not automatically run as SYSTEM, but rather by a specific user. Second, we must see to it that each user has only the minimum possible permissions that it needs to execute builds.

1) To achieve the first goal, we can use the “Authorize Project” plugin.

The “Authorize Project” plugin allows us to define both default and per-project build authorization. It offers the following four options:

• Run as the user who triggered the build
• Run as anonymous
• Run as the specified user
• Run as SYSTEM

Builds can be triggered either manually, or by a series of “automatic” triggers, such as: “build periodically,” “discover pull requests,” “poll SCM” and so on. When triggered manually, by pressing the play button for example, the build will run as the user who manually initiated it. Every time a build is triggered by an automatic method, it will run as whatever user has been defined by the system as the “default” build authorization user.

However, if “per-project build authorization” has been defined for that project, the build will run as the designated user regardless of whether it was triggered automatically or manually.

The plugin should therefore be used not only to ensure that the default user is no longer defined as SYSTEM, but also to reduce the attack surface by configuring per-project build authorization, with different users that have explicit permissions for specific projects, whenever possible.

Per-project build authorization can be configured for Pipelines, Freestyle and Multi-configuration projects in the project screen under “Authorization”. Note, however, that it cannot currently be configured for Multibranch Pipelines. Since Multibranch Pipelines are a widely used feature of Jenkins, not being able to configure per-project build authorization poses a major security issue. If you’d like to encourage the contributors of Jenkins to provide a solution, feel free to vote on the issue here.

2) To achieve our second goal (making sure each user has the minimum possible permissions that it needs to execute builds), we can use the “Role-Based Authorization Strategy” plugin.

This plugin allows system administrators to create roles with particular permissions and then assign them to users or groups. The roles are divided into three categories:

• Item roles — allow users to perform the actions specified, but only on the items (such as projects, credentials) that match the specified regex.
• Node roles — allow users to perform the actions specified, but only on the agents that match the specified regex.
• Global roles — allow users to perform the specified action on all objects in the system.

Once the roles have been configured, they can then be assigned to users or groups, according to the authentication method in use.

“Matrix Authorization Strategy” is another potentially useful plugin to address this issue.

So what exactly should I do?

Now that we’re familiar with the threat and with the tools we can use to address it, here are few specific steps you should follow:

1. To prevent builds from running as SYSTEM, configure an alternate default build authorization strategy and per-project build authorization strategy.
2. To minimize user privileges, assign human users and build users specific, limited roles using the “Role-Based Authorization Strategy” plugin.
3. To minimize the attack surface, remove any unused plugins to reduce available steps.

How to configure build authorization strategies

1. Install the “Authorize Project” plugin.
2. Because the “Authorize Project” plugin doesn’t support configuring per-project authorization for Multibranch Pipelines, configure the “Project Default Build Authorization” under “Configure Global Security” to use a specific user for all Multibranch Pipelines. That user should not be used by a human and it should usually have only Agent/Build for specific nodes and Job/Read for all Multibranch Pipelines configured using the “Role-Based Authorization Strategy” plugin.
3. Configure “Per-project configurable Build Authorization” under “Configure Global Security” to enable one or more of the following: “run as specific user”, “run as anonymous” and “run as user who triggered build”. Note that the first is the best option of the three (in terms of both security and function).
4. For Pipelines or Freestyle jobs, configure each job’s “Authorization” in the job’s screen to “run as specific user” with a user specially created for that job or group of jobs using an item and node roles that consist of minimal privileges such as Agent/Build and Job/Read only.

Best practices for configuring user permissions

Use the “Role-based Authorization Strategy” plugin to configure the following settings:

1. Add all users except administrators to a global role that has only the Overall/Read permission. If more global roles are needed, assign them to a minimum number of users.
2. Because roles are defined using regular expressions, it is recommended to enforce project naming conventions to allow easily configuring granular roles.
3. Create specific roles for both items (e.g. Jobs, Credentials) and agents with patterns matching the projects and nodes they need access to.
4. Avoid assigning users with the Job/Configure, Job/Delete, Job/Create, Job/Move, Run/Delete, Run/Replay, Agent/Configure, Agent/Connect, Agent/Delete, Agent/Disconnect and Agent/Provision permissions, since these pose more risk.
5. Consider removing the Job/Build permission from users to prevent a scenario where a specific pipeline was manipulated/tampered with, and then executed by unsuspecting privileged users, allowing the compromised build to have elevated privileges.

If all of these actions seem a bit daunting and complex, note that you can use the Configuration as Code plugin to simplify the process considerably. Jenkins Configuration as Code provides the ability to define Jenkins configuration as a simple, yaml syntax. You can see examples of how to do that here.

Seeing that Jenkins configurations are dynamic we need to ensure that these recommended configurations remain unchanged and that we are able to detect any major change that might have a negative influence on our attack surface. In case your environment is connected to the Cider platform, you can view any such changes under the “Pipeline Risks” page.