Using GoCD API

Are you familiar with GoCD? If not, let me provide a very brief introduction about GoCD before getting into the topic.

Official website for GoCD can be found at https://www.gocd.org. In brief, it is an open source continuous delivery server to model and visualize complex workflows with ease.

This is what Wikipedia says:

GoCD is an open source tool which is used in software development to achieve continuous delivery of software. It supports automating the entire build-test-release process from code check-in to deployment. It helps to keep producing valuable software in short cycles and ensure that the software can be reliably released at any time.

That explains the whole scenario. There are few terms that you have to know here.

• A Pipeline consists of multiple stages, each of which will be run in order.
• A Stage consists of multiple jobs, each of which can run independently of the others.
• A Job consists of multiple tasks, each of which will be run in order.
• A Task is an action that needs to be performed. Usually, it is a single command.
• A Material is a cause for a pipeline to run. This could be a commit made to a source code repository or a timer trigger.
• Agents run the jobs assigned to them.

Now I am moving into the topic. So, what is GoCD API? We can use this API to access GoCD API endpoints through which we can get information about the server’s configuration and build history. In addition to that, it can be used to update configuration and execute builds.

Access the API

GoCD API can be accessed from https://go-server-url:8154/go/api. All data can be sent and received as JSON, specifically application/vnd.go.cd.v1+json .

All APIs require you to authenticate yourself using your username and password.

For example, say we want to access metadata about a pipeline called “my_pipeline” that we created in GoCD. This is a very easy task with GoCD api. You have to execute the following curl call.

$ curl 'https://go-server-url:8154/go/api/admin/pipelines /my_pipeline' \
      -u 'username:password' \
      -H 'Accept: application/vnd.go.cd.v4+json' \
      -i \
      -o gocd_data.json

Here you have to pass the GoCD user’s username and password which you use to log in to GoCD. And that’s all. This then returns a JSON output as follows. It is saved into a file called ‘gocd_data.json’.

HTTP/1.1 200 OK
Content-Type: application/vnd.go.cd.v4+json; charset=utf-8
ETag: "e064ca0fe5d8a39602e19666454b8d77"
{
  "_links": {
    "self": {
      "href": "https://go-server-url:8154/go/api/admin/pipelines/my_pipeline"
    },
    "doc": {
      "href": "https://api.gocd.io/#pipeline-config"
    },
    "find": {
      "href": "https://go-server-url:8154/go/api/admin/pipelines/:name"
    }
  },
  "label_template": "${COUNT}",
  "enable_pipeline_locking": false,
  "name": "my_pipeline",
  "template": null,
  "origin": {
    "type": "local",
    "file": "cruise-config.xml"
  },
  "params": [],
  "environment_variables": [],
  "materials": [
    {
      "type": "git",
      "attributes": {
        "url": "git@github.com:example/sample_repo.git",
        "destination": "code",
        "filter": {
            "ignore": [
                    "**/*.*",
                    "**/*.html"
            ]
        },
        "invert_filter": false,
        "name": "git",
        "auto_update": true,
        "branch": "master",
        "submodule_folder": null,
        "shallow_clone": true
      }
    }
  ],
  "stages": [
    {
      "name": "my_stage",
      "fetch_materials": true,
      "clean_working_directory": false,
      "never_cleanup_artifacts": false,
      "approval": {
        "type": "success",
        "authorization": {
          "roles": [],
          "users": []
        }
      },
      "environment_variables": [],
      "jobs": [
        {
          "name": "my_job",
          "run_instance_count": null,
          "timeout": 0,
          "environment_variables": [],
          "resources": [
                "Linux",
                "Java"
          ],
          "tasks": [
            {
              "type": "exec",
              "attributes": {
                "run_if": [
                  "passed"
                ],
                "on_cancel": {
                  "type": "exec",
                  "attributes": {
                    "command": "ls",
                    "working_directory": null
                  }
                },
                "command": "sleep",
                "arguments": [
                  "10"
                ],
                "working_directory": null
              }
            }
         ],
          "tabs": [
            {
              "name": "cobertura",
              "path": "target/site/cobertura/*.xml"
            }
          ],
          "artifacts": [
            {
              "source": "target",
              "destination": "result",
              "type": "build"
            },
            {
              "source": "test",
              "destination": "res1",
              "type": "test"
            }
          ],
          "properties": null
        }
      ]
    }
  ],
  "tracking_tool": null,
  "timer": null
}

Extract data

To extract data from this JSON output, I am using jq, which is a very high-level functional programming language with support for backtracking and managing streams of JSON data. It is a lightweight and flexible command-line JSON processor. It is like sed for JSON data. We can use it to slice, filter, map and transform structured data with the same ease that sed, awk, grep provide.

For example, say we want to extract the set of environment variables that belong to job1 in stage1. This can be achieved with the following bash script.

stage="stage1"
job="job1"
env_variable_values=$(jq --arg stage "$stage" --arg job "$job" '.stages[] | select(.name==$stage) | .jobs[] | select(.name==$job) | .environment_variables[] | .value' gocd_data.json)

Hope this had been useful for you. :)