Hashicorp Vault: Part 3 SHORT-TERM ACCESS TO GOOGLE CLOUD SECRET ENGINE WITH GOLANG

VAULT SHORT-TERM ACCESS TO GOOGLE CLOUD SECRET ENGINE WITH GOLANG

Antakya — 2021 Sabun Çiçeği

In this blog I will show you how to use Hashicorp Vault to generate and save GCP OAuth token using Golang. Firstly, we will obtain a specific service account for GCP credentials. Secondly, we will configure the GCP auth backend in Vault using a JSON file that contains the GCP credentials. Then, we will create a token role using a roleset JSON file. Finally, we will use the token to start and stop an instance in GCP using the Compute API.

note: *WIL is the personal notes I took as a result of my research and that I want to share. For more detailed information, you should visit Hashicorp Vault’s own site.

I will explain and show some examples in three parts about Hashicorp Vault.

1. General explanation about Hashicorp Vault
2. General Secret Storage (with example code)
3. Short-term access to Google Cloud Secret Engine (this blog)

To begin, we need to create a specific service account for Vault to use before writing the code.

• Go to the Google Cloud Console
• Hamburger Menu -> IAM&Admin -> Service Accounts

• Click the Create Service Account

• Choose any name for the Service Account. It’s not important what you choose, but it should be easy to understand. Once you’ve selected a name, click on the “Create” button.

• We need to assign some roles to this account. To do this, we will create a Vault user other than the admin and grant this user only the roles that we specify. The roles that we will allow for this service are:

1. Service Account Key Admin
2. Service Account Token Creator (It will be used to create Auth tokens that we can use for 3rd party software.)
3. Security Reviewer
4. Security Admin
5. Service Account Admin

• Once we have completed the previous steps, we can proceed to create a new service account by clicking “Done” without making any additional customizations.

• Next, we need to select “Create Key” by clicking on the three dots on the right-hand side of the Service Account. This will allow us to create our credential file in the JSON format, as shown in the example below.

We now need to enable two APIs to make the service account accessible. The first one is the IAM API, which allows administrators to manage access control to their GCP resources. This enables them to set up and manage secure and fine-grained access policies for their organization.

• To activate the IAM API, search for it in the search bar and enable it.

The second API that we need to enable is the Cloud Resource Manager API. This is a service in Google Cloud Platform that helps you manage your cloud resources by providing a unified view of all your resources and helping you organize and manage them in a structured way. We will use this API to determine and assign the roleset of Vault.

• To activate the Cloud Resource Manager API, search for it in the search bar and enable it.

In HashiCorp Vault, a roleset is used to define the relationship between the entities that will access the secrets and the secrets themselves. This allows us to define the access controls and permissions for each entity based on their roles.

• We need to add the following JSON file to our code to use the Cloud Resource Manager API. This file will determine the roleset that Vault can have.

In the context of GCP secret vault, the roleset you provided is used to define the following:

• “project”: the GCP project ID that this roleset applies to.
• “secret_type”: the type of secret that this roleset applies to. In this case, it’s an access token.
• “token_scopes”: the list of scopes required by the entity to access the secret.
• “bindings”: the GCP IAM policy bindings that define the permissions for the entities. In this case, the resource is the GCP project identified by its ID, and the role assigned is “roles/editor”, which gives the entity access to edit resources in the project.

We have added the Vault user to Google Cloud Platform. We have activated the necessary permissions and packages to the Vault user. After that, it remains to open the Vault server and store our valuable information. Let’s start the server temporarily.

vault server -dev

When the server starts in -dev mod, the warning will appear below. The address registered in the environment variables should be replaced with http, as we will not be using the vault address currently certified. In addition, since we will send data to the Vault with the help of Golang from the outside, a security token must be found. That’s why we save the Root Token somewhere to use as well.

WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.

You may need to set the following environment variables:

PowerShell:
    $env:VAULT_ADDR="http://127.0.0.1:8200"
cmd.exe:
    set VAULT_ADDR=http://127.0.0.1:8200

The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.

Unseal Key: ZHU7Znx8ocIsG5HJ6dZtNxbHSt1qZezuxDoID61kMAE=
Root Token: hvs.dmpAuR3a0sXLC6HZwdjQDzrw

Development mode should NOT be used in production installations!

After a long adjustment, now let’s come to the most fun part that we all love. Now it’s time to code!

This function sets environment variables for the Vault address and token. First, it sets the VAULT_ADDR environment variable to the local Vault address of “http://127.0.0.1:8200". Then, it sets the VAULT_TOKEN environment variable to the token for the current Vault session.

func setEnv() error {
    // Set environment variables for Vault address and token
    if err := os.Setenv("VAULT_ADDR", "http://127.0.0.1:8200"); err != nil {
        log.Printf("Cannot set VAULT_ADDR")
        return err
    }
    if err := os.Setenv("VAULT_TOKEN", "hvs.R6hbTiWnHWDe9FuvTKgJ8Iew"); err != nil {
        log.Printf("Cannot set VAULT_TOKEN")
        return err
    }
    return nil
}

This function creates and returns a new client for connecting to Vault, a popular tool for managing secrets and sensitive information. It first sets up a configuration for the client by using the vault.DefaultConfig() function and setting the Vault server address to an environment variable VAULT_ADDR. The function then creates a new client using the configured settings and returns it. If there are any errors during the configuration or creation process, the function returns an error along with the client.

func createVaultClient() (*vault.Client, error) {
    // Configure Vault client with environment variables
    config := vault.DefaultConfig()
    config.Address = os.Getenv("VAULT_ADDR")

    // Create a new Vault client
    return vault.NewClient(config)
}

This function creates a mount point for the Google Cloud Platform secret engine using the provided project ID and Vault client, and returns an error if the mount point cannot be created.

func createMountPoint(projectID string, vaultclient *vault.Client) error {
    // Create a mount point for the GCP secret engine
    path := "gcp/" + projectID
    mountInput := &vault.MountInput{
        Type: "gcp",
    }
    if err := vaultclient.Sys().Mount(path, mountInput); err != nil {
        log.Fatalf("Cannot create mount point: %v", err)
        return err
    }
    return nil
}

This code reads the contents of a JSON file containing GCP credentials, writes the contents to a Vault secret engine at a specific path based on the project ID, and returns an error if any issues occur during the write process.

func writeConfig(projectID string, vaultclient *vault.Client) error {
    contents, err := ioutil.ReadFile("csm-cred.json")
    if err != nil {
        log.Fatalf("Cannot read GCP credentials: %v", err)
        return err
    }
    data := map[string]interface{}{"credentials": string(contents[:])}
    path := "gcp/" + projectID + "/config"
    _, err = vaultclient.Logical().Write(path, data)

    if err != nil {
        log.Fatalf("Cannot write GCP credentials: %v", err)
        return err
    }
    return nil
}

This code reads a JSON file containing data for a GCP roleset, unmarshals the data into a map, and then writes the roleset to a specific path in Vault’s GCP secret engine for the specified project ID using the provided Vault client.

func writeRoleset(projectID string, vaultclient *vault.Client) error {
    var data map[string]interface{}

    //read JSON file
    bytes, err := ioutil.ReadFile("roleset.json")
    if err != nil {
        log.Fatalf("Cannot read roleset: %v", err)
        return err
    }

    // Unmarshal the JSON data into a map
    err = json.Unmarshal(bytes, &data)
    if err != nil {
        log.Fatalf("Cannot unmarshal roleset: %v", err)
        return err
    }

    path := "gcp/" + projectID + "/roleset/my-token-roleset"
    _, err = vaultclient.Logical().Write(path, data)
    if err != nil {
        log.Fatalf("Cannot write roleset: %v", err)
        return err
    }

    fmt.Printf("roleset data: ", data)
    return nil
}

This function generates a token for the given projectID by reading the generated token from the specified path in Vault after waiting for 8 seconds. Its waiting because the function which before this is saving the secret value into vault and it takes a little time.

func generateToken(projectID string, vaultclient *vault.Client) (string, error) {
 // Wait for Vault to generate a token for the roleset
 time.Sleep(8 * time.Second)

 // Read the generated token from Vault
 pathReq := "gcp/" + projectID + "/roleset/my-token-roleset/token"
 secret, err := vaultclient.Logical().Read(pathReq)
 if err != nil {
  log.Fatalf("Cannot read token: %v", err)
  return "", err
 }

 // Extract the token from the secret
 token := secret.Data["token"].(string)
 return token, nil
}

This code sends an HTTP POST request to stop a Google Cloud Platform virtual machine named “test-csm-vm” in the “us-central1-a” zone of the “csm-pro” project using a bearer token for authentication, and then logs the response. This demonstrates to show the vault is running correctly.

func stopVM(token string) error {
 // Define the request to stop the VM
 url := "https://compute.googleapis.com/compute/v1/projects/csm-pro/zones/us-central1-a/instances/test-csm-vm/stop"
 req, _ := http.NewRequest("POST", url, nil)
 req.Header.Add("Authorization", "Bearer "+token)

 // Send the request
 res, err := http.DefaultClient.Do(req)
 if err != nil {
  log.Fatalf("Cannot do request: %v", err)
  return err
 }
 defer res.Body.Close()

 // Print the response
 fmt.Println(res)

 return nil
}

Once we have completed the previous steps, we can proceed to the main code, which will call all of the other functions. This code will configure the GCP auth backend in Vault using the JSON file containing the GCP credentials, create a token role using the roleset JSON file, and use the token to start and stop an instance in GCP using the Compute API.

func main() {
 // Initialize variables
 projectID := "222ddx6"

 // Set environment variables for Vault
 if err := setEnv(); err != nil {
  log.Fatalf("Cannot set environment variables: %v", err)
 }

 // Create a new Vault client
 vaultclient, err := vault.NewClient(vault.DefaultConfig())
 if err != nil {
  log.Fatalf("Cannot create Vault client: %v", err)
 }
 defer vaultclient.ClearToken()

 // Create a mount point for the GCP secret engine
 path := "gcp/" + projectID
 mountInput := &vault.MountInput{
  Type: "gcp",
 }
 if err := vaultclient.Sys().Mount(path, mountInput); err != nil {
  log.Fatalf("projectID already exists: %v", err)
 }

 // Write GCP credentials to Vault
 if err := writeConfig(projectID, vaultclient); err != nil {
  log.Fatalf("Cannot write GCP credentials: %v", err)
 }

 // Write roleset data to Vault
 if err := writeRoleset(projectID, vaultclient); err != nil {
  log.Fatalf("Cannot write roleset: %v", err)
 }

 // Generate a token for the roleset
 token, err := generateToken(projectID, vaultclient)
 if err != nil {
  log.Fatalf("Cannot generate token: %v", err)
 }

 // Stop the VM
 if err := stopVM(token); err != nil {
  log.Fatalf("Cannot stop VM: %v", err)
 }

This concludes my Medium article series on my experience working with and learning about Hashicorp Vault. Thank you so much for reading and for your patience thus far.

You can find the complete code that I’ve written on my GitHub account.
https://github.com/pawntoqueen/HCVault_GCP_Secret_Engine

I hope that my studies have been helpful to you, and that you can apply what you’ve learned to your own work with Vault. If you have any questions or comments, please feel free to reach out to me.

Thank you again for your time and attention, and I wish you all the best in your future!