vmacwrites
Published in

vmacwrites

Delete a bucket that is not empty

Remove a Cloud Object Storage (COS) bucket that is not empty. Use a Terraform script to recursively delete all the objects of a Cloud Object Storage (COS) bucket using MinIO client.

I am working on an use-case where I use Cloud Object Storage(COS) bucket to store a lot of files. The cloud service provisioning for the use-case happens using Terraform scripts.The tough part was to delete the COS bucket using terraform destroy . The destroy fails if the bucket it not empty.

Use this workaround if and only if your terraform resource doesn't support force-delete=true attribute or if don’t want to use the attribute.

Photo by Jeremy McKnight on Unsplash

So, I started looking for options to use with Terraform and got reminded of the Opensource MinIO client.

MinIO is a High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storage service. Use MinIO to build high performance infrastructure for machine learning, analytics and application data workloads.

But, the challenge here is I want to use MinIO client only with terraform destroy . Here comes the Terraform Destroy provisioner to the rescue

If when = destroy is specified, the provisioner will run when the resource it is defined within is destroyed.

resource "cos_instance" "cos" {
# ...
provisioner "local-exec" {
when = destroy
command = "echo 'Destroy-time provisioner'"
}
}

Destroy provisioners are run before the resource is destroyed. If they fail, Terraform will error and rerun the provisioners again on the next terraform apply. Due to this behavior, care should be taken for destroy provisioners to be safe to run multiple times.

This is awesome. I thought I am all set and ready to rock-n-roll. But, then I hit a roadblock — You can’t set environment variables in a Destroy provisioner and MinIO expects HMAC credentials and bucket name.

To see how I cracked this, check the main.tf file in the GitHub repository

  1. Clone the repository to deploy COS
 git clone https://github.com/VidyasagarMSC/cos-object-cleanup.git

2. Create terraform.tfvars file from the template and update the file with your details

cp terraform.tfvars.template terraform.tfvars

3. Run both terraform commands

terraform init terraform apply

Destroy

terraform destroy

By now, you should see that I have used anull_resourcewith triggers to set the required environment variables for the shell script. Alternatively, you can also use local_file to generate a shell script on-the-fly in Terraform.

If the workaround worked for you, don’t forget to drop a 🌟on the GitHub repo and 👏 for this article.

If you see an error, feel free to open an issue in the GitHub repository.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Vidyasagar Machupalli

Vidyasagar Machupalli

Developer, IBMer, Speaker, Blogger, Teetotaller, Geek & many more…