GCP Resource Manager with Python V1.2 — Managing folders
Creating, listing and deleting folders under google cloud organization with python
In last 2 blogs of Google cloud resource manager series, i have talked about hierarchal listing projects and folders under google cloud organization and graphical representation of it using python and Graphviz.
In this blog posts i will talk about creating and deleting a folders under google cloud organisation to host the projects using python.
What is a folder in google cloud platform
In the Google Cloud Platform (GCP), organizing resources efficiently is crucial for effective management and control. The hierarchical structure provided by folders enables you to group related projects and resources, making it easier to manage permissions, apply policies, and track costs.
Plan your resource hierarchy carefully, considering factors such as organizational structure, logic grouping of project.
Lets create folders in google cloud with python
Prerequisite — authenticate and access the Google Cloud Resource Manager
Prerequisites for this will be same like previous blog, except the fact that it will need service account to use more permissive roles — Org level permissions to create and delete projects in google cloud.
I will use resourcemanager.organizationAdmin role, but i recommend to use much granular permissions for specific use case in production environment.
resourcemanager.folders.create- Allows creating folders.resourcemanager.folders.delete- Allows deleting folders.resourcemanager.folders.get- Allows retrieving folder information.
# Creating service account
gcloud iam service-accounts create rm-svc \
--description="can list all folders and projects" \
--display-name="resource-manager-walk"
# Assigning roles
gcloud organizations add-iam-policy-binding \
ORGANIZATION_ID --member="serviceAccount:rm-svc@someproject.iam.gserviceaccount.com" \
--role="resourcemanager.organizationAdmin"
# Creating service account key
gcloud iam service-accounts keys create rm-svc.json --iam-account \
rm-svc@someproject.iam.gserviceaccount.com
# impersonating service account
gcloud auth activate-service-account —key-file=rm-svc.json
# Installing python client for google cloud resource manager
pip install google-cloud-resource-managerGoogle Cloud follows a hierarchical structure for organizing resources, which starts at the organization level. Under the organization, you can create folders to group projects and further nest additional folders within them.
Lets print out the folder details, and hierarchy so we know where we want to create the folder, under organization or under a folder. You will need to provide a parent id (folder id or an org id) in order to create the folder.
Lets print out all the folders details under the org — A list with tuple of folder id and folder’s display name.
Lets say org id is ‘12345'
from typing import Dict, List, Union
from google.cloud import resourcemanager_v3
def get_folders(
parent_id: str = "organizations/12345",
folders: Union[List[str], None] = None,
) -> List[str]:
"""
Recursively lists all subfolders under a given parent (folder or organisation).
Args:
parent_id (str): The ID of the parent folder to list subfolders for.
Defaults to "organizations/12345".
folders (Dict[str, Dict[str, str]]): A dictionary to store the subfolders in.
Defaults to an empty dictionary.
Returns:
A dictionary of subfolders under the given parent folder.
Each dictionary key is the folder ID, while the value contains the subfolder's name,
display name and parent.
"""
if folders is None:
folders = []
client = resourcemanager_v3.FoldersClient()
request = resourcemanager_v3.ListFoldersRequest(
parent=parent_id,
)
page_result = client.list_folders(request=request)
for pages in page_result:
folders.append((pages.name, pages.display_name))
get_folders(parent_id=pages.name, folders=folders)
return folders
print(get_folders(
parent_id="organizations/12345"))output will look something like this
[(‘folders/126, ‘my-folder1’), (‘folders/234’, ‘my-folder2’), (‘folders/321’, ‘my-folde4’)]
And also print out the folders hierarchy so we better know where to create folder — Will use the code from last blog post
from typing import Dict, List, Union
from google.cloud import resourcemanager_v3
def projects_in_folder(folder_id: str) -> List[str]:
"""
Retrieves a list of active project names within a specified folder.
Args:
folder_id (str): The ID of the folder.
Returns:
List[str]: A list of active project names within the folder.
"""
client = resourcemanager_v3.ProjectsClient()
query = f'parent.type:folder parent.id:{folder_id}'
request = resourcemanager_v3.SearchProjectsRequest(query=query)
response = client.search_projects(request=request)
projects = []
for project in response:
if project.state == resourcemanager_v3.Project.State.ACTIVE:
projects.append(project.display_name)
return projects
def get_folder_hierarchy(
parent_id: str = "organizations/12345",
hierarchy: Union[Dict[str, Union[str, List[str]]], None] = None,
) -> Dict[str, Union[str, List[str]]]:
"""
Retrieves the folder hierarchy of the Google Cloud resource structure starting from the specified parent ID.
Args:
parent_id (str): The ID of the parent resource to start the hierarchy from.
Defaults to "organizations/12345".
hierarchy (Union[Dict[str, Union[str, List[str]]], None]): The dictionary representing the folder hierarchy.
Defaults to None.
Returns:
Dict[str, Union[str, List[str]]]: A dictionary representing the folder hierarchy of the resource structure.
The keys represent folder names, and the values can either be lists of
project names within a folder or sub-hierarchies (dictionaries).
"""
if hierarchy is None:
hierarchy = {}
client = resourcemanager_v3.FoldersClient()
request = resourcemanager_v3.ListFoldersRequest(parent=parent_id)
response = client.list_folders(request=request)
for folder in response:
folder_id = folder.name.split('/')[-1]
folder_name = folder.display_name
projects = projects_in_folder(folder_id)
if projects:
hierarchy[folder_name] = projects
pass
else:
sub_hierarchy = get_folder_hierarchy(parent_id=folder.name, hierarchy={})
if sub_hierarchy:
hierarchy[folder_name] = sub_hierarchy
return hierarchy
folder_hierarchy_data = get_folder_hierarchy()
print(folder_hierarchy_data)Output will look something like this
{‘my-folder1’: [‘project1’], ‘my-folder2’: [‘project2, ‘project3’], ‘my-folder3’: {‘my-folder4’: [‘project4’]}
Lets create a folder — will use ‘my-folder4’ as parent and its respective folder id, ‘folders/321’ as parent_id
from google.cloud import resourcemanager_v3
def create_folder(folder_name: str, parent_id: str):
"""
Creates a new Google Cloud folder.
Args:
folder_name (str): The name of the folder to be created.
parent_id (str): The parent ID of the folder. It can be an organization ID or a folder ID.
Raises:
google.api_core.exceptions.InvalidArgument: If the provided arguments are invalid.
google.api_core.exceptions.AlreadyExists: If a folder with the same name already exists.
Returns:
str: The ID of the created folder.
"""
client = resourcemanager_v3.FoldersClient()
folder = {
"display_name": folder_name,
"parent": parent_id,
}
create_folder_request = resourcemanager_v3.CreateFolderRequest(
folder=folder
)
response = client.create_folder(request=create_folder_request)
folder_id = response.name.split("/")[-1]
print(f"Created folder: {folder_id}")
return folder_id
if __name__ == '__main__':
folder_name = "Your medium Name"
parent_id = 'folders/321'
# you can also use "organizations/12345" as parent_id
create_folder(folder_name, parent_id)
And your folder is created. You can run the other scripts to verify the folder created and its hierarchy. You can now create new projects under this new folder — will show you how in next blog post.
Cleanup — Deleting the folder
If you need to delete the folder after testing, it can be done easily with below python code.
from google.cloud import resourcemanager_v3
def delete_folder(folder_id: str):
"""
Deletes a Google Cloud folder.
Args:
folder_id (str): The ID of the folder to be deleted.
Raises:
google.api_core.exceptions.NotFound: If the specified folder is not found.
google.api_core.exceptions.PermissionDenied: If the caller does not have the necessary permissions.
Returns:
None
"""
client = resourcemanager_v3.FoldersClient()
folder_name = f"folders/{folder_id}"
delete_folder_request = resourcemanager_v3.DeleteFolderRequest(
name=folder_name
)
client.delete_folder(request=delete_folder_request)
print(f"Deleted folder: {folder_id}")
if __name__ == '__main__':
folder_id = 'folders/321'
delete_folder(folder_id)I hope you enjoyed this blog post. see you on next one.
If you found my content helpful, buy me a coffee to show your support.
Connect with me in Linkedin — @akhilesh-mishra-0ab886124
Follow me for more content on Google Cloud, Terraform, Python and other Devops tools.
