Migrating GCP datastore across different projects
Currently i am developing a side project called Packet Distribution. I decided to use GCP datastore as a database. In GCP we can organize the needs (including the datastore) of our project by creating GCP project . Unfortunately i forgot to create new GCP project for the Packet Distribution, the data for Packet Distribution is already in that old GCP project and also, i have deployed the service in it. So i try to create new GCP project and find a way to migrate the data from the previous GCP project into the new one.
In nutshell, here are the step that need to migrate the datastore into others project :
- Let say we have project 1 and project 2 and want to migrate datastore from project 1 to project 2
- Export datastore data into the bucket project 1.
- Give project 2 access to the project 1 bucket.
- In project 2, use Storage Transfer Service to migrate the data.
- In project 2, from datastore, import the exported data from project 1 bucket.
Export datastore data into the bucket project 1
As you can see here in datastore menu, we have 3 data. These are the data that we are trying to migrate into project 2. Click import/export menu and then click export.
After that you will be prompted the export options, you need to fill some field here. The important things here is the bucket it self, if you don’t have a bucket, you can create it, but the default bucket should be ok. For me i use the default bucket first-fortress-356512.appspot.com. Click export.
Give project 2 access to the project 1 bucket
In order to give access, we need to know project 2 service account. In project 2, Let’s navigate to IAM and choose service accounts menu. If you don’t have service account you can create it first.
In project 1, navigate to cloud storage, edit access for the bucket and add principal. copy paste the project 2 service account and give storage admin role. Now project 2 will have access to project 1 cloud bucket.
In project 2, use Storage Transfer Service to migrate the data
in data transfer menu, click create transfer job. for the source type and destination type we leave it as the default value since we are going to migrate the data from GCP project.
Then we need to input the source bucket and the destination bucket. Then create the transfer jobs.
In project 2, import the exported data from project 1 bucket
after the transfer job is finished, we can import the exported data by import it from the bucket.
We need to choose this file in the bucket. after that we import the data.
Finally we will see the data in project 2 datastore.
