在現在大混雲/跨雲時代,越來越多夥伴需要在私有機房環境建立 k8s 集群,Google Cloud 提供了一套企業級的 K8s — Anthos,除了簡化安裝程序,還提供許多方便管理的功能,讓管理 k8s 集群越來越容易。
如果手上沒有機器但又想要玩玩看該怎麼辦呢?Google Cloud 非常貼心的提供腳本讓你透過 GCE 玩玩看,但請注意請勿用於正式環境。
這次我會拿 Terraform + Hybrid deployment 的腳本來改造一下:
安裝後預設會提供以下架構:
其中 VxLAN 提供的 L2 連線至關重要,如果用 GCE 的 VPC 會失敗
開始安裝
1.下載範例並切換目錄:
git clone git@github.com:GoogleCloudPlatform/anthos-samples.git
cd anthos-samples/anthos-bm-gcp-terraform
2. 在 Google Cloud Project 並產生 Service Account 給予足夠權限並取得憑證
PROJECT_ID="gcpsa-sandbox" #專案 ID
SA_NAME="feng-anthos-bm" # Service Account 名稱
gcloud iam service-accounts create $SA_NAME --project=$PROJECT_ID
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/owner"
gcloud iam service-accounts keys create sa-key.json \
--iam-account="$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com"
3. 修改 terraform.tfvars.sample 後改名為 terraform.tfvars
project_id = "gcpsa-sandbox"
region = "asia-east1"
zone = "asia-east1-a"
credentials_file = "sa-key.json"
abm_cluster_id = "cluster1"
anthos_service_account_name = "baremetal-gcr"
resources_path = "./resources"
machine_type = "n1-standard-8"
instance_count = {
"controlplane" : 1
"worker" : 2
}
...
我修改了 project_id , region , zone , credentials_file,另外一次開六台有點太多的話可以考慮把 controlplane 改為 1 台,worker 維持 2 台
題外話:最近剛好 C3 剛好在 Preview,而且是免費,只收硬碟的費用,超佛!想用的話可以改參數:
- machine_type = c3-highcpu-8
- region = us-central1
- zone = us-central1-a
- min_cpu_platform =
重新命名: mv terraform.tfvars.sample terraform.tfvars
4. Terrafrom 起手式三套 init , plan , apply
如果 Terraform 太新(超過 1.4)的話會沒辦法裝,可以用 tfswicher 切換版本至 1.3.9
terraform init
terraform plan
terraform apply
最後你會看到下一步的輸出
Outputs:
abm_version = "1.14.1"
admin_vm_ssh = <<EOT
################################################################################
## AnthosBM on Google Compute Engine VM with Terraform ##
## (Run the following commands) ##
## (Note that the 1st command should have you SSH'ed into the admin host) ##
################################################################################
> gcloud compute ssh tfadmin@cluster1-abm-ws0-001 --project=gcpsa-sandbox --zone=asia-east1-a
# ------------------------------------------------------------------------------
# You must be SSH'ed into the admin host cluster1-abm-ws0-001 as tfadmin user now
# ------------------------------------------------------------------------------
> export GOOGLE_APPLICATION_CREDENTIALS=/home/tfadmin/terraform-sa.json && \
sudo -E ./run_initialization_checks.sh && \
sudo -E bmctl create config -c cluster1 && \
sudo -E cp ~/cluster1.yaml bmctl-workspace/cluster1 && \
sudo -E bmctl create cluster -c cluster1
################################################################################
5. 部署集群
照著上一步輸出跑~
# 連線到 work stataion
gcloud compute ssh tfadmin@cluster1-abm-ws0-001 --project=gcpsa-sandbox --zone=asia-east1-a
# 執行指令
export GOOGLE_APPLICATION_CREDENTIALS=/home/tfadmin/terraform-sa.json && \
sudo -E ./run_initialization_checks.sh && \
sudo -E bmctl create config -c cluster1 && \
sudo -E cp ~/cluster1.yaml bmctl-workspace/cluster1 && \
sudo -E bmctl create cluster -c cluster1
等一段時間後就完成嚕~
接下來設定要操作 Cluster 可以先設定 KUBECONFIG 環境變數,這樣就不用每次都帶 kubeconfig 路徑
export CLUSTER_ID=cluster1
export KUBECONFIG=$HOME/bmctl-workspace/$CLUSTER_ID/$CLUSTER_ID-kubeconfig
驗證一下
tfadmin@cluster1-abm-ws0-001:~$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
cluster1-abm-cp1-001 Ready control-plane,master 4h13m v1.25.5-gke.1001
cluster1-abm-w1-001 Ready worker 4h10m v1.25.5-gke.1001
cluster1-abm-w2-001 Ready worker 4h10m v1.25.5-gke.1001
另外可以到 Console 看一下狀況
可以先忽略 error 部分,初始安裝好還沒設定 Connect Gateway,所以還沒辦法用當前帳號看到詳細資訊,不過已經可以看到 Cluster1 的資源為 16 CPU , 63.09 GB Ram,之後有機會再繼續展示其他 Anthos 功能~
刪除集群
我按照說明跑發現噴錯誤QQ,我自己先猜可能跟 GCE 綁定的 Service Account 有關
sudo bmctl reset --cluster $CLUSTER_ID
* ClusterOperations check failed: invalid Cluster Operations location: Get "https://compute.googleapis.com/compute/v1/projects/gcpsa-sandbox/zones?alt=json&prettyPrint=false": metadata: GCE metadata "instance/service-accounts/default/token?scopes=https%3A%2F%2Fwww.googleapis.com%2Fauth%2f Cloud-platform" not defined
所以我的做法是先回到自己的終端機刪除 Infra
terraform destroy --auto-approve
再檢查一下 Console 會發現還有 admin-bm-site1,那就手動 DISCONNECT 吧