KubeSeal Encrypts Secret in GKE

Published in

輕鬆小品-k8s的點滴

Mar 27, 2023

最近測試通過CI/CD來新版的Apigee hybrid，發現安裝方法裡有用到一個新工具: Kubeseal。這個工具的產生是肇因於K8S Secret 是一個base64的Hash值，如果將Secret一起存放到Source Repository裏，無疑是將帳密或是Service Key就在網路上裸奔了。kubeseal這個工具就是可以通過kubeseal Controller將原始的K8S Secret YAML File加密後，以加密形式推送到Source Repository中，當這個加密YAML被Apply到叢集中後，對應的Controller就會自動將其解密，並產生跟原始Secret File一樣的K8S secret在對應的Namespace中。

Seal on a blue container, Generated by Imagen.

爬了幾篇網路上的文件發現都是翻譯自一篇2022–06的文件，該文件加密的命令是通過kubectl port-forward機制連接對應kubeseal controller服務，但測試都一直不成功。

kubeseal --secret-file ./secret.yaml --sealed-secret-file sealed-secret-example.yaml

error: cannot fetch certificate: error trying to reach service: dial tcp 192.168.0.53:8080: i/o timeout

後來看了一下verbose message, 發現kubeseal CLI應該會通過K8S API去找尋kubeseal Controller的服務(sealed-secrets-controller)的Endpoint, 然後跟EndPoint 去取得cert.pem來對Secret.yaml來加密。

如果你跟我遇到一樣的問題，簡單的修正方法，是要加上一條防火牆Rule，允許GKE的control plane(master_ip_cidr) 來access kubeseal這個服務的8080 port，或是將sealed-secrets-controller的這個K8S服務由ClusterIP修改為LoadBalancer，就可以成功了。

kubeseal -f secret.yaml -w seal-secret.yaml
cat seal-secret.yaml

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  creationTimestamp: null
  name: secret-example
  namespace: default
spec:
  encryptedData:
    secret: AgBKhUpl26FK9IorzepgZr74tjpxb6K7RUADXYVhJY7HCz/l7tnAG6fWWXCqjlVaBHZ9s9CbfHobplbefE4SJx0aJLtkSvsqmXuPANS6AA4oHK/AFY4iAhO88AzBpJA8cg2P+ylPnRGGE2GEvoJ6LJV4Pfng5QuQ2O1d5Vam/uft5g2upSz3x5yNoMqRxTKoxmokL6OOb8U+kkxuyDNzQoTXXsLJYg/+f4hgNTG6PZ137pNrwuOsRmYAvoPUVBB7uRsiZOPP55RNGNPRwH9o6K5k2n7NSgYJskFM5tY3QWubIMiunCTWhFvS1gLsgUIjE4chitXRbp0dIV96PMEffXAFn1ws26OJcMQNUAqW+kx0Ez8SZD5PvFvyktIPgnUEUdS4QGBhwXrAopU8LOB0VKnGYUXZ5k6qPDchf0OHs75ljrSIr3FIKm//RD/k0Bd9vjkX4ocbZZ0AHp28eZjjW4Yo9KfZAAA3SmN7cwCOOhBNRvWN+HnrQN7yfdENHDdIdisks0W8o4vdWmwVYKVsJWFxltyistwYvGUk0Ici7W4WEixPn+rwOgR++nRK7n94qFxDbVJTjgdBfY4VewNDHyCCArM6SWMxiDNAx87lv2e7wl4GP6xn+cxng5QwXAbPDUBUqCOO//88WUlT4R5Q1VB54s9hpf/n8w+UKr7RAM9nd1JqsfLHp95dL05NxMqA8dPiBt+XY/DYpsG+5aQ1lA==
  template:
    metadata:
      creationTimestamp: null
      name: secret-example
      namespace: default

通過這個方法也可以直接拿取sealed-secrets-controller服務的cert.pem後，就直接由kubeseal CLI加密對應的secret.yaml。

kubeseal --fetch-cert > cert.pem
kubeseal -f secret.yaml --cert ./cert.pem 
{
  "kind": "SealedSecret",
  "apiVersion": "bitnami.com/v1alpha1",
  "metadata": {
    "name": "secret-example",
    "namespace": "default",
    "creationTimestamp": null
  },
  "spec": {
    "template": {
      "metadata": {
        "name": "secret-example",
        "namespace": "default",
        "creationTimestamp": null
      }
    },
    "encryptedData": {
      "secret": "AgDedBGBODoSxWnl0Ty4VhzCaKeXAK2EantbJbTh6EqgjtHVd3Zj+sd/vf25Tmf9CjqgfD8CpXQOXMx7WerJNBLGphQW1FHkSQ1ggV3Xq+/03iQvI1uOtxAnP5LYezHJoWS/7SZU4UOrAhdp0aA6bocmVsZV+IjP8gEfhP5t3572nvJydZbaX0Kwc6jBwpQrlEbdA+ccI7RFJ8ynetWaxOTa/N7Qi+fLUtP+rKItoKfPePCImAZQkuLsORcw6wQ4lfKxqiFLaGRcnqu/w5VEywXkVdG7AhYwwtd6TKl4JlX0IbO6YA0eTqUf+sKnBG/d1lS/H5wAvMSBLq2wa7QW9JH0KAt5y5ptQUDeV+eCuVT2c+9CtC1wM600U1sqauoCBaKcagiaxoWoPbq3MrG+Dh8B40sGWIQ9SeC4HfjdZxF0dGlhx4Ilg/KkEXLaRU4FuW9ii6iIDdmO/mXQzyVayHN5RScsbPyGbttFWhbjNQ8c5Do2ufYcAYoqwC6GEIanR/hTZ2UDTjV1Fz4TZfv84CLpLq1XlxEbPbyH0tqhiPMJbYhaUA//5kXpaE0eXxNHVvNBY3sBxk/3TD7eEilMmo22Sodc2GTOKvwG38d1hSpHCa1NtlR+77l5TgCxtUjLr8xmXNgEhmgqjJvv8oz4pkhcSBSJaVfQh42osgbQtmNQ3fuNwvZ9nNvm6evbOx1OONjs/K5uKmRQSVC35kFWew=="
    }
  }
}

KubeSeal Encrypts Secret in GKE

Written by Shawn Ho