Trendyol Tech
Published in

Trendyol Tech

Photo by Jason Pofahl on Unsplash

Manage Kubernetes Admission Webhook's certificates with cert-manager CA Injector and Vault PKI📝 🔐⛵️

📦 Table of Contents

Kubernetes Admission Controllers ⛵️

cert-manager and CA Injector 📝

Vault PKI (Public Key Infrastructure) 🔐

Installation 💻

$ minikube start -p demo
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
vault-0 1/1 Running 0 32s
$ kubectl exec vault-0 -- vault login rootSuccess! You are now authenticated. The token information displayed below is already stored in the token helper. You do NOT need to run “vault login” again. Future Vault requests will automatically use this token.Key Value
— — — — -
token root
token_accessor cDLx7PbVXcY3ibzweBBgki0h
token_duration ∞
token_renewable false
token_policies [“root”]
identity_policies []
policies [“root”]
$ kubectl exec -ti vault-0 -- /bin/sh
/ $
https://gist.github.com/developer-guy/0b128945dbc14f6bdd6009d6f648d4f3
https://gist.github.com/developer-guy/d5cfd97f781b3a1f0812544a4ee99560
https://gist.github.com/developer-guy/d544ae1f299c74cc1baa738c0a853719
$ kubectl get pods -n cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-57d89b9548-wmlrl 1/1 Running 0 9m39s
cert-manager-cainjector-5bcf77b697-rsbh9 1/1 Running 0 9m39s
cert-manager-webhook-8687fc66d4–9hfvq 1/1 Running 0 9m39s
$ kubectl create serviceaccount issuer -n platform$ ISSUER_SECRET_REF=$(kubectl get serviceaccount issuer -n platform -o json | jq -r ".secrets[].name"); echo $ISSUER_SECRET_REF$ issuer-token-7z8jj
https://gist.github.com/developer-guy/ad2093fc5b78e5fdbf399a03ea8062df
https://gist.github.com/developer-guy/829035a6df9b22c97e401dbdccc0328b
$ kubectl get certificates -n platform config-sidecar-injector-service
NAME READY SECRET AGE
config-sidecar-injector-service True config-admission-webhook-tls 5s

$ kubectl view-secret config-admission-webhook-tls -n platform tls.crt | cfssl certinfo -cert -
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
annotations:
cert-manager.io/inject-ca-from: platform/config-sidecar-injector-service
$ yq e ".webhooks[0].clientConfig.caBundle" <(k get mutatingwebhookconfigurations.admissionregistration.k8s.io config-sidecar-injector -oyaml | k neat) | base64 -D | cfssl certinfo -cert -{
“subject”: {
“common_name”: “config-sidecar-injector-service.platform.svc”,
“names”: [
“config-sidecar-injector-service.platform.svc”
]
},
“issuer”: {
“common_name”: “config-sidecar-injector-service.platform.svc”,
“names”: [
“config-sidecar-injector-service.platform.svc”
]
},
“serial_number”: “568486349258568295357093419160671126608237098261”,
“sans”: [
“config-sidecar-injector-service”,
“config-sidecar-injector-service.platform”,
“config-sidecar-injector-service.platform.svc”
],
“not_before”: “2022–01–04T18:40:41Z”,
“not_after”: “2023–01–04T18:41:10Z”,
“sigalg”: “SHA256WithRSA”,
“authority_key_id”: “4D:13:A6:D2:85:CF:36:98:FD:65:3E:F5:27:A5:38:EF:40:71:90:3E”,
“subject_key_id”: “4D:13:A6:D2:85:CF:36:98:FD:65:3E:F5:27:A5:38:EF:40:71:90:3E”,
“pem”: “ — — -BEGIN CERTIFICATE — — -\nMIID5jCCAs6gAwIBAgIUY5PPPi69t64ZpuNVWFRuPu6o9RUwDQYJKoZIhvcNAQEL\nBQAwNzE1MDMGA1UEAxMsY29uZmlnLXNpZGVjYXItaW5qZWN0b3Itc2VydmljZS5w\nbGF0Zm9ybS5zdmMwHhcNMjIwMTA0MTg0MDQxWhcNMjMwMTA0MTg0MTEwWjA3MTUw\nMwYDVQQDEyxjb25maWctc2lkZWNhci1pbmplY3Rvci1zZXJ2aWNlLnBsYXRmb3Jt\nLnN2YzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL4B3h/Ps2xyumsd\nxkm+Qnl5ZFcRWc0AmEFQRxDNS0z7T/MSjSNoFb+TbuE2hhmbxkFPA45/dUotxp9i\n6ZNMglirzwaxAyI+8MRGkKRoHKqNN/gj8MC9aUqhy38CImbl2AYiGD0jPx/GTj45\nyimUIr3QUTaU9TCQCSigjTzOnG4FIkEp35CPDJg5KM0exD7ItE8TdabwIYwI5BZp\n7o1eJjoOUHf9PufZcgBY0mxaMYVwfKuKz1dq/e/34qGniFduZe0XPMBTUKQxuH6U\nR8gWlKTOl7i+AKT/uATDX22I1TTeKXf6ymGvn4jz52+dY/DfKxlxk88U70XiGAkv\ne+2du38CAwEAAaOB6TCB5jAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB\n/zAdBgNVHQ4EFgQUTROm0oXPNpj9ZT71J6U470BxkD4wHwYDVR0jBBgwFoAUTROm\n0oXPNpj9ZT71J6U470BxkD4wgYIGA1UdEQR7MHmCH2NvbmZpZy1zaWRlY2FyLWlu\namVjdG9yLXNlcnZpY2WCKGNvbmZpZy1zaWRlY2FyLWluamVjdG9yLXNlcnZpY2Uu\ncGxhdGZvcm2CLGNvbmZpZy1zaWRlY2FyLWluamVjdG9yLXNlcnZpY2UucGxhdGZv\ncm0uc3ZjMA0GCSqGSIb3DQEBCwUAA4IBAQBrKZxo1tPLdwn0mZN64mya8P6DsUnf\nsW2rnetsjc5kJYTV6p/8iov/yQPmsnN9bIZSc87wOTa6QF1fL0jlhWVvUz+9DZPv\nwfdYPvv31KQj+9WNquiaXKr/uELTmIYXeD1/ckJx9ZLE0WjUnfMkRxGxsIiB9JYu\n3WzOIqQV5czS2UubrKsvvGmtPpdfK7JJsWyk9Z4Hga78SEPNmErayYk3zjEB5rMK\n6+bVQIP00P/h89iwwjdBcL7DQDdociKQznL/L2Dm2rtUbkVMPi42WAn6xilaGmVJ\n643hBOsPIVBRtI0g2pquGutx00t0kw2LZCS+81rkz7t+9miiy+x7T72c\n — — -END CERTIFICATE — — -\n”
}
$ kubectl apply -f samples/ –dry-run=serverError from server: error when creating “examples/auto/pod1.yaml”: admission webhook “config-sidecar-injector-service.platform.svc” denied the request: could not find configs

How to monitor certificates? 👀

  • PEM encoded files by path or scanning directories
  • Kubeconfigs with embedded certificates or file references
  • TLS Secrets from a Kubernetes cluster
$ helm repo add enix https://charts.enix.io
$ helm install x509-certificate-exporter enix/x509-certificate-exporter
https://enix.io/en/blog/avoiding-certificate-expiration-kubernetes-infrastructure

How to accomplish hot-reloading your HTTP server with renewed certificates without having downtime? ✨

$ kubectl apply -f examples/auto/ --dry-run=serverserviceaccount/auto-sa unchanged (server dry run)Error from server (InternalError): error when creating “examples/auto/pod1.yaml”: Internal error occurred: failed calling webhook “config-sidecar-injector-service.platform.svc”: Post “https://config-sidecar-injector-service.platform.svc:443/mutate?timeout=30s": x509: certificate has expired or is not yet valid: current time 2022–01–04T19:31:03Z is after 2022–01–04T19:25:35Z
https://gist.github.com/developer-guy/de82fb8e97557ec711ae2dd79ac1d029
$ DOM="localhost"$ PORT="8080"$ openssl s_client -servername $DOM -connect $DOM:$PORT | openssl x509 -noout -dates

🎯 Conclusion

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
developer-guy

🇹🇷KCD Turkey Organizer🎖Best Sigstore Evangelist🐦SSCS Twitter Community Admin✍️@chainguard_dev Fan📦Container Addict📅Organizer at @cloudnativetr•@devopstr