Open in app

Sign in

Write

Sign in

Kubernetes Multi Cluster İçin HAProxy ile Proxy Nasıl Hazırlanır ?

Serdarcan Büyükdereli
Arabam Labs
Published in
7 min readNov 3, 2023

Merhaba! Bu makalede, Kubernetes Multi Cluster’da HAProxy’le High Availability olucak şekilde erişimimizi daha erişilebilir hale getirmek için HAProxy’le bir proxy kurmanın kolay adımlarını keşfedeceğiz.

Hadi başlayalım!

Kubespray ile Cluster nasıl kurulur üzerine yazıya burdan erişebilirsiniz.

Konsept

1- Clusterların Tanımlanması

  • tls verify değinilmesi
  • kube config file tasarımı

2- Keepalived Kurulumu ve keepalived.conf un hazırlanması.

3- HAProxy Kurulumu ve haproxy.cfg’ın hazırlanması.

Farklı domainlerle HAProxy ile istediğimiz cluster a erişeceğiz. Aşağıdaki gibi 3 adet cluster olduğunu düşünelim ;

kube-system namespace’i içerisinde configmap’te kubeadm-config te aşağıdaki gibi bir yapı görebilirsiniz. Kube config file dan cluster’a erişim sağlarken bu domainlerden erişim sağlayabildiğimizi gösterir. Diğer domainlerden erişmeye çalıştığımız zaman aldığımız hata ;

“http: proxy error: x509: certificate is valid for 10.96.0.1, 172.29.217.209, not 107.124.83.31"

apiServer:
  certSANs:
  - kubernetes
  - kubernetes.default
  - kubernetes.default.svc
  - kubernetes.default.svc.example-testing-cluster
  - localhost
  - 127.0.0.1
  - example-test-kube-master01
  - example-test-kube-master02
  - example-test-kube-master03
  - example-test-kube-master04
  - example-test-kube-master05
  - lb-apiserver.kubernetes.local
  - 192.168.10.31
  - 192.168.10.32
  - 192.168.10.33
  - 192.168.10.34
  - 192.168.10.35

How to add a new hostname or IP address to a Standalone Kubernetes API server - Kloudle Website

A short post on adding a new hostname or IP address to the PKI certificate of a Kubernetes cluster so that TLS…

kloudle.com

Çözüm olarak bu kubeadm ile sertifikaya eklenerek halledilebilir.

Ayrıca Cluster üstünde herhangi bir değişiklik config file da --insecure-skip-tls-verify şeklinde de çözüm yapılabilir. Ben bu makalede cluster üzerinde hiç bir değişiklik yapmadan --insecure-skip-tls-verify kısmını kullanacağım.

Örnek olarak aşağıda gösterebilirim ;

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJYytPeDVlWDFieGd3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TXpFd01UY3hOREkzTlRKYUZ3MHpNekV3TVRReE5ETXlOVEphTUJVeApFekFSQmdOVkJBTVRDbXQxWW1WeWJtVjBaWE13Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLCkFvSUJBUUNvbXVQVHBBNG90THZmRzhMT0lUVG9paERGS2Z0c3hicHJhUDFYNFY1aFQwUG5pRGZWZk5lSzJmZUcKZVdEU0ZNZDZmZGRTTFJPY3RDTFhveUNCRE9sQk5XMlZleENwWGk3R2dYSzc0TXMzVXRZcHVJUG1rSHg5cjliawo3Ujd2ZE5TVTh4dGdLek9qUERTbmVWYnhZUVQwWDVGWGpCZG1XWmJEcE9LRzU3MEN3Ym1BS21QcEpHOHhrNTFFCkhBSUp0SDlFVGt4Y1lPb3pEMHBmdk95V1pQREJQVEhESUxGbWFhNlhXNUJZNHZhY3dOa1RhelQrUUt4ZDRlWlUKM1hMc0RSQ29xNGp4M0ZkMzBPMWtzSWFpOW4wZWQxKy9LUDNGS1A5NzVZek1nNTF0b2Z3ZHBwL2pJWGowM2lUMApiYzVoZUUydldYTklQZkJHd1FBcVA5WHRGUEZkQWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJTZmlJYUd0UU52VXZaeDJGc01DbjQxNFpraFJ6QVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQWtNNklnRTY2Qgp4c091WWZjUXh1dEN0YytCV3JWOG9HVEo5S2hSUUZ3TjhZa0g3dVpOVzJVRzZiT0hZWWZxd1dVWkNJb1NIb3lICmQxcWxFa1RzQWR0ZG8vOGtOaDF3OVVLakVMTmJjL1ZjeitZZHplWTF3Nm1qdmZRcml0QVZBSE1yK3BzVVorOEcKcHU1cnFlSzZ3WlcwejFlT01Td1hzUUhJVEhVZmpwSzV4TmQ1U0ZVbTUxTzdjY2F5OExxNTB0QXUvU2QrTGFrcgptbndZWHRjVHFydzRVZXpoOEp1UkozTkhVSmlVUnRvaVJjMFl0TFBuQWFVc0dyZXhFZUdoM0dxc1hENEhWak4zClZPME9EWWQ0cEJCV0ZRamFaK2xrWk03cGU3NGt2TkdYa1QxOXVOSlV1cFkva0FUMER0UWZESXNLK09PLzVIdmsKWjYydkxlblZVYUlJCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    server: <https://172.30.1.2:6443>
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURJVENDQWdtZ0F3SUJBZ0lJQzdLVUJlMnRIczh3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TXpFd01UY3hOREkzTlRKYUZ3MHlOREV3TVRZeE5ETXlOVFphTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTFQMGRSclhJbUV3Nnd1NS8KSk5KUVJhZzFIUnZVaDFyeTlteUhlUFAyM3V6dzdudE9ENkhkTldDRWNwYk5ZWTBjNzVHblhnRGFGMG4rdmxqNApJN2g1NE12TjBzTE9IVEpzcG5GL1hBTmhqTnNHN3VCUW1NWWRRZ0hkOWU1MDJxb0ZTUzNndFMrR2tkRmJ1SHpQCmpDVFVBSTFPT21QUmJkLzhhTXJuMWYwcjZNMzlSZjd2Y1YxWDZNczZ6Ym03VkpjUkNmdTJMY0RFYnpHMHE2bEkKbGs5WG1qb2FQNFAxb25iUzBpR2xsZSt2cTBaSERJZUhiSXloYWk3RzVRR0tNWnZvSTZXTjFVbFhlNmtrdm5LMwp0MS9SZ1dFTk1uT2pmN3NqYVplYmdhWUZlNXlrVy96azluV2tHekZtTldDVFJqc0dzVmszanBQeEJYamtoTWxFCjJScExRd0lEQVFBQm8xWXdWREFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RBWURWUjBUQVFIL0JBSXdBREFmQmdOVkhTTUVHREFXZ0JTZmlJYUd0UU52VXZaeDJGc01DbjQxNFpraApSekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBSS9SMDdSMmhWUXptYUtMK0lqUlRFUkY1bjlaU2k1bXVnaHlMCkV0eEhuL1NUQ3NBa3N3MzFjLzZSWVA1M1F5N3Nyb1MxMGFra1BBRWpRTk1FNmRIdTFjNStJZE11T2w3Vmc0UlkKVkJMZjgrNUIwaG5JazBWeWVRYXRobkU0UDdqOE5vUGd1amJPMU12WHFKTUdLZGpYRXVpdS8rRzUyazZYR2FnVgo1bTZGUE4wa3ZPUHE0a1FuNjVHdjdqQ1kzbWUxaHZuZ2c0ZCtWY1pvcEhDZmtwMXZaYTF5Z3hQekRRaEhQeXZRCmYyK1NnMEs3NllGUGwrcVBvUjdscCszekcxbHhrc2o5VXYyODFNbzBuN3o1SUh4a1poNUloamc3THFqalpDZlQKZ0F2S2V5WDl1SDF5OThVYjZqK0wwcnJvUHV3S2xKaFdhVWx4WGNoV2NGUEowS1lOSHc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMVAwZFJyWEltRXc2d3U1L0pOSlFSYWcxSFJ2VWgxcnk5bXlIZVBQMjN1enc3bnRPCkQ2SGROV0NFY3BiTllZMGM3NUduWGdEYUYwbit2bGo0STdoNTRNdk4wc0xPSFRKc3BuRi9YQU5oak5zRzd1QlEKbU1ZZFFnSGQ5ZTUwMnFvRlNTM2d0UytHa2RGYnVIelBqQ1RVQUkxT09tUFJiZC84YU1ybjFmMHI2TTM5UmY3dgpjVjFYNk1zNnpibTdWSmNSQ2Z1MkxjREViekcwcTZsSWxrOVhtam9hUDRQMW9uYlMwaUdsbGUrdnEwWkhESWVICmJJeWhhaTdHNVFHS01adm9JNldOMVVsWGU2a2t2bkszdDEvUmdXRU5Nbk9qZjdzamFaZWJnYVlGZTV5a1cvemsKOW5Xa0d6Rm1OV0NUUmpzR3NWazNqcFB4Qlhqa2hNbEUyUnBMUXdJREFRQUJBb0lCQUFMa2RSTlN3S2lnMEdONQpUYXk0Vm5LN3VBenZKZEc0d0hmc1BXSzlIV3NsbUpnRWdLamhyOWxidURDK0w0ZlZuTUQ4SDJPOWF6akp2blhqClMydGd3L2J6alpZQkl6OXVRZDk0eHh2K3g4SmI3eG9yWGF6VmMreERqam9heE9kN1M1c1V6bXBuZk5KYUgzNi8KNHltcURsNzdqRVpxdUp0ZllWUWkvVWNBcXluOEs4NjcwTUc3MDJYVEk0VWphcVNJS2FaeHhYdTJVVG11bWxvQwowNTZzdytiaUhRYzIySUZWNUtYZlNVeU1YZlFabWpCcXJlaGVPeUtyZHZvdk5QTy8yOVcyVU9FSFptcitRUzdOCjZKR2pqalh4WDVjME0xY2NudmIwOUJLanY5OElVek9DZWlWa2RWcGhST0hIbHdIUTJrRy9KdkIrcFhmUTFPMTkKZURkUmFsRUNnWUVBNnFHdHVEcXdTdzhtSExTUDh1OW9lR0I1SG91L1RuanFMMkU1cWxPaHh4dWhvWldrbWFNNQpmSlFPZ1UrWUxyS1JBSDY1dm9Qbkw2S3ozTXRJVVpmUm10czl4b0VzTjllaDJSMzBkM3dTdWxhUFNYQXVEb2l6CkRhT2YwVVhSRGdiVkQ1TU5LSDlZdk1qVW9kNEd3NlIxbklGb2xFVkxKTlJQNjA1dnFZUEg3dDhDZ1lFQTZHTFgKK2wwTlh6ZEM3dGgyVzY2V0dETXJLNkRwbXNlN0pIZUxWbjVsZlFrNzRHaFRwWlN5c3U0aEw5UDAvSVhIbWZiVQptRWl0cENVVUNtckpBSVgvV09sdzBDVnZqRUYyUmlVUnMvNUpHMWFwRGMzV1pXb1pVa3liY3VoM244bmRYK2dZCkd0bFA0NkQvTlY2eG1rMCt4elhjNFlTb1RoS1ZmTnZxcVgwZlJCMENnWUJLTllpMFZCdWVJZFIwRDhmWGxsRDgKZ2NiTkduWVBZNzd0N1ZwSHZOdldGYTh4WkJzRFhOUzV5am9zTy9odURKeEFBOVRMcFB4aDNKdjJvWFRtR1paNQozQTNWMHRVaGRGb0VTYTBnZldXTytnR253SXRRblh2RVBKUTNwRlZTa21hZzNtbXkvVlFib1lsZkNOSXEzc0UwCkZ0bjB5MVhRK0RXdXZzZ2lYVVBIRVFLQmdRQ3k5QXNsL2FwaWZJZktybDdXMDJjNlU0MFhGOTlORmlvRE54dlMKQzV2VDRkeCtQMWpLMWV1VzV0SWZwcld1RU1kWXhUK0oxYzRCN1p1MU9lNTN2TGlhbTRvSWdBdWFrbjZtSGIyNApRa1hDQzhZN254dFBGWmxTU3pRYUZQS3AxaVhueVM5bTNab3J6THM3T2lUQ2ttR1p2dHpOdmY1dHdObmxRT1ZBCmg5NGY4UUtCZ0ZRVkJhYVNUdzQwTE5DM0pCNHZ3OTF2Vm5PWCs1dHpWQlpvRHB3YnBkejhSOG9zaG13M0VCT3oKS3ZpRE5Ma0Y1NUY1YXNxZ25INTFrWnNseGQvQm5mWHYxZ3Q5QnN3SGpCVGR2eXE3Z2ZPK0NxbG5BSjRCdnlQMApPL1FsZ0cxWUxKcHpwRFJsY3FzbFRQeFZFTU05REJvdUxTdVAyZVNyVmpNaksrdlE5OEdNCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==

certificate-authority-data: içerisinde base64 CA certificate kısmını bulunduruyor. certSANs da belirttiğim domainler bu certificate içerisinde verify olmuş şekilde bulunuyor .

Insecure kısma aşağıdaki config file’da görebilirsiniz.

v1 apiVersion: v1
clusters:
- cluster:
    insecure-skip-tls-verify: true
    server: <https://172.30.1.2:6443>
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURJVENDQWdtZ0F3SUJBZ0lJQzdLVUJlMnRIczh3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TXpFd01UY3hOREkzTlRKYUZ3MHlOREV3TVRZeE5ETXlOVFphTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTFQMGRSclhJbUV3Nnd1NS8KSk5KUVJhZzFIUnZVaDFyeTlteUhlUFAyM3V6dzdudE9ENkhkTldDRWNwYk5ZWTBjNzVHblhnRGFGMG4rdmxqNApJN2g1NE12TjBzTE9IVEpzcG5GL1hBTmhqTnNHN3VCUW1NWWRRZ0hkOWU1MDJxb0ZTUzNndFMrR2tkRmJ1SHpQCmpDVFVBSTFPT21QUmJkLzhhTXJuMWYwcjZNMzlSZjd2Y1YxWDZNczZ6Ym03VkpjUkNmdTJMY0RFYnpHMHE2bEkKbGs5WG1qb2FQNFAxb25iUzBpR2xsZSt2cTBaSERJZUhiSXloYWk3RzVRR0tNWnZvSTZXTjFVbFhlNmtrdm5LMwp0MS9SZ1dFTk1uT2pmN3NqYVplYmdhWUZlNXlrVy96azluV2tHekZtTldDVFJqc0dzVmszanBQeEJYamtoTWxFCjJScExRd0lEQVFBQm8xWXdWREFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RBWURWUjBUQVFIL0JBSXdBREFmQmdOVkhTTUVHREFXZ0JTZmlJYUd0UU52VXZaeDJGc01DbjQxNFpraApSekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBSS9SMDdSMmhWUXptYUtMK0lqUlRFUkY1bjlaU2k1bXVnaHlMCkV0eEhuL1NUQ3NBa3N3MzFjLzZSWVA1M1F5N3Nyb1MxMGFra1BBRWpRTk1FNmRIdTFjNStJZE11T2w3Vmc0UlkKVkJMZjgrNUIwaG5JazBWeWVRYXRobkU0UDdqOE5vUGd1amJPMU12WHFKTUdLZGpYRXVpdS8rRzUyazZYR2FnVgo1bTZGUE4wa3ZPUHE0a1FuNjVHdjdqQ1kzbWUxaHZuZ2c0ZCtWY1pvcEhDZmtwMXZaYTF5Z3hQekRRaEhQeXZRCmYyK1NnMEs3NllGUGwrcVBvUjdscCszekcxbHhrc2o5VXYyODFNbzBuN3o1SUh4a1poNUloamc3THFqalpDZlQKZ0F2S2V5WDl1SDF5OThVYjZqK0wwcnJvUHV3S2xKaFdhVWx4WGNoV2NGUEowS1lOSHc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMVAwZFJyWEltRXc2d3U1L0pOSlFSYWcxSFJ2VWgxcnk5bXlIZVBQMjN1enc3bnRPCkQ2SGROV0NFY3BiTllZMGM3NUduWGdEYUYwbit2bGo0STdoNTRNdk4wc0xPSFRKc3BuRi9YQU5oak5zRzd1QlEKbU1ZZFFnSGQ5ZTUwMnFvRlNTM2d0UytHa2RGYnVIelBqQ1RVQUkxT09tUFJiZC84YU1ybjFmMHI2TTM5UmY3dgpjVjFYNk1zNnpibTdWSmNSQ2Z1MkxjREViekcwcTZsSWxrOVhtam9hUDRQMW9uYlMwaUdsbGUrdnEwWkhESWVICmJJeWhhaTdHNVFHS01adm9JNldOMVVsWGU2a2t2bkszdDEvUmdXRU5Nbk9qZjdzamFaZWJnYVlGZTV5a1cvemsKOW5Xa0d6Rm1OV0NUUmpzR3NWazNqcFB4Qlhqa2hNbEUyUnBMUXdJREFRQUJBb0lCQUFMa2RSTlN3S2lnMEdONQpUYXk0Vm5LN3VBenZKZEc0d0hmc1BXSzlIV3NsbUpnRWdLamhyOWxidURDK0w0ZlZuTUQ4SDJPOWF6akp2blhqClMydGd3L2J6alpZQkl6OXVRZDk0eHh2K3g4SmI3eG9yWGF6VmMreERqam9heE9kN1M1c1V6bXBuZk5KYUgzNi8KNHltcURsNzdqRVpxdUp0ZllWUWkvVWNBcXluOEs4NjcwTUc3MDJYVEk0VWphcVNJS2FaeHhYdTJVVG11bWxvQwowNTZzdytiaUhRYzIySUZWNUtYZlNVeU1YZlFabWpCcXJlaGVPeUtyZHZvdk5QTy8yOVcyVU9FSFptcitRUzdOCjZKR2pqalh4WDVjME0xY2NudmIwOUJLanY5OElVek9DZWlWa2RWcGhST0hIbHdIUTJrRy9KdkIrcFhmUTFPMTkKZURkUmFsRUNnWUVBNnFHdHVEcXdTdzhtSExTUDh1OW9lR0I1SG91L1RuanFMMkU1cWxPaHh4dWhvWldrbWFNNQpmSlFPZ1UrWUxyS1JBSDY1dm9Qbkw2S3ozTXRJVVpmUm10czl4b0VzTjllaDJSMzBkM3dTdWxhUFNYQXVEb2l6CkRhT2YwVVhSRGdiVkQ1TU5LSDlZdk1qVW9kNEd3NlIxbklGb2xFVkxKTlJQNjA1dnFZUEg3dDhDZ1lFQTZHTFgKK2wwTlh6ZEM3dGgyVzY2V0dETXJLNkRwbXNlN0pIZUxWbjVsZlFrNzRHaFRwWlN5c3U0aEw5UDAvSVhIbWZiVQptRWl0cENVVUNtckpBSVgvV09sdzBDVnZqRUYyUmlVUnMvNUpHMWFwRGMzV1pXb1pVa3liY3VoM244bmRYK2dZCkd0bFA0NkQvTlY2eG1rMCt4elhjNFlTb1RoS1ZmTnZxcVgwZlJCMENnWUJLTllpMFZCdWVJZFIwRDhmWGxsRDgKZ2NiTkduWVBZNzd0N1ZwSHZOdldGYTh4WkJzRFhOUzV5am9zTy9odURKeEFBOVRMcFB4aDNKdjJvWFRtR1paNQozQTNWMHRVaGRGb0VTYTBnZldXTytnR253SXRRblh2RVBKUTNwRlZTa21hZzNtbXkvVlFib1lsZkNOSXEzc0UwCkZ0bjB5MVhRK0RXdXZzZ2lYVVBIRVFLQmdRQ3k5QXNsL2FwaWZJZktybDdXMDJjNlU0MFhGOTlORmlvRE54dlMKQzV2VDRkeCtQMWpLMWV1VzV0SWZwcld1RU1kWXhUK0oxYzRCN1p1MU9lNTN2TGlhbTRvSWdBdWFrbjZtSGIyNApRa1hDQzhZN254dFBGWmxTU3pRYUZQS3AxaVhueVM5bTNab3J6THM3T2lUQ2ttR1p2dHpOdmY1dHdObmxRT1ZBCmg5NGY4UUtCZ0ZRVkJhYVNUdzQwTE5DM0pCNHZ3OTF2Vm5PWCs1dHpWQlpvRHB3YnBkejhSOG9zaG13M0VCT3oKS3ZpRE5Ma0Y1NUY1YXNxZ25INTFrWnNseGQvQm5mWHYxZ3Q5QnN3SGpCVGR2eXE3Z2ZPK0NxbG5BSjRCdnlQMApPL1FsZ0cxWUxKcHpwRFJsY3FzbFRQeFZFTU05REJvdUxTdVAyZVNyVmpNaksrdlE5OEdNCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==

Bu şekilde her türlü erişimini sağlayabilirsiniz.

Eğer 3 Cluster’a da bir config file ile erişmek istiyorsanız. Kube config file içerisinde aşağıdaki gibi 3 cluster’da ekleyebilirsiniz.

v1 apiVersion: v1
clusters:
- cluster:
    insecure-skip-tls-verify: true
    server: <https://test.example.com:443>
  name: kubernetes
- cluster:
    insecure-skip-tls-verify: true
    server: <https://staging.example.com:443>
  name: kubernetes
- cluster:
    insecure-skip-tls-verify: true
    server: <https://production.example.com:443>
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-test-admin
  name: kubernetes-admin@test
- context:
    cluster: kubernetes
    user: kubernetes-staging-admin
  name: kubernetes-admin@staging
- context:
    cluster: kubernetes
    user: kubernetes-production-admin
  name: kubernetes-admin@production
current-context: kubernetes-admin@production
kind: Config
preferences: {}
users:
- name: kubernetes-test-admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURJVENDQWdtZ0F3SUJBZ0lJQzdLVUJlMnRIczh3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TXpFd01UY3hOREkzTlRKYUZ3MHlOREV3TVRZeE5ETXlOVFphTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTFQMGRSclhJbUV3Nnd1NS8KSk5KUVJhZzFIUnZVaDFyeTlteUhlUFAyM3V6dzdudE9ENkhkTldDRWNwYk5ZWTBjNzVHblhnRGFGMG4rdmxqNApJN2g1NE12TjBzTE9IVEpzcG5GL1hBTmhqTnNHN3VCUW1NWWRRZ0hkOWU1MDJxb0ZTUzNndFMrR2tkRmJ1SHpQCmpDVFVBSTFPT21QUmJkLzhhTXJuMWYwcjZNMzlSZjd2Y1YxWDZNczZ6Ym03VkpjUkNmdTJMY0RFYnpHMHE2bEkKbGs5WG1qb2FQNFAxb25iUzBpR2xsZSt2cTBaSERJZUhiSXloYWk3RzVRR0tNWnZvSTZXTjFVbFhlNmtrdm5LMwp0MS9SZ1dFTk1uT2pmN3NqYVplYmdhWUZlNXlrVy96azluV2tHekZtTldDVFJqc0dzVmszanBQeEJYamtoTWxFCjJScExRd0lEQVFBQm8xWXdWREFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RBWURWUjBUQVFIL0JBSXdBREFmQmdOVkhTTUVHREFXZ0JTZmlJYUd0UU52VXZaeDJGc01DbjQxNFpraApSekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBSS9SMDdSMmhWUXptYUtMK0lqUlRFUkY1bjlaU2k1bXVnaHlMCkV0eEhuL1NUQ3NBa3N3MzFjLzZSWVA1M1F5N3Nyb1MxMGFra1BBRWpRTk1FNmRIdTFjNStJZE11T2w3Vmc0UlkKVkJMZjgrNUIwaG5JazBWeWVRYXRobkU0UDdqOE5vUGd1amJPMU12WHFKTUdLZGpYRXVpdS8rRzUyazZYR2FnVgo1bTZGUE4wa3ZPUHE0a1FuNjVHdjdqQ1kzbWUxaHZuZ2c0ZCtWY1pvcEhDZmtwMXZaYTF5Z3hQekRRaEhQeXZRCmYyK1NnMEs3NllGUGwrcVBvUjdscCszekcxbHhrc2o5VXYyODFNbzBuN3o1SUh4a1poNUloamc3THFqalpDZlQKZ0F2S2V5WDl1SDF5OThVYjZqK0wwcnJvUHV3S2xKaFdhVWx4WGNoV2NGUEowS1lOSHc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMVAwZFJyWEltRXc2d3U1L0pOSlFSYWcxSFJ2VWgxcnk5bXlIZVBQMjN1enc3bnRPCkQ2SGROV0NFY3BiTllZMGM3NUduWGdEYUYwbit2bGo0STdoNTRNdk4wc0xPSFRKc3BuRi9YQU5oak5zRzd1QlEKbU1ZZFFnSGQ5ZTUwMnFvRlNTM2d0UytHa2RGYnVIelBqQ1RVQUkxT09tUFJiZC84YU1ybjFmMHI2TTM5UmY3dgpjVjFYNk1zNnpibTdWSmNSQ2Z1MkxjREViekcwcTZsSWxrOVhtam9hUDRQMW9uYlMwaUdsbGUrdnEwWkhESWVICmJJeWhhaTdHNVFHS01adm9JNldOMVVsWGU2a2t2bkszdDEvUmdXRU5Nbk9qZjdzamFaZWJnYVlGZTV5a1cvemsKOW5Xa0d6Rm1OV0NUUmpzR3NWazNqcFB4Qlhqa2hNbEUyUnBMUXdJREFRQUJBb0lCQUFMa2RSTlN3S2lnMEdONQpUYXk0Vm5LN3VBenZKZEc0d0hmc1BXSzlIV3NsbUpnRWdLamhyOWxidURDK0w0ZlZuTUQ4SDJPOWF6akp2blhqClMydGd3L2J6alpZQkl6OXVRZDk0eHh2K3g4SmI3eG9yWGF6VmMreERqam9heE9kN1M1c1V6bXBuZk5KYUgzNi8KNHltcURsNzdqRVpxdUp0ZllWUWkvVWNBcXluOEs4NjcwTUc3MDJYVEk0VWphcVNJS2FaeHhYdTJVVG11bWxvQwowNTZzdytiaUhRYzIySUZWNUtYZlNVeU1YZlFabWpCcXJlaGVPeUtyZHZvdk5QTy8yOVcyVU9FSFptcitRUzdOCjZKR2pqalh4WDVjME0xY2NudmIwOUJLanY5OElVek9DZWlWa2RWcGhST0hIbHdIUTJrRy9KdkIrcFhmUTFPMTkKZURkUmFsRUNnWUVBNnFHdHVEcXdTdzhtSExTUDh1OW9lR0I1SG91L1RuanFMMkU1cWxPaHh4dWhvWldrbWFNNQpmSlFPZ1UrWUxyS1JBSDY1dm9Qbkw2S3ozTXRJVVpmUm10czl4b0VzTjllaDJSMzBkM3dTdWxhUFNYQXVEb2l6CkRhT2YwVVhSRGdiVkQ1TU5LSDlZdk1qVW9kNEd3NlIxbklGb2xFVkxKTlJQNjA1dnFZUEg3dDhDZ1lFQTZHTFgKK2wwTlh6ZEM3dGgyVzY2V0dETXJLNkRwbXNlN0pIZUxWbjVsZlFrNzRHaFRwWlN5c3U0aEw5UDAvSVhIbWZiVQptRWl0cENVVUNtckpBSVgvV09sdzBDVnZqRUYyUmlVUnMvNUpHMWFwRGMzV1pXb1pVa3liY3VoM244bmRYK2dZCkd0bFA0NkQvTlY2eG1rMCt4elhjNFlTb1RoS1ZmTnZxcVgwZlJCMENnWUJLTllpMFZCdWVJZFIwRDhmWGxsRDgKZ2NiTkduWVBZNzd0N1ZwSHZOdldGYTh4WkJzRFhOUzV5am9zTy9odURKeEFBOVRMcFB4aDNKdjJvWFRtR1paNQozQTNWMHRVaGRGb0VTYTBnZldXTytnR253SXRRblh2RVBKUTNwRlZTa21hZzNtbXkvVlFib1lsZkNOSXEzc0UwCkZ0bjB5MVhRK0RXdXZzZ2lYVVBIRVFLQmdRQ3k5QXNsL2FwaWZJZktybDdXMDJjNlU0MFhGOTlORmlvRE54dlMKQzV2VDRkeCtQMWpLMWV1VzV0SWZwcld1RU1kWXhUK0oxYzRCN1p1MU9lNTN2TGlhbTRvSWdBdWFrbjZtSGIyNApRa1hDQzhZN254dFBGWmxTU3pRYUZQS3AxaVhueVM5bTNab3J6THM3T2lUQ2ttR1p2dHpOdmY1dHdObmxRT1ZBCmg5NGY4UUtCZ0ZRVkJhYVNUdzQwTE5DM0pCNHZ3OTF2Vm5PWCs1dHpWQlpvRHB3YnBkejhSOG9zaG13M0VCT3oKS3ZpRE5Ma0Y1NUY1YXNxZ25INTFrWnNseGQvQm5mWHYxZ3Q5QnN3SGpCVGR2eXE3Z2ZPK0NxbG5BSjRCdnlQMApPL1FsZ0cxWUxKcHpwRFJsY3FzbFRQeFZFTU05REJvdUxTdVAyZVNyVmpNaksrdlE5OEdNCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
- name: kubernetes-staging-admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURJVENDQWdtZ0F3SUJBZ0lJQzdLVUJlMnRIczh3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TXpFd01UY3hOREkzTlRKYUZ3MHlOREV3TVRZeE5ETXlOVFphTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTFQMGRSclhJbUV3Nnd1NS8KSk5KUVJhZzFIUnZVaDFyeTlteUhlUFAyM3V6dzdudE9ENkhkTldDRWNwYk5ZWTBjNzVHblhnRGFGMG4rdmxqNApJN2g1NE12TjBzTE9IVEpzcG5GL1hBTmhqTnNHN3VCUW1NWWRRZ0hkOWU1MDJxb0ZTUzNndFMrR2tkRmJ1SHpQCmpDVFVBSTFPT21QUmJkLzhhTXJuMWYwcjZNMzlSZjd2Y1YxWDZNczZ6Ym03VkpjUkNmdTJMY0RFYnpHMHE2bEkKbGs5WG1qb2FQNFAxb25iUzBpR2xsZSt2cTBaSERJZUhiSXloYWk3RzVRR0tNWnZvSTZXTjFVbFhlNmtrdm5LMwp0MS9SZ1dFTk1uT2pmN3NqYVplYmdhWUZlNXlrVy96azluV2tHekZtTldDVFJqc0dzVmszanBQeEJYamtoTWxFCjJScExRd0lEQVFBQm8xWXdWREFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RBWURWUjBUQVFIL0JBSXdBREFmQmdOVkhTTUVHREFXZ0JTZmlJYUd0UU52VXZaeDJGc01DbjQxNFpraApSekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBSS9SMDdSMmhWUXptYUtMK0lqUlRFUkY1bjlaU2k1bXVnaHlMCkV0eEhuL1NUQ3NBa3N3MzFjLzZSWVA1M1F5N3Nyb1MxMGFra1BBRWpRTk1FNmRIdTFjNStJZE11T2w3Vmc0UlkKVkJMZjgrNUIwaG5JazBWeWVRYXRobkU0UDdqOE5vUGd1amJPMU12WHFKTUdLZGpYRXVpdS8rRzUyazZYR2FnVgo1bTZGUE4wa3ZPUHE0a1FuNjVHdjdqQ1kzbWUxaHZuZ2c0ZCtWY1pvcEhDZmtwMXZaYTF5Z3hQekRRaEhQeXZRCmYyK1NnMEs3NllGUGwrcVBvUjdscCszekcxbHhrc2o5VXYyODFNbzBuN3o1SUh4a1poNUloamc3THFqalpDZlQKZ0F2S2V5WDl1SDF5OThVYjZqK0wwcnJvUHV3S2xKaFdhVWx4WGNoV2NGUEowS1lOSHc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMVAwZFJyWEltRXc2d3U1L0pOSlFSYWcxSFJ2VWgxcnk5bXlIZVBQMjN1enc3bnRPCkQ2SGROV0NFY3BiTllZMGM3NUduWGdEYUYwbit2bGo0STdoNTRNdk4wc0xPSFRKc3BuRi9YQU5oak5zRzd1QlEKbU1ZZFFnSGQ5ZTUwMnFvRlNTM2d0UytHa2RGYnVIelBqQ1RVQUkxT09tUFJiZC84YU1ybjFmMHI2TTM5UmY3dgpjVjFYNk1zNnpibTdWSmNSQ2Z1MkxjREViekcwcTZsSWxrOVhtam9hUDRQMW9uYlMwaUdsbGUrdnEwWkhESWVICmJJeWhhaTdHNVFHS01adm9JNldOMVVsWGU2a2t2bkszdDEvUmdXRU5Nbk9qZjdzamFaZWJnYVlGZTV5a1cvemsKOW5Xa0d6Rm1OV0NUUmpzR3NWazNqcFB4Qlhqa2hNbEUyUnBMUXdJREFRQUJBb0lCQUFMa2RSTlN3S2lnMEdONQpUYXk0Vm5LN3VBenZKZEc0d0hmc1BXSzlIV3NsbUpnRWdLamhyOWxidURDK0w0ZlZuTUQ4SDJPOWF6akp2blhqClMydGd3L2J6alpZQkl6OXVRZDk0eHh2K3g4SmI3eG9yWGF6VmMreERqam9heE9kN1M1c1V6bXBuZk5KYUgzNi8KNHltcURsNzdqRVpxdUp0ZllWUWkvVWNBcXluOEs4NjcwTUc3MDJYVEk0VWphcVNJS2FaeHhYdTJVVG11bWxvQwowNTZzdytiaUhRYzIySUZWNUtYZlNVeU1YZlFabWpCcXJlaGVPeUtyZHZvdk5QTy8yOVcyVU9FSFptcitRUzdOCjZKR2pqalh4WDVjME0xY2NudmIwOUJLanY5OElVek9DZWlWa2RWcGhST0hIbHdIUTJrRy9KdkIrcFhmUTFPMTkKZURkUmFsRUNnWUVBNnFHdHVEcXdTdzhtSExTUDh1OW9lR0I1SG91L1RuanFMMkU1cWxPaHh4dWhvWldrbWFNNQpmSlFPZ1UrWUxyS1JBSDY1dm9Qbkw2S3ozTXRJVVpmUm10czl4b0VzTjllaDJSMzBkM3dTdWxhUFNYQXVEb2l6CkRhT2YwVVhSRGdiVkQ1TU5LSDlZdk1qVW9kNEd3NlIxbklGb2xFVkxKTlJQNjA1dnFZUEg3dDhDZ1lFQTZHTFgKK2wwTlh6ZEM3dGgyVzY2V0dETXJLNkRwbXNlN0pIZUxWbjVsZlFrNzRHaFRwWlN5c3U0aEw5UDAvSVhIbWZiVQptRWl0cENVVUNtckpBSVgvV09sdzBDVnZqRUYyUmlVUnMvNUpHMWFwRGMzV1pXb1pVa3liY3VoM244bmRYK2dZCkd0bFA0NkQvTlY2eG1rMCt4elhjNFlTb1RoS1ZmTnZxcVgwZlJCMENnWUJLTllpMFZCdWVJZFIwRDhmWGxsRDgKZ2NiTkduWVBZNzd0N1ZwSHZOdldGYTh4WkJzRFhOUzV5am9zTy9odURKeEFBOVRMcFB4aDNKdjJvWFRtR1paNQozQTNWMHRVaGRGb0VTYTBnZldXTytnR253SXRRblh2RVBKUTNwRlZTa21hZzNtbXkvVlFib1lsZkNOSXEzc0UwCkZ0bjB5MVhRK0RXdXZzZ2lYVVBIRVFLQmdRQ3k5QXNsL2FwaWZJZktybDdXMDJjNlU0MFhGOTlORmlvRE54dlMKQzV2VDRkeCtQMWpLMWV1VzV0SWZwcld1RU1kWXhUK0oxYzRCN1p1MU9lNTN2TGlhbTRvSWdBdWFrbjZtSGIyNApRa1hDQzhZN254dFBGWmxTU3pRYUZQS3AxaVhueVM5bTNab3J6THM3T2lUQ2ttR1p2dHpOdmY1dHdObmxRT1ZBCmg5NGY4UUtCZ0ZRVkJhYVNUdzQwTE5DM0pCNHZ3OTF2Vm5PWCs1dHpWQlpvRHB3YnBkejhSOG9zaG13M0VCT3oKS3ZpRE5Ma0Y1NUY1YXNxZ25INTFrWnNseGQvQm5mWHYxZ3Q5QnN3SGpCVGR2eXE3Z2ZPK0NxbG5BSjRCdnlQMApPL1FsZ0cxWUxKcHpwRFJsY3FzbFRQeFZFTU05REJvdUxTdVAyZVNyVmpNaksrdlE5OEdNCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
- name: kubernetes-production-admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURJVENDQWdtZ0F3SUJBZ0lJQzdLVUJlMnRIczh3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TXpFd01UY3hOREkzTlRKYUZ3MHlOREV3TVRZeE5ETXlOVFphTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQTFQMGRSclhJbUV3Nnd1NS8KSk5KUVJhZzFIUnZVaDFyeTlteUhlUFAyM3V6dzdudE9ENkhkTldDRWNwYk5ZWTBjNzVHblhnRGFGMG4rdmxqNApJN2g1NE12TjBzTE9IVEpzcG5GL1hBTmhqTnNHN3VCUW1NWWRRZ0hkOWU1MDJxb0ZTUzNndFMrR2tkRmJ1SHpQCmpDVFVBSTFPT21QUmJkLzhhTXJuMWYwcjZNMzlSZjd2Y1YxWDZNczZ6Ym03VkpjUkNmdTJMY0RFYnpHMHE2bEkKbGs5WG1qb2FQNFAxb25iUzBpR2xsZSt2cTBaSERJZUhiSXloYWk3RzVRR0tNWnZvSTZXTjFVbFhlNmtrdm5LMwp0MS9SZ1dFTk1uT2pmN3NqYVplYmdhWUZlNXlrVy96azluV2tHekZtTldDVFJqc0dzVmszanBQeEJYamtoTWxFCjJScExRd0lEQVFBQm8xWXdWREFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RBWURWUjBUQVFIL0JBSXdBREFmQmdOVkhTTUVHREFXZ0JTZmlJYUd0UU52VXZaeDJGc01DbjQxNFpraApSekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBSS9SMDdSMmhWUXptYUtMK0lqUlRFUkY1bjlaU2k1bXVnaHlMCkV0eEhuL1NUQ3NBa3N3MzFjLzZSWVA1M1F5N3Nyb1MxMGFra1BBRWpRTk1FNmRIdTFjNStJZE11T2w3Vmc0UlkKVkJMZjgrNUIwaG5JazBWeWVRYXRobkU0UDdqOE5vUGd1amJPMU12WHFKTUdLZGpYRXVpdS8rRzUyazZYR2FnVgo1bTZGUE4wa3ZPUHE0a1FuNjVHdjdqQ1kzbWUxaHZuZ2c0ZCtWY1pvcEhDZmtwMXZaYTF5Z3hQekRRaEhQeXZRCmYyK1NnMEs3NllGUGwrcVBvUjdscCszekcxbHhrc2o5VXYyODFNbzBuN3o1SUh4a1poNUloamc3THFqalpDZlQKZ0F2S2V5WDl1SDF5OThVYjZqK0wwcnJvUHV3S2xKaFdhVWx4WGNoV2NGUEowS1lOSHc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMVAwZFJyWEltRXc2d3U1L0pOSlFSYWcxSFJ2VWgxcnk5bXlIZVBQMjN1enc3bnRPCkQ2SGROV0NFY3BiTllZMGM3NUduWGdEYUYwbit2bGo0STdoNTRNdk4wc0xPSFRKc3BuRi9YQU5oak5zRzd1QlEKbU1ZZFFnSGQ5ZTUwMnFvRlNTM2d0UytHa2RGYnVIelBqQ1RVQUkxT09tUFJiZC84YU1ybjFmMHI2TTM5UmY3dgpjVjFYNk1zNnpibTdWSmNSQ2Z1MkxjREViekcwcTZsSWxrOVhtam9hUDRQMW9uYlMwaUdsbGUrdnEwWkhESWVICmJJeWhhaTdHNVFHS01adm9JNldOMVVsWGU2a2t2bkszdDEvUmdXRU5Nbk9qZjdzamFaZWJnYVlGZTV5a1cvemsKOW5Xa0d6Rm1OV0NUUmpzR3NWazNqcFB4Qlhqa2hNbEUyUnBMUXdJREFRQUJBb0lCQUFMa2RSTlN3S2lnMEdONQpUYXk0Vm5LN3VBenZKZEc0d0hmc1BXSzlIV3NsbUpnRWdLamhyOWxidURDK0w0ZlZuTUQ4SDJPOWF6akp2blhqClMydGd3L2J6alpZQkl6OXVRZDk0eHh2K3g4SmI3eG9yWGF6VmMreERqam9heE9kN1M1c1V6bXBuZk5KYUgzNi8KNHltcURsNzdqRVpxdUp0ZllWUWkvVWNBcXluOEs4NjcwTUc3MDJYVEk0VWphcVNJS2FaeHhYdTJVVG11bWxvQwowNTZzdytiaUhRYzIySUZWNUtYZlNVeU1YZlFabWpCcXJlaGVPeUtyZHZvdk5QTy8yOVcyVU9FSFptcitRUzdOCjZKR2pqalh4WDVjME0xY2NudmIwOUJLanY5OElVek9DZWlWa2RWcGhST0hIbHdIUTJrRy9KdkIrcFhmUTFPMTkKZURkUmFsRUNnWUVBNnFHdHVEcXdTdzhtSExTUDh1OW9lR0I1SG91L1RuanFMMkU1cWxPaHh4dWhvWldrbWFNNQpmSlFPZ1UrWUxyS1JBSDY1dm9Qbkw2S3ozTXRJVVpmUm10czl4b0VzTjllaDJSMzBkM3dTdWxhUFNYQXVEb2l6CkRhT2YwVVhSRGdiVkQ1TU5LSDlZdk1qVW9kNEd3NlIxbklGb2xFVkxKTlJQNjA1dnFZUEg3dDhDZ1lFQTZHTFgKK2wwTlh6ZEM3dGgyVzY2V0dETXJLNkRwbXNlN0pIZUxWbjVsZlFrNzRHaFRwWlN5c3U0aEw5UDAvSVhIbWZiVQptRWl0cENVVUNtckpBSVgvV09sdzBDVnZqRUYyUmlVUnMvNUpHMWFwRGMzV1pXb1pVa3liY3VoM244bmRYK2dZCkd0bFA0NkQvTlY2eG1rMCt4elhjNFlTb1RoS1ZmTnZxcVgwZlJCMENnWUJLTllpMFZCdWVJZFIwRDhmWGxsRDgKZ2NiTkduWVBZNzd0N1ZwSHZOdldGYTh4WkJzRFhOUzV5am9zTy9odURKeEFBOVRMcFB4aDNKdjJvWFRtR1paNQozQTNWMHRVaGRGb0VTYTBnZldXTytnR253SXRRblh2RVBKUTNwRlZTa21hZzNtbXkvVlFib1lsZkNOSXEzc0UwCkZ0bjB5MVhRK0RXdXZzZ2lYVVBIRVFLQmdRQ3k5QXNsL2FwaWZJZktybDdXMDJjNlU0MFhGOTlORmlvRE54dlMKQzV2VDRkeCtQMWpLMWV1VzV0SWZwcld1RU1kWXhUK0oxYzRCN1p1MU9lNTN2TGlhbTRvSWdBdWFrbjZtSGIyNApRa1hDQzhZN254dFBGWmxTU3pRYUZQS3AxaVhueVM5bTNab3J6THM3T2lUQ2ttR1p2dHpOdmY1dHdObmxRT1ZBCmg5NGY4UUtCZ0ZRVkJhYVNUdzQwTE5DM0pCNHZ3OTF2Vm5PWCs1dHpWQlpvRHB3YnBkejhSOG9zaG13M0VCT3oKS3ZpRE5Ma0Y1NUY1YXNxZ25INTFrWnNseGQvQm5mWHYxZ3Q5QnN3SGpCVGR2eXE3Z2ZPK0NxbG5BSjRCdnlQMApPL1FsZ0cxWUxKcHpwRFJsY3FzbFRQeFZFTU05REJvdUxTdVAyZVNyVmpNaksrdlE5OEdNCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==

Şimdi sıradaki adım, 2 sunucuya Keepalived ve HAProxy kurulumu ve yapılandırılması işlemidir.

HAProxy ve Keepalived kurulumu

Ubuntu 22.04 2 adet sunucuya aşağıda update ve HAProxy ve keepalived yükleniyor.

  • Server-01 192.168.10.51
  • Server-02 192.168.10.52
  • VIP → 192.168.10.50
sudo apt update
sudo apt upgrade
sudo timedatectl set-timezone Europe/Istanbul
sudo add-apt-repository ppa:vbernat/haproxy-2.8
apt-get install keepalived
apt-get install haproxy=2.8.\\*

Server-01 için keepalived konfigürasyonu

vi /etc/keepalived/keepalived.conf
# Global Settings for notifications
global_defs {

}

# Define the script used to check if haproxy is still working
vrrp_script chk_haproxy {
    script "/usr/bin/killall -0 haproxy"
    interval 2
    weight 2
}

# Configuration for Virtual Interface
vrrp_instance LB_VIP {
    interface ens192
    state MASTER        # set to BACKUP on the peer machine
    priority 101        # set to  99 on the peer machine
    virtual_router_id 30

    smtp_alert          # Enable Notifications Via Email

    authentication {
        auth_type PASS
        auth_pass mYP@312da    # Password for accessing vrrpd. Same on all devices
    }
    unicast_src_ip 192.168.10.51 # Private IP address of master
    unicast_peer {
        192.168.10.52
   }

    # The virtual ip address shared between the two loadbalancers
    virtual_ipaddress {
        192.168.10.50
    }

    # Use the Defined Script to Check whether to initiate a fail over
    track_script {
        chk_haproxy
    }
}

Yukarada göründüğü gibi server-01 ve server-02 birbirleri arasında peer kurar ve bu konfigürasyona göre MASTER server-01 seçiliyor. HAProxy service i öldüğü durumda anında BACKUP a geçmektedir.

Ekstra olarak /etc/sysctl.conf içerisine aşağıdaki komutları eklememiz gerekmektedir.

net.ipv4.ip_forward = 1
net.ipv4.ip_nonlocal_bind = 1

Yukarıdaki config’ler sayesinde server üstünde ip olmasına rağmen bind edebiliriz.

Server-02 için keepalived konfigürasyonu

vi /etc/keepalived/keepalived.conf
# Global Settings for notifications
global_defs {

}

# Define the script used to check if haproxy is still working
vrrp_script chk_haproxy {
    script "/usr/bin/killall -0 haproxy"
    interval 2
    weight 2
}

# Configuration for Virtual Interface
vrrp_instance LB_VIP {
    interface ens192
    state BACKUP        # set to BACKUP on the peer machine
    priority 100        # set to  99 on the peer machine
    virtual_router_id 30

    smtp_alert          # Enable Notifications Via Email

    authentication {
        auth_type PASS
        auth_pass mYP@312da    # Password for accessing vrrpd. Same on all devices
    }
    unicast_src_ip 192.168.10.52 # Private IP address of master
    unicast_peer {
        192.168.10.51
   }

    # The virtual ip address shared between the two loadbalancers
    virtual_ipaddress {
        192.168.10.50
    }

    # Use the Defined Script to Check whether to initiate a fail over
    track_script {
        chk_haproxy
    }
}

Yukarıda server-01’den farklı unicast_src_ip unicast_peer state priority

bu değişkenler vardır. unicast_src_ip unicast_peer bu değerler server iplerinden olduğundan dolayı değişmektedir.

  • state → BACKUP (yedek olarak durmaktadır.)
  • priority → 100 (priority olarak master dan az olması gerekmektedir.)

Ekstra olarak /etc/sysctl.conf içerisine aşağıdaki komutları eklememiz gerekmekedir.

net.ipv4.ip_forward = 1
net.ipv4.ip_nonlocal_bind = 1

Yukarıdaki config’ler sayesinde server üstünde ip olmasına rağmen bind edebiliriz.

Keepalived ve sysctl deki konfigürasyonları bitirdikten daha sonra indirdiğimiz HAProxy 2.8.x LTS in konfigürasyonunu yapabiliriz.

vi /etc/haproxy/haproxy.cfg
# Load Balancer Configuration for Production Environment

global
    maxconn 10000

defaults
    log global
    log /dev/log local0 notice
    option dontlognull
    mode tcp
    retries 3
    timeout client 60s
    timeout connect 60s
    timeout server 60s
    timeout check 5s

listen example-stats
    mode http
    bind 192.168.10.50:7000
    stats enable
    stats uri /

frontend example-k8s-frontend
    mode tcp
    bind 192.168.10.50:443
    tcp-request inspect-delay 5s
    tcp-request content accept if { req.ssl_hello_type 1 }

    acl example-test-domain req_ssl_sni -i test.example.com
    acl example-staging-domain req_ssl_sni -i staging.example.com
    acl example-prod-domain req_ssl_sni -i production.example.com

    use_backend example-test-k8s-backend if example-test-domain
    use_backend example-staging-k8s-backend if example-staging-domain
    use_backend example-production-k8s-backend if example-prod-domain

backend example-test-k8s-backend
    mode tcp
    balance source
    option tcplog
    option tcp-check
    default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
    server example-test-master01 192.168.20.51:6443 check check-ssl verify none inter 10000
    server example-test-master02 192.168.20.52:6443 check check-ssl verify none inter 10000
    server example-test-master03 192.168.20.53:6443 check check-ssl verify none inter 10000

backend example-staging-k8s-backend
    mode tcp
    balance source
    option tcplog
    option tcp-check
    default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
    server example-staging-master01 192.168.30.51:6443 check check-ssl verify none inter 10000
    server example-staging-master02 192.168.30.52:6443 check check-ssl verify none inter 10000
    server example-staging-master03 192.168.30.53:6443 check check-ssl verify none inter 10000

backend example-production-k8s-backend
    mode tcp
    balance source
    option tcplog
    option tcp-check
    default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
    server example-production-master01 192.168.40.51:6443 check check-ssl verify none inter 10000
    server example-production-master02 192.168.40.52:6443 check check-ssl verify none inter 10000
    server example-production-master03 192.168.40.53:6443 check check-ssl verify none inter 10000
    server example-production-master04 192.168.40.54:6443 check check-ssl verify none inter 10000
    server example-production-master05 192.168.40.55:6443 check check-ssl verify none inter 10000

Yukarıdaki yapılandırmada, 3 farklı cluster üzerindeki Master kube API sunucularına erişim sağlayabiliriz.

  • test cluster 3 master node
  • staging cluster 3 master node
  • production cluster 5 master node

bulundurmaktadır.

API erişimi tcp olduğundan dolayı (http ile çalışamaz) mode tcp seçilmiştir.

HAProxy istatistiklerini 7000 portunda görebiliriz.

Domainler ile beraber 443 portundan erişebiliriz.

acl example-test-domain req_ssl_sni -i test.example.com
acl example-staging-domain req_ssl_sni -i staging.example.com
acl example-prod-domain req_ssl_sni -i production.example.com

use_backend example-test-k8s-backend if example-test-domain
use_backend example-staging-k8s-backend if example-staging-domain
use_backend example-production-k8s-backend if example-prod-domain

HAProxy’ye geldiğimiz domain ile istediğimiz backend yönlendirmelerini yapıyoruz.

References:

How to add a new hostname or IP address to a Standalone Kubernetes API server - Kloudle Website

A short post on adding a new hostname or IP address to the PKI certificate of a Kubernetes cluster so that TLS…

kloudle.com

kubernetes-handbook/load-balancer/setup-haproxy.md at master · shawnsong/kubernetes-handbook

Kubernetes Cluster Setup Handbook and Best Practices - kubernetes-handbook/load-balancer/setup-haproxy.md at master ·…

github.com

Kubernetes
Kubernetes Cluster
Haproxy
DevOps
Keepalived

--

--

Serdarcan Büyükdereli
Arabam Labs

Written by Serdarcan Büyükdereli

116 Followers
Writer for

https://serdarcanb.github.io

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams