Open in app

Sign in

Write

Sign in

Authentication and Authorization with “PINNIPED” on Kubernetes Detail walkthrough!

Ark Innovations
4 min readDec 5, 2022

--

Ark Innovation article by Kushal.

Kubernetes supports various authentication mechanisms in order to deploy in production and security meets. Kubernetes must authenticate using IDP such as LDAP or Auth 2.

We will POC integrate with PINNIPED in this article.

Architecture

Components:

  1. CLI: Command line interface used for user authentication.
  2. Concierge: Handles local cluster authentication.
  3. Supervisor: Handles multi level clusters so that user can login only once.

Installation:

$ brew install vmware-tanzu/pinniped/pinniped-cli
$ kubectl apply -f https://get.pinniped.dev/v0.19.0/install-pinniped-concierge-crds.yaml
 
$ kubectl apply -f https://get.pinniped.dev/v0.19.0/install-pinniped-concierge-resources.yaml
 
#verify installation
$ kubectl get po -n pinniped-concierge
NAME                                                 READY   STATUS    RESTARTS   AGE
pinniped-concierge-54647c8857-dm7lt                  1/1     Running   0          25s
pinniped-concierge-54647c8857-pc2qg                  1/1     Running   0          25s
pinniped-concierge-kube-cert-agent-7fcbfc754-9rc66   1/1     Running   0          14s
Gitlab
Application
Authenticator & Create a secret-less

YAML Generated

$ cat ~/.kube/c4-sjt-gitlab.yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeU1EZ3hNREl4TURBek1Wb1hEVE15TURnd056SXhNREF6TVZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTC92CldRd3JQWVlXSDFOdXpzWHZNdzdRMXdFcEFDSVJhU0NlcVhTODUreWRwdGI0R053OWd5d2pGZ0ZjSlV6NEd3d3AKd1VtWjBMU3Q2UzlQOXN6eG9QVjFIKy9FUVk3L244RjNYSWtqNnJXcWJPOU9FcXVjdjhBNTkrTXJWTk1qRU94Wgpxd0NRUzRiM0N6T3NWQVptL3o0bmx5MThUR09LWVNJcTBqZ1doUmUzN0U0dWtmSXNJdGtWalZ0OVZsL01NY2pqCm5VMmVLM0orRUx4am1LN09Fdk9EOWh2VEVZRW9EUFBQWDBack1QbDVMQmQvQ3c0MnJ6enp2SjllUjRzQmhUSk8KQnNCUWFTNG04b1dyd1k0ZkFPeVVIOEp3NytDcTlvQThxcTVNc2MvYjVTMGd1amcxYUNsTVJ3NlV6M3Z2VzRQSQpVWHJRQkJxTVZPaWhaVzR1YTJVQ0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZJeFYxN1NRYTlqaENuc21CZXFIaVpMSjlwQ3RNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBSSt5QlBXY0F1MUJPYzNJUHdxcApJM3diSHVQVzd3NDh4Szg2U1dMY2Y3eEl5NGFqWWNpZTNja1UvQWxOUTF5bFhRd2NFYU5Wd0dIb0tvaEVldDNxCjV6QW5EWm9XMXpjVUFoK1BrNVdEYTRaQnlvQklhMUVWNURFK2dTTTM3Zk9XQzE4Y0Q1R29aOUgvRCtFK3BTQVQKNFVtWHVrb2dJMlk4dU1pSUNSaXhiM1E4MHhoTnE2emtad2pmWDlpWlpLS3hxdHFWRVlNRkRKb2FpODh2aFJYYQo4WW1hclJnVEVGRVJrQ3oyY0w0aHovQVhjbGtEeU12djdtYlYwMFFKQW9jbEpMaXQzNm0xaHllbFQxeEIyN0luCkZwTWEreEZJMTU0RE5uUXNWa0VOcE1HWlR4Q294bEdCd1R3cFB1T3Z5bmN1WlJ0TkYvSlJqY1QzLzl4dEZSUVUKOXZjPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://10.100.**.**:****
  name: kubernetes-pinniped
contexts:
- context:
    cluster: kubernetes-pinniped
    user: kubernetes-admin-pinniped
  name: kubernetes-admin@kubernetes-pinniped
current-context: kubernetes-admin@kubernetes-pinniped
kind: Config
preferences: {}
users:
- name: kubernetes-admin-pinniped
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - login
      - oidc
      - --enable-concierge
      - --concierge-api-group-suffix=pinniped.dev
      - --concierge-authenticator-name=gitlab
      - --concierge-authenticator-type=jwt
      - --concierge-endpoint=https://10.100.**.**:****
      - --concierge-ca-bundle-data=LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeU1EZ3hNREl4TURBek1Wb1hEVE15TURnd056SXhNREF6TVZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTC92CldRd3JQWVlXSDFOdXpzWHZNdzdRMXdFcEFDSVJhU0NlcVhTODUreWRwdGI0R053OWd5d2pGZ0ZjSlV6NEd3d3AKd1VtWjBMU3Q2UzlQOXN6eG9QVjFIKy9FUVk3L244RjNYSWtqNnJXcWJPOU9FcXVjdjhBNTkrTXJWTk1qRU94Wgpxd0NRUzRiM0N6T3NWQVptL3o0bmx5MThUR09LWVNJcTBqZ1doUmUzN0U0dWtmSXNJdGtWalZ0OVZsL01NY2pqCm5VMmVLM0orRUx4am1LN09Fdk9EOWh2VEVZRW9EUFBQWDBack1QbDVMQmQvQ3c0MnJ6enp2SjllUjRzQmhUSk8KQnNCUWFTNG04b1dyd1k0ZkFPeVVIOEp3NytDcTlvQThxcTVNc2MvYjVTMGd1amcxYUNsTVJ3NlV6M3Z2VzRQSQpVWHJRQkJxTVZPaWhaVzR1YTJVQ0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZJeFYxN1NRYTlqaENuc21CZXFIaVpMSjlwQ3RNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBSSt5QlBXY0F1MUJPYzNJUHdxcApJM3diSHVQVzd3NDh4Szg2U1dMY2Y3eEl5NGFqWWNpZTNja1UvQWxOUTF5bFhRd2NFYU5Wd0dIb0tvaEVldDNxCjV6QW5EWm9XMXpjVUFoK1BrNVdEYTRaQnlvQklhMUVWNURFK2dTTTM3Zk9XQzE4Y0Q1R29aOUgvRCtFK3BTQVQKNFVtWHVrb2dJMlk4dU1pSUNSaXhiM1E4MHhoTnE2emtad2pmWDlpWlpLS3hxdHFWRVlNRkRKb2FpODh2aFJYYQo4WW1hclJnVEVGRVJrQ3oyY0w0aHovQVhjbGtEeU12djdtYlYwMFFKQW9jbEpMaXQzNm0xaHllbFQxeEIyN0luCkZwTWEreEZJMTU0RE5uUXNWa0VOcE1HWlR4Q294bEdCd1R3cFB1T3Z5bmN1WlJ0TkYvSlJqY1QzLzl4dEZSUVUKOXZjPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
      - --issuer=https://gitlab.arkinnovations.com
      - --client-id=b4f00aa85c57792698c28bf7d14bed82b2bec506c96bb8973049d493edc1f39c
      - --scopes=openid,email
      - --request-audience=b4f00aa85c57792698c28bf7d14bed82b2bec506c96bb8973049d493edc1f39c
      command: /usr/local/bin/pinniped
      env: []
      installHint: The pinniped CLI does not appear to be installed.  See https://get.pinniped.dev/cli
        for more details
      provideClusterInfo: true

Browser pop up for CLI Authentication.

$ export KUBECONFIG=~/.kube/ark-gitlab.yaml
 
$ kubectl get nodes
Authorize the account

Review Authorizations & save credentials:

$ kubectl get nodes
 
Log in by visiting this link:
 
    https://gitlab.arkinnovation.com/oauth/authorize?access_type=offline&client_id=b4f00aa85c57792698c28bf7d14bed82b2bec506c96bb8973049d493edc1f39c&code_challenge=0TbQn_ALFa58QTcHVMLnV7eWmI1t-zI2x2U3I1SEuW8&code_challenge_method=S256&nonce=f4875baf814944f1556ac06218cb48df&redirect_uri=http%3A%2F%2F127.0.0.1%3A58418%2Fcallback&response_type=code&scope=email+openid&state=fcf5352deffabb425ac487c02ae130b7
 
Error from server (Forbidden): nodes is forbidden: User "user@arkinnovation.com" cannot list resource "nodes" in API group "" at the cluster scope
$ pinniped whoami
Current cluster info:
 
Name: kubernetes-pinniped
URL: https://10.100.**.**.****
 
Current user info:
 
Username: user@arkinnovation.com
Groups: system:authenticated

Grant cluster role to admin user.

$ kubectl create clusterrolebinding gitlab-admin --clusterrole cluster-admin --user user@arkinnovation.com
clusterrolebinding.rbac.authorization.k8s.io/gitlab-admin created
$ kubectl get nodes
NAME           STATUS   ROLES                  AGE   VERSION
group_a_mst1   Ready    control-plane,master   21d   v1.23.6
group_a_mst2   Ready    control-plane,master   21d   v1.23.6
group_a_mst3   Ready    control-plane,master   21d   v1.23.6
group_a_wrk0   Ready    <none>                 21d   v1.23.6
group_a_wrk1   Ready    <none>                 21d   v1.23.6
group_a_wrk2   Ready    <none>                 21d   v1.23.6
group_a_wrk3   Ready    <none>                 21d   v1.23.6
group_a_1wrk4   Ready    <none>                 21d   v1.23.6
 
$ kubectl auth can-i '*' '*'
 
yes

Session File:

...
    scopes:
    - email
    - openid
  lastUsedTimestamp: "2022-10-01T11:24:34Z"
  tokens:
    id:
      claims:
        aud: b4f0c57792698c28bf7d14bed82b2bec506c96bb8973049d493edc1f39c
        auth_time: 1661440688
        email: user@arkinnovation.com
        email_verified: true
        exp: 1662031672
        groups:
        - Groups will be displayed here
        -**
        -  **
...
        groups_direct:
        - **
        - **
...
Authorization by group

Delete the existing JWT authenticator and recreate it.

$ kubectl delete JWTAuthenticator gitlab
 
$ kubectl apply -n pinniped-concierge -f-<<EOD
apiVersion: authentication.concierge.pinniped.dev/v1alpha1
kind: JWTAuthenticator
metadata:
  name: gitlab
spec:
  issuer: https://gitlab.arkinnovation.com
  audience: 103a4193c61234a3ecdddc647996c80186c3d95bf72c3507ec4e27e4
  claims:
    username: email
    groups: groups_direct
EOD

Regenerate the kube-config.

$ pinniped get kubeconfig --oidc-scopes openid,email --oidc-client-id 93a4193c61234a3ecdddc647996c80186c3d95bf72c3507ec4e27e4ea2d35928 > ~/.kube/ark-gitlab.yaml

Members of the group will be shown.

$ pinniped whoami
Current cluster info:
 
Name: kubernetes-pinniped
URL: https://10.100.**.**.****
 
Current user info:
 
Username: user@arkinnovation.com
Groups: "","" system:authenticated

Grant the GROUP_A the cluster role.

$ kubectl create clusterrolebinding gitlab-admin --clusterrole cluster-admin --group 'GROUP_A'
clusterrolebinding.rbac.authorization.k8s.io/gitlab-admin created

Testing:

$ kubectl get nodes
NAME           STATUS   ROLES                  AGE   VERSION
group_a_mst1   Ready    control-plane,master   21d   v1.23.6
group_a_mst2   Ready    control-plane,master   21d   v1.23.6
group_a_mst3   Ready    control-plane,master   21d   v1.23.6
group_a_wrk0   Ready    <none>                 21d   v1.23.6
group_a_wrk1   Ready    <none>                 21d   v1.23.6
group_a_wrk2   Ready    <none>                 21d   v1.23.6
group_a_wrk3   Ready    <none>                 21d   v1.23.6
group_a_1wrk4   Ready    <none>                 21d   v1.23.6
 
$ kubectl auth can-i '*' '*'
 
yes

Summary:

Compare to other K8’s OIDC authentication pinniped is quite easier because it will not allow kube-API server arguments. It leverages gitlab as IDP provider which allow multiple Active Directory Domain to authenticate.

If you liked this article, don’t forget to leave a clap and follow!

Kubernetes
Kubernetes Cluster
Authentication
Ark Innovations
K8s

--

--

Ark Innovations

Written by Ark Innovations

25 Followers

Innovation begins with us

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams