Managing Cluster Access and Permissions

Page last updated:

This topic describes how to grant Kubernetes cluster access and namespace permissions to users in Enterprise Pivotal Container Service (Enterprise PKS).

Overview

Enterprise PKS admin users can grant other users permissions to specific clusters by using the PKS CLI and kubectl.

If you are an Enterprise PKS admin user, you can do the following:

After you grant access, you must create a ClusterRoleBinding or a RoleBinding for the user you gave access to the cluster.

For more information, see RoleBinding and ClusterRoleBinding and Default Roles and Role Bindings in the Kubernetes documentation.

Prerequisites

Before setting up cluster access for users in Enterprise PKS, you must have the following:

  • Access to an Enterprise PKS admin user account. For information on how to create PKS admin users, see Managing Enterprise PKS Admin Users with UAA.
  • A PKS API fully qualified domain name (FQDN) of your PKS deployment.
  • A LDAP or SAML identity provider configured by the operator in the Enterprise PKS tile.

Grant Cluster Access to a User

After the cluster admin grants cluster access to end users, the Kubernetes end user can use the Kubernetes Command Line Interface (kubectl) to connect to the cluster. End users can only do actions that are permitted by the cluster admin. They cannot create, resize, or delete clusters.

Note: Before you can grant cluster access to Kubernetes end users, you must enable OpenID Connect (OIDC) by selecting Enable UAA as OIDC provider in Ops Manager Installation Dashboard > Enterprise PKS > Settings > UAA. After you enable OIDC, you must run pks get-credentials to update your existing kubeconfig file.

The following diagram outlines the workflow to grant cluster access to a user who belongs to an identity provider group:

This diagram shows the workflow that operators and cluster admins use to grant cluster access to users.

To grant cluster access to users, do the following:

  1. Log in to PKS by running following command:

    pks login -u USERNAME -p PASSWORD -a PKS-API --ca-cert CERT-PATH
    

    Where:

    • USERNAME is your cluster admin username. This is the username created for your organization’s LDAP or SAML identity provider.
    • PASSWORD is your cluster admin password.
    • PKS-API is the FQDN you use to access the PKS API.
    • CERT-PATH is the path to your root CA certificate. Provide the certificate to validate the PKS API certificate with SSL.

    Note: If your operator has configured Enterprise PKS to use a SAML identity provider, you must include an additional SSO flag to use the above command. For information about the SSO flags, see the section for the above command in PKS CLI. For information about configuring SAML, see Configure SAML as an Identity Provider in the Installing topic for your IaaS.

  2. Confirm that you can successfully connect to a cluster and use kubectl as a cluster admin by running the following command:

    pks get-credentials CLUSTER-NAME
    

    This step creates a ClusterRoleBinding for the cluster admin.

    Note: If your operator has configured Enterprise PKS to use a SAML identity provider, you must include an additional SSO flag to use the above command. For information about the SSO flags, see the section for the above command in PKS CLI. For information about configuring SAML, see Configure SAML as an Identity Provider in the Installing topic for your IaaS.

  3. When prompted, re-enter your password.

  4. Create a YAML file with either the Role or ClusterRole for your Kubernetes end user. Use the following example as a template:

    kind: ROLE-TYPE
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      namespace: NAMESPACE
      name: ROLE-OR-CLUSTER-ROLE-NAME
    rules:
    - apiGroups:
      resources: RESOURCE
      verbs: API-REQUEST-VERB
    

    Where:

    • ROLE-TYPE is the type of role you are creating. This must be either Role or ClusterRole.
    • NAMESPACE is the namespace within the cluster. This is omitted when creating a ClusterRole.
    • ROLE-OR-CLUSTER-ROLE-NAME is the name of the Role or ClusterRole you are creating. This name is created by the cluster admin.
    • RESOURCE is the resource you are granting access to. It must be specified in a comma-separated array. For example: ["pod-reader"]
    • API-REQUEST-VERB is the request verb used to specify resource requests. For more information, see Determine the Request Verb in the Kubernetes documentation.
  5. Create the Role or ClusterRole resource defined in your YAML file by running the following command:

    kubectl create -f ROLE-CONFIGURATION.yml
    

    Where ROLE-CONFIGURATION.yml is the YAML file you created in the above step.

  6. Create a YAML file containing either a ClusterRoleBinding or a RoleBinding for the Kubernetes end user. Use the following example as a template:

    kind: ROLE-BINDING-TYPE
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: ROLE-OR-CLUSTER-ROLE-BINDING-NAME
      namespace: NAMESPACE
    subjects:
    - kind: User
      name: USERNAME
      apiGroup: rbac.authorization.k8s.io
    roleRef:
      kind: ROLE-TYPE
      name: ROLE-OR-CLUSTER-ROLE-NAME
      apiGroup: rbac.authorization.k8s.io
    

    Where:

    • ROLE-BINDING-TYPE is the type of role binding you are creating. This must be either RoleBinding or ClusterRoleBinding.
    • ROLE-OR-CLUSTER-ROLE-BINDING-NAME is the name of the role binding. This is given by the cluster admin.
    • NAMESPACE is the namespace within the cluster. This is omitted when creating a ClusterRole.
    • USERNAME is the Kubernetes end user username. This is the username created for your organization’s LDAP or SAML identity provider.
      If you configured UAA OIDC Username Prefix in Ops Manager Installation Dashboard > Enterprise PKS > Settings > UAA, you must prepend USERNAME with the prefix you configured. For more information, see UAA in the Installing topic for your IaaS.
    • ROLE-TYPE is the type of role you created in the previous step. This must be either Role or ClusterRole.
    • ROLE-OR-CLUSTER-ROLE-NAME is the name of the Role or ClusterRole you are creating.
  7. Create the RoleBinding or ClusterRoleBinding resource defined in your YAML file by running following command:

    kubectl apply -f ROLE-BINDING-CONFIGURATION.yml
    

    Where ROLE-BINDING-CONFIGURATION.yml is the YAML file you created in the above step.

  8. Share the following information with your Kubernetes end users:

    • PKS API FQDN
    • Cluster name

Obtain Cluster Access as a User

To obtain cluster access, the end user must perform the following actions:

  1. Fetch the kubeconfig file by running one of the following command:

    • If you want to validate the PKS API certificate with SSL, run the following command:

      pks get-kubeconfig CLUSTER-NAME -u USERNAME -a PKS-API --ca-cert CERT-PATH
      

      Where:

      • CLUSTER-NAME is the cluster name provided by the cluster admin.
      • USERNAME is the Kubernetes end user username. This is the username created for your organization’s LDAP or SAML identity provider.
      • PKS-API is the FQDN you use to access the PKS API.
      • CERT-PATH is the path to your root CA certificate. Provide the certificate to validate the PKS API certificate with SSL.

      For example:

      $ pks get-kubeconfig my-cluster -u naomi -a api.pks.example.com \
        --ca-cert /var/tempest/workspaces/default/root_ca_certificate

    • If your CA is trusted and you want to skip SSL validation, run the following command:

      pks get-kubeconfig CLUSTER-NAME -u USERNAME -a PKS-API -k
      

      Where -k is the shortcut flag to skip SSL validation.

      For example:

      $ pks get-kubeconfig my-cluster -u naomi -a api.pks.example.com -k

    Note: If your operator has configured Enterprise PKS to use a SAML identity provider, you must include an additional SSO flag to use the above command. For information about the SSO flags, see the section for the above command in PKS CLI. For information about configuring SAML, see Configure SAML as an Identity Provider in the Installing topic for your IaaS.

  2. When prompted, enter your password.

  3. The PKS CLI generates a kubeconfig for the cluster you have access to. Review the following example kubeconfig file:

    apiVersion: v1
    clusters:
    - cluster:
        certificate-authority-data: PROVIDED-BY-ADMIN
        server: PROVIDED-BY-ADMIN
      name: PROVIDED-BY-ADMIN
    contexts:
    - context:
        cluster: PROVIDED-BY-ADMIN
        user: PROVIDED-BY-USER
      name:  PROVIDED-BY-ADMIN
    current-context: PROVIDED-BY-ADMIN
    kind: Config
    preferences: {}
    users:
    - name: PROVIDED-BY-USER
      user:
        auth-provider:
          config:
            client-id: pks_cluster_client
            cluster_client_secret: ""
            id-token: PROVIDED-BY-USER
            idp-issuer-url: https://PROVIDED-BY-ADMIN:8443/oauth/token
            refresh-token:  PROVIDED-BY-USER
          name: oidc
    
  4. Access the cluster using kubectl. For more information about kubectl commands, see Overview of kubectl in the Kubernetes documentation.

Grant Cluster Access to a Group

Cluster admins can grant access to an identity provider group by creating a ClusterRoleBinding or RoleBinding for that group. You can only grant access to an identity provider group if you use a LDAP or SAML identity provider for UAA. You can configure a LDAP or SAML identity provider in Ops Manager Installation Dashboard > Enterprise PKS > Settings > UAA.

Note: If you are using a LDAP group, you must confirm that the LDAP group you are giving access to has been whitelisted in the Enterprise PKS tile. To do this, review External Groups Whitelist in Ops Manager Installation Dashboard > Enterprise PKS > Settings > UAA.

The procedure for granting cluster access to an identity provider group is similar to the procedure in Grant Cluster Access to a User above.

To grant cluster access to an identity provider group, do the procedure in Grant Cluster Access to a User above and replace step 6 with the following:

  1. In the YAML file for a ClusterRoleBinding or a RoleBinding, replace the subjects section with the following:

    subjects:
    - kind: Group
      name: NAME-OF-GROUP
      apiGroup: rbac.authorization.k8s.io
    

    Use the following example as a template:

    kind: ROLE-BINDING-TYPE
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: ROLE-OR-CLUSTER-ROLE-BINDING-NAME
      namespace: NAMESPACE
    subjects:
    - kind: Group
      name: NAME-OF-GROUP
      apiGroup: rbac.authorization.k8s.io
    roleRef:
      kind: ROLE-TYPE
      name: ROLE-OR-CLUSTER-ROLE-NAME
      apiGroup: rbac.authorization.k8s.io
    

    Where:

    • ROLE-BINDING-TYPE is the type of role binding you are creating. This must be either RoleBinding or ClusterRoleBinding.
    • ROLE-OR-CLUSTER-ROLE-BINDING-NAME is the name of the role binding. This is given by the cluster admin.
    • NAMESPACE is the namespace within the cluster. This is omitted when creating a ClusterRole.
    • NAME-OF-GROUP is the identity provider group name. This name is case sensitive.
      If you configured UAA OIDC Groups Prefix in Ops Manager Installation Dashboard > Enterprise PKS > Settings > UAA, you must prepend NAME-OF-GROUP with the prefix you configured. For more information, see UAA in the Installing topic for your IaaS.
    • ROLE-TYPE is the type of role you created in the previous step. This must be either Role or ClusterRole.
    • ROLE-OR-CLUSTER-ROLE-NAME is the name of the Role or ClusterRole you are creating.

Please send any feedback you have to pks-feedback@pivotal.io.