Verifying Enterprise PKS Installation on vSphere with NSX-T

Page last updated:

This topic provides general instructions for verifying the installation of an NSX‑T VM, either an NSX‑T Manager VM or an NSX‑T Edge Node VM, that you have deployed for use with Enterprise PKS.

Prerequisites

The following procedures must be executed from an SSH session into your Ops Manager. For information about creating an SSH keypair, see one of the following:

Install the PKS and Kubectl CLIs

You will manage your environment using the Pivotal Container Service Command Line Interface (PKS CLI) and the Kubernetes CLI (kubectl). Complete the following steps to deploy the PKS and Kubernetes CLIs.

  1. To copy CLIs to your Ops Manager using SCP, run the following PSCP commands:

    pscp c:\temp\pks-linux-amd64-PKS-VERSION ubuntu@IP-ADDRESS:/home/ubuntu/pks
    pscp c:\temp\kubectl-linux-amd64-STEMCELL-VERSION ubuntu@IP-ADDRESS:/home/ubuntu/pks
    

    Where:

    • PKS-VERSION is your PKS build ID.
    • STEMCELL-VERSION is your stemcell version ID.
    • IP-ADDRESS is the IP Address for Ops Manager host.

    For example:

    $ pscp c:\temp\pks\pks-linux-amd64-1.5.0-build.291 ubuntu@10.197.79.152:/home/ubuntu/pks
    $ pscp c:\temp\pks\kubectl-linux-amd64-1.14.5 ubuntu@10.197.79.152:/home/ubuntu/pks
    

  2. To make the CLIs executable, run the following command:

    chmod +x pks-linux-amd64-VERSION
    chmod +x kubectl-linux-amd64-VERSION
    

    Where:

    • PKS-VERSION is your PKS build ID.
    • STEMCELL-VERSION is your stemcell version ID.

    For example:

    $ chmod +x pks-linux-amd64-1.5.0-build.291
    $ chmod +x kubectl-linux-amd64-1.14.5
    

  3. To rename and move the files to your PATH, run the following commands:

    sudo mv pks-linux-amd64-VERSION /usr/local/bin/pks
    sudo mv kubectl-linux-amd64-VERSION /usr/local/bin/kubectl
    

    Where:

    • PKS-VERSION is your PKS build ID.
    • STEMCELL-VERSION is your stemcell version ID.

    For example:

    $ sudo mv pks-linux-amd64-1.5.0-build.291 /usr/local/bin/pks
    $ sudo mv kubectl-linux-amd64-1.14.5 /usr/local/bin/kubectl
    

Manage User Accounts Using UAA

You will manage accounts and authentication using User Account and Authentication (UAA) and the UAA Command Line Interface (UAAC).

Install UAAC and Configure UAA

Complete the following steps to deploy UAAC and UAA.

  1. To install UAAC, run the following commands:

    cd /usr/local/bin
    sudo apt -y install ruby ruby-dev gcc build-essential g++
    sudo gem install cf-uaac
    
  2. To verify installation, run the following command:

    uaac -v
    
  3. To add PKS API hostname and IP address to a DNS server, complete the following steps:

    1. Retrieve the DNS server IP address from vSphere.
    2. Retrieve the PKS API Hostname from the PKS tile > PKS API > API Hostname (FQDN).
    3. Add the DNS server IP address and the PKS API Hostname to the /etc/hosts file using the following format:

      IP-ADDRESS HOSTNAME
      

      Where:

      • IP-ADDRESS is the DNS Server IP Address.
      • PKS-HOSTNAME is the PKS API Hostname.

      For example:

      $ vi /etc/hosts
      127.0.0.1 localhost
      ...
      10.0.0.7 api.pks.vsphere.local
      

  4. To target PKS, run the following UAAC command:

    uaac target https://PKS-HOSTNAME:8443 --skip-ssl-validation
    

    Where PKS-HOSTNAME is the PKS API Hostname.

    For example:

    $ uaac target https://api.comet.pks.local:8443 --skip-ssl-validation
    Unknown key: Max-Age = 86400
    Target: https://api.comet.pks.local:8443
    

  5. To retrieve the UAA admin secret, complete the following steps:

    1. Retrieve the token from Ops Manager: PKS Tile > Credentials > Pks Uaa Management Admin Client.
    2. Click Link to Credential.
    3. Copy the secret value.
  6. To fetch the UAA admin token, run the following command:

    uaac token client get admin -s SECRET
    

    Where SECRET is the secret copied in the preceding step.

    For example:

    $ uaac token client get admin -s syMZA-FUlvdEXgXrgI-SDmkUVuP5rlzA
    Unknown key: Max-Age = 86400
    Successfully fetched token via client credentials grant.
    Target: https://api.comet.pks.local:8443
    Context: admin, from client admin
    

Add PKS Admin Users to UAA

Complete the following steps to add the pks-admin and k8s-admin admin accounts to UAA.

  1. To add the PKS Admin user to UAA, run the following commands:

    uaac user add pks-admin --emails ADMIN-EMAIL-ADDRESS -p PASSWORD
    uaac member add pks.clusters.admin pks-admin
    

    Where:

    • ADMIN-EMAIL-ADDRESS is the email address for the admin user account.
    • PASSWORD is the password for the user account.

    For example:

    $ uaac user add pks-admin --emails pks-admin@example.com -p aZ1by2cX3d4!
    user account successfully added 

    $ uaac member add pks.clusters.admin pks-admin success

  2. To add the PKS Manage user to UAA, run the following commands:

    uaac user add k8s-admin --emails MANAGE-EMAIL-ADDRESS -p PASSWORD
    uaac member add pks.clusters.manage k8s-admin
    

    Where:

    • MANAGE-EMAIL-ADDRESS is the email address for the PKS Manage user account.
    • PASSWORD is the password for the user account.

    For example:

    $ uaac user add k8s-admin --emails k8s-admin@example.com -p aZ1by2cX3d4!
    user account successfully added

    $ uaac member add pks.clusters.manage k8s-admin success

Log In to PKS

Validate the new UAA admin account by logging in to PKS.

  1. To log in to PKS as pks-admin run the following command:

    pks login -a "PKS-HOSTNAME" -u "pks-admin" -p "PASSWORD" -k
    

    Where:

    • PKS-HOSTNAME is the PKS API Hostname.
    • PASSWORD is the password for the user account.

    For example:

    $ pks login -a "api.pks.vsphere.local" -u "pks-admin" -p "aE1ry2cJ3d2!" -k
    
    API Endpoint: api.pks.vsphere.local User: pks-admin Login successful.

  2. To log in to PKS as k8s-admin run the following command:

    pks login -a "PKS-HOSTNAME" -u "k8s-admin" -p "PASSWORD" -k
    

    Where:

    • PKS-HOSTNAME is the PKS API Hostname.
    • PASSWORD is the password for the user account.

    For example:

    $ pks login -a "api.comet.pks.local" -u "k8s-admin" -p "rtXare1!" -k

    API Endpoint: api.comet.pks.local User: k8s-admin Login successful.

Manage PKS Using BOSH

Create a BOSH Alias

  1. To create a BOSH alias, complete one of the following:

    • To create a BOSH alias using the alias command, complete the following:

      1. Go to BOSH Director tile > Credentials > Bosh Commandline Credentials.
      2. Click Link to Credential.
      3. Copy the credential string, including the surrounding double quotes.
      4. To create a BOSH alias run the following command:

        alias bosh=CREDENTIAL-STRING  
        

        Where:

        • CREDENTIAL-STRING is the entire credential string from the Bosh Commandline Credentials, including the surrounding double quotes.

        For example:

        $ alias bosh="BOSH_CLIENT=ops_manager BOSH_CLIENT_SECRET=sNc1itad BOSH_CA_CERT=/var/tempest/workspaces/default/root_ca_certificate BOSH_ENVIRONMENT=ip-from-bosh_commandline_credentials bosh"
        

    • To create a BOSH alias using the source command, complete the following:

      1. Go to BOSH Director tile > Credentials > Bosh Commandline Credentials.
      2. Click Link to Credential.
      3. Copy the credential string content.
      4. Create a file named bosh_evn.sh with the following modified credential string content:

        export BOSH_CLIENT=BOSH-CLIENT
        export BOSH_CLIENT_SECRET=BOSH-CLIENT-SECRET
        export BOSH_CA_CERT=BOSH-CA-CERT
        export BOSH_ENVIRONMENT=BOSH-ENVIRONMENT-IP
        alias om=om-linux
        

        Where:

        • BOSH-CLIENT is the BOSH_CLIENT value from the BOSH command line credentials.
        • BOSH-CLIENT-SECRET is the BOSH_CLIENT_SECRET value from the BOSH command line credentials.
        • BOSH-CA-CERT is the BOSH_CA_CERT path value from the BOSH command line credentials.
        • IP-ADDRESS-OF-BOSH is the BOSH_ENVIRONMENT IP Address value from the BOSH command line credentials.

        For example:

        export BOSH_CLIENT=ops_manager 
        export BOSH_CLIENT_SECRET=sgFGsM_cOPrw5IhrIeIX1JDFxPR6LcgR 
        export BOSH_CA_CERT=/var/tempest/workspaces/default/root_ca_certificate 
        export BOSH_ENVIRONMENT=10.0.16.5 
        
      5. Run the following command:

        source bosh_env.sh
        

List BOSH Deployments

  1. To retrieve a list of BOSH deployments, run the following command:

    bosh deployments
    

List BOSH VMs

  1. To retrieve a list of BOSH VMs, run the following command:

    bosh vms
    

List BOSH Components

  1. To retrieve a list of bosh components. run the following command:

    bosh releases
    

Use BOSH to SSH to a Kubernetes Node

  1. To SSH to a Kubernetes node, run the following command:

    bosh ssh -d 'SERVICE-ID' DESTINATION-ID
    

    Where:

    • SERVICE-ID is your node’s service instance id.
    • DESTINATION-ID is your destination host’s ID. For example, you can provide an ID such as a master host, UAA host or Diego Cell ID as a destination. For more information about bosh ssh, see SSH in the Cloud Foundry BOSH Commands documentation.

    For example:

    $ bosh ssh -d 'service-instance_9dd62330-fcea-469f-a50d-27c746622662' master/b1f99643-6905-4427-a01c-c26fc9101d40
    

Use BOSH to Delete a Kubernetes Cluster

If you want to delete a Kubernetes cluster that failed to deploy, and you cannot delete the cluster using the PKS CLI, you can delete the cluster using BOSH.

  1. To delete a Kubernetes cluster, run the following command:

    bosh -e pks delete-deployment -d SERVICE-ID
    

    Where SERVICE-ID is your node’s service instance id.

    For example:

    $ bosh -e pks delete-deployment -d service-instance_3bd9069f-aafd-4525-a145-f3e48570c055
    

Manage a Kubernetes Cluster using PKS and Kubernetes

You can manage your Kubernetes clusters using both PKS and Kubernetes command line commands.

Retrieve Cluster Credentials

  1. To retrieve your Kubernetes cluster credentials, run the following:

    pks get-credentials CLUSTER-NAME
    

    Where CLUSTER-NAME is the name of your cluster.

Validate the Kubernetes Configuration File

  1. To verify that the Kubernetes configuration file exists, run the following command:

    ls -l ~/.kube/config
    

Connect to Kubernetes Cluster

To manage a Kubernetes cluster using Kubernetes CLI commands, you must first connect to the cluster.

  1. Retrieve your Kubernetes cluster credentials. For more information, see Retrieve Cluster Credentials above.
  2. Verify that the Kubernetes configuration file exists. For more information, see Validate the Kubernetes Configuration File above.
  3. To create a DNS record for the Kubernetes cluster and master node IP address, complete the following steps:

    1. Run the following command to get the information:

      pks cluster CLUSTER-NAME
      
    2. Locate the “Kubernetes Master Host” and “Kubernetes Master IP(s)” values.

    3. To create a DNS record, create or modify the /etc/hosts file to have a new line using these values:

      MASTER-IP MASTER-HOST
      

      Where:

      • MASTER-IP is the Kubernetes Master IP(s) collected in the preceding step.
      • MASTER-HOST is the Kubernetes Master Host collected in the preceding step.

      For example:

      $ vi /etc/hosts
      10.197.100.130 tea-cluster
      

  4. To connect to the Kubernetes cluster, run the following command:

    kubectl config use-context CLUSTER-NAME
    
  5. To confirm your connection list your cluster nodes. For more information see List Kubernetes Cluster Nodes below.

List Kubernetes Cluster Nodes

  1. To list the cluster nodes of the currently connected Kubernetes cluster, run the following command:

    kubectl get nodes
    

Please send any feedback you have to pks-feedback@pivotal.io.