Verifying Enterprise PKS Installation on vSphere with NSX-T

Prerequisites
Install the PKS and Kubectl CLIs
Manage User Accounts Using UAA
Manage PKS Using BOSH
Manage a Kubernetes Cluster using PKS and Kubernetes

Page last updated:

This topic provides general instructions for verifying the installation of an NSX‑T VM, either an NSX‑T Manager VM or an NSX‑T Edge Node VM, that you have deployed for use with Enterprise PKS.

Prerequisites

The following procedures must be executed from an SSH session into your Ops Manager. For information about creating an SSH keypair, see one of the following:

Generate an SSH keypair for installing Ops Manager v2.6 on vSphere in the Pivotal Knowledge Base
How to generate an SSH key pair for installing Ops Manager v2.6 on vSphere (71143) in the VMware Support Knowledge Base

Install the PKS and Kubectl CLIs

You will manage your environment using the Pivotal Container Service Command Line Interface (PKS CLI) and the Kubernetes CLI (kubectl). Complete the following steps to deploy the PKS and Kubernetes CLIs.

To copy CLIs to your Ops Manager using SCP, run the following PSCP commands:

pscp c:\temp\pks-linux-amd64-PKS-VERSION ubuntu@IP-ADDRESS:/home/ubuntu/pks
pscp c:\temp\kubectl-linux-amd64-STEMCELL-VERSION ubuntu@IP-ADDRESS:/home/ubuntu/pks

Where:

PKS-VERSION is your PKS build ID.
STEMCELL-VERSION is your stemcell version ID.
IP-ADDRESS is the IP Address for Ops Manager host.

For example:

$ pscp c:\temp\pks\pks-linux-amd64-1.5.0-build.291 ubuntu@10.197.79.152:/home/ubuntu/pks
$ pscp c:\temp\pks\kubectl-linux-amd64-1.14.5 ubuntu@10.197.79.152:/home/ubuntu/pks

To make the CLIs executable, run the following command:

chmod +x pks-linux-amd64-VERSION
chmod +x kubectl-linux-amd64-VERSION

Where:

PKS-VERSION is your PKS build ID.
STEMCELL-VERSION is your stemcell version ID.

For example:

$ chmod +x pks-linux-amd64-1.5.0-build.291
$ chmod +x kubectl-linux-amd64-1.14.5

To rename and move the files to your PATH, run the following commands:

sudo mv pks-linux-amd64-VERSION /usr/local/bin/pks
sudo mv kubectl-linux-amd64-VERSION /usr/local/bin/kubectl

Where:

PKS-VERSION is your PKS build ID.
STEMCELL-VERSION is your stemcell version ID.

For example:

$ sudo mv pks-linux-amd64-1.5.0-build.291 /usr/local/bin/pks
$ sudo mv kubectl-linux-amd64-1.14.5 /usr/local/bin/kubectl

Manage User Accounts Using UAA

You will manage accounts and authentication using User Account and Authentication (UAA) and the UAA Command Line Interface (UAAC).

Install UAAC and Configure UAA

Complete the following steps to deploy UAAC and UAA.

To install UAAC, run the following commands:

cd /usr/local/bin
sudo apt -y install ruby ruby-dev gcc build-essential g++
sudo gem install cf-uaac

To verify installation, run the following command:
```
uaac -v
```
To add PKS API hostname and IP address to a DNS server, complete the following steps:
1. Retrieve the DNS server IP address from vSphere.
2. Retrieve the PKS API Hostname from the PKS tile > PKS API > API Hostname (FQDN).
3. Add the DNS server IP address and the PKS API Hostname to the /etc/hosts file using the following format:
```
IP-ADDRESS HOSTNAME
```
  Where:
  - IP-ADDRESS is the DNS Server IP Address.
  - PKS-HOSTNAME is the PKS API Hostname.
  For example:
```
$ vi /etc/hosts
127.0.0.1 localhost
...
10.0.0.7 api.pks.vsphere.local
```

To target PKS, run the following UAAC command:

uaac target https://PKS-HOSTNAME:8443 --skip-ssl-validation

Where PKS-HOSTNAME is the PKS API Hostname.

For example:

$ uaac target https://api.comet.pks.local:8443 --skip-ssl-validation
Unknown key: Max-Age = 86400
Target: https://api.comet.pks.local:8443

To retrieve the UAA admin secret, complete the following steps:
1. Retrieve the token from Ops Manager: PKS Tile > Credentials > Pks Uaa Management Admin Client.
2. Click Link to Credential.
3. Copy the secret value.

To fetch the UAA admin token, run the following command:

uaac token client get admin -s SECRET

Where SECRET is the secret copied in the preceding step.

For example:

$ uaac token client get admin -s syMZA-FUlvdEXgXrgI-SDmkUVuP5rlzA
Unknown key: Max-Age = 86400
Successfully fetched token via client credentials grant.
Target: https://api.comet.pks.local:8443
Context: admin, from client admin

Add PKS Admin Users to UAA

Complete the following steps to add the pks-admin and k8s-admin admin accounts to UAA.

To add the PKS Admin user to UAA, run the following commands:

uaac user add pks-admin --emails ADMIN-EMAIL-ADDRESS -p PASSWORD
uaac member add pks.clusters.admin pks-admin

Where:

ADMIN-EMAIL-ADDRESS is the email address for the admin user account.
PASSWORD is the password for the user account.

For example:

$ uaac user add pks-admin --emails pks-admin@example.com -p aZ1by2cX3d4! user account successfully added

$ uaac member add pks.clusters.admin pks-admin success

To add the PKS Manage user to UAA, run the following commands:

uaac user add k8s-admin --emails MANAGE-EMAIL-ADDRESS -p PASSWORD
uaac member add pks.clusters.manage k8s-admin

Where:

MANAGE-EMAIL-ADDRESS is the email address for the PKS Manage user account.
PASSWORD is the password for the user account.

For example:

$ uaac user add k8s-admin --emails k8s-admin@example.com -p aZ1by2cX3d4! user account successfully added

$ uaac member add pks.clusters.manage k8s-admin success

Log In to PKS

Validate the new UAA admin account by logging in to PKS.

To log in to PKS as pks-admin run the following command:

pks login -a "PKS-HOSTNAME" -u "pks-admin" -p "PASSWORD" -k

Where:

PKS-HOSTNAME is the PKS API Hostname.
PASSWORD is the password for the user account.

For example:

$ pks login -a "api.pks.vsphere.local" -u "pks-admin" -p "aE1ry2cJ3d2!" -k


API Endpoint: api.pks.vsphere.local
User: pks-admin
Login successful.

To log in to PKS as k8s-admin run the following command:

pks login -a "PKS-HOSTNAME" -u "k8s-admin" -p "PASSWORD" -k

Where:

PKS-HOSTNAME is the PKS API Hostname.
PASSWORD is the password for the user account.

For example:

$ pks login -a "api.comet.pks.local" -u "k8s-admin" -p "rtXare1!" -k

API Endpoint: api.comet.pks.local User: k8s-admin Login successful.

Manage PKS Using BOSH

Create a BOSH Alias

To create a BOSH alias, complete one of the following:
- To create a BOSH alias using the alias command, complete the following:
  1. Go to BOSH Director tile > Credentials > Bosh Commandline Credentials.
  2. Click Link to Credential.
  3. Copy the credential string, including the surrounding double quotes.
  4. To create a BOSH alias run the following command:
```
alias bosh=CREDENTIAL-STRING  
```
    Where:
    - CREDENTIAL-STRING is the entire credential string from the Bosh Commandline Credentials, including the surrounding double quotes.
    For example:
```
$ alias bosh="BOSH_CLIENT=ops_manager BOSH_CLIENT_SECRET=sNc1itad BOSH_CA_CERT=/var/tempest/workspaces/default/root_ca_certificate BOSH_ENVIRONMENT=ip-from-bosh_commandline_credentials bosh"
```
- To create a BOSH alias using the source command, complete the following:
  1. Go to BOSH Director tile > Credentials > Bosh Commandline Credentials.
  2. Click Link to Credential.
  3. Copy the credential string content.
  4. Create a file named bosh_evn.sh with the following modified credential string content:
```
export BOSH_CLIENT=BOSH-CLIENT
export BOSH_CLIENT_SECRET=BOSH-CLIENT-SECRET
export BOSH_CA_CERT=BOSH-CA-CERT
export BOSH_ENVIRONMENT=BOSH-ENVIRONMENT-IP
alias om=om-linux
```
    Where:
    - BOSH-CLIENT is the BOSH_CLIENT value from the BOSH command line credentials.
    - BOSH-CLIENT-SECRET is the BOSH_CLIENT_SECRET value from the BOSH command line credentials.
    - BOSH-CA-CERT is the BOSH_CA_CERT path value from the BOSH command line credentials.
    - IP-ADDRESS-OF-BOSH is the BOSH_ENVIRONMENT IP Address value from the BOSH command line credentials.
    For example:
```
export BOSH_CLIENT=ops_manager 
export BOSH_CLIENT_SECRET=sgFGsM_cOPrw5IhrIeIX1JDFxPR6LcgR 
export BOSH_CA_CERT=/var/tempest/workspaces/default/root_ca_certificate 
export BOSH_ENVIRONMENT=10.0.16.5 
```
  5. Run the following command:
```
source bosh_env.sh
```

List BOSH Deployments

To retrieve a list of BOSH deployments, run the following command:
```
bosh deployments
```

List BOSH VMs

To retrieve a list of BOSH VMs, run the following command:
```
bosh vms
```

List BOSH Components

To retrieve a list of bosh components. run the following command:
```
bosh releases
```

Use BOSH to SSH to a Kubernetes Node

To SSH to a Kubernetes node, run the following command:
```
bosh ssh -d 'SERVICE-ID' DESTINATION-ID
```
Where:
- SERVICE-ID is your node’s service instance id.
- DESTINATION-ID is your destination host’s ID. For example, you can provide an ID such as a master host, UAA host or Diego Cell ID as a destination. For more information about bosh ssh, see SSH in the Cloud Foundry BOSH Commands documentation.
For example:
```
$ bosh ssh -d 'service-instance_9dd62330-fcea-469f-a50d-27c746622662' master/b1f99643-6905-4427-a01c-c26fc9101d40
```

Use BOSH to Delete a Kubernetes Cluster

If you want to delete a Kubernetes cluster that failed to deploy, and you cannot delete the cluster using the PKS CLI, you can delete the cluster using BOSH.

To delete a Kubernetes cluster, run the following command:

bosh -e pks delete-deployment -d SERVICE-ID

Where SERVICE-ID is your node’s service instance id.

For example:

$ bosh -e pks delete-deployment -d service-instance_3bd9069f-aafd-4525-a145-f3e48570c055

Manage a Kubernetes Cluster using PKS and Kubernetes

You can manage your Kubernetes clusters using both PKS and Kubernetes command line commands.

Retrieve Cluster Credentials

To retrieve your Kubernetes cluster credentials, run the following:
```
pks get-credentials CLUSTER-NAME
```
Where CLUSTER-NAME is the name of your cluster.

Validate the Kubernetes Configuration File

To verify that the Kubernetes configuration file exists, run the following command:
```
ls -l ~/.kube/config
```

Connect to Kubernetes Cluster

To manage a Kubernetes cluster using Kubernetes CLI commands, you must first connect to the cluster.

Retrieve your Kubernetes cluster credentials. For more information, see Retrieve Cluster Credentials above.
Verify that the Kubernetes configuration file exists. For more information, see Validate the Kubernetes Configuration File above.
To create a DNS record for the Kubernetes cluster and master node IP address, complete the following steps:
1. Run the following command to get the information:
```
pks cluster CLUSTER-NAME
```
2. Locate the “Kubernetes Master Host” and “Kubernetes Master IP(s)” values.
3. To create a DNS record, create or modify the /etc/hosts file to have a new line using these values:
```
MASTER-IP MASTER-HOST
```
  Where:
  - MASTER-IP is the Kubernetes Master IP(s) collected in the preceding step.
  - MASTER-HOST is the Kubernetes Master Host collected in the preceding step.
  For example:
```
$ vi /etc/hosts
10.197.100.130 tea-cluster
```
To connect to the Kubernetes cluster, run the following command:
```
kubectl config use-context CLUSTER-NAME
```
To confirm your connection list your cluster nodes. For more information see List Kubernetes Cluster Nodes below.

List Kubernetes Cluster Nodes

To list the cluster nodes of the currently connected Kubernetes cluster, run the following command:
```
kubectl get nodes
```

Please send any feedback you have to pks-feedback@pivotal.io.

Create a pull request or raise an issue on the source for this page in GitHub