Verifying Enterprise PKS Installation on vSphere with NSX-T
Page last updated:
Warning: VMware Enterprise PKS v1.6 is no longer supported because it has reached the End of General Support (EOGS) phase as defined by the Support Lifecycle Policy. To stay up to date with the latest software and security updates, upgrade to a supported version.
This topic provides general instructions for verifying the installation of an NSX‑T VM, either an NSX‑T Manager VM or an NSX‑T Edge Node VM, that you have deployed for use with Enterprise PKS.
Prerequisites
The following procedures must be executed from an SSH session into your Ops Manager. For information about creating an SSH keypair, see one of the following:
- Generate an SSH keypair for installing Ops Manager v2.6 on vSphere in the Pivotal Knowledge Base
- How to generate an SSH key pair for installing Ops Manager v2.6 on vSphere (71143) in the VMware Support Knowledge Base
Install the PKS and Kubectl CLIs
You will manage your environment using the Pivotal Container Service Command Line Interface (PKS CLI) and the Kubernetes CLI (kubectl). Complete the following steps to deploy the PKS and Kubernetes CLIs.
To copy CLIs to your Ops Manager using SCP, run the following PSCP commands:
pscp c:\temp\pks-linux-amd64-PKS-VERSION ubuntu@IP-ADDRESS:/home/ubuntu/pks pscp c:\temp\kubectl-linux-amd64-STEMCELL-VERSION ubuntu@IP-ADDRESS:/home/ubuntu/pks
Where:
PKS-VERSION
is your PKS build ID.STEMCELL-VERSION
is your stemcell version ID.IP-ADDRESS
is the IP Address for Ops Manager host.
For example:
$ pscp c:\temp\pks\pks-linux-amd64-1.5.0-build.291 ubuntu@10.197.79.152:/home/ubuntu/pks $ pscp c:\temp\pks\kubectl-linux-amd64-1.14.5 ubuntu@10.197.79.152:/home/ubuntu/pks
To make the CLIs executable, run the following command:
chmod +x pks-linux-amd64-VERSION chmod +x kubectl-linux-amd64-VERSION
Where:
PKS-VERSION
is your PKS build ID.STEMCELL-VERSION
is your stemcell version ID.
For example:
$ chmod +x pks-linux-amd64-1.5.0-build.291 $ chmod +x kubectl-linux-amd64-1.14.5
To rename and move the files to your PATH, run the following commands:
sudo mv pks-linux-amd64-VERSION /usr/local/bin/pks sudo mv kubectl-linux-amd64-VERSION /usr/local/bin/kubectl
Where:
PKS-VERSION
is your PKS build ID.STEMCELL-VERSION
is your stemcell version ID.
For example:
$ sudo mv pks-linux-amd64-1.5.0-build.291 /usr/local/bin/pks $ sudo mv kubectl-linux-amd64-1.14.5 /usr/local/bin/kubectl
Manage User Accounts Using UAA
You will manage accounts and authentication using User Account and Authentication (UAA) and the UAA Command Line Interface (UAAC).
Install UAAC and Configure UAA
Complete the following steps to deploy UAAC and UAA.
To install UAAC, run the following commands:
cd /usr/local/bin sudo apt -y install ruby ruby-dev gcc build-essential g++ sudo gem install cf-uaac
To verify installation, run the following command:
uaac -v
To add PKS API hostname and IP address to a DNS server, complete the following steps:
- Retrieve the DNS server IP address from vSphere.
- Retrieve the PKS API Hostname from the PKS tile > PKS API > API Hostname (FQDN).
Add the DNS server IP address and the PKS API Hostname to the
/etc/hosts
file using the following format:IP-ADDRESS HOSTNAME
Where:
IP-ADDRESS
is the DNS Server IP Address.PKS-HOSTNAME
is the PKS API Hostname.
For example:
$ vi /etc/hosts 127.0.0.1 localhost ... 10.0.0.7 api.pks.vsphere.local
- Retrieve the DNS server IP address from vSphere.
To target PKS, run the following UAAC command:
uaac target https://PKS-HOSTNAME:8443 --skip-ssl-validation
Where
PKS-HOSTNAME
is the PKS API Hostname.For example:
$ uaac target https://api.comet.pks.local:8443 --skip-ssl-validation Unknown key: Max-Age = 86400 Target: https://api.comet.pks.local:8443
To retrieve the UAA admin secret, complete the following steps:
- Retrieve the token from Ops Manager: PKS Tile > Credentials > Pks Uaa Management Admin Client.
- Click Link to Credential.
- Copy the
secret
value.
- Retrieve the token from Ops Manager: PKS Tile > Credentials > Pks Uaa Management Admin Client.
To fetch the UAA admin token, run the following command:
uaac token client get admin -s SECRET
Where
SECRET
is the secret copied in the preceding step.For example:
$ uaac token client get admin -s syMZA-FUlvdEXgXrgI-SDmkUVuP5rlzA Unknown key: Max-Age = 86400 Successfully fetched token via client credentials grant. Target: https://api.comet.pks.local:8443 Context: admin, from client admin
Add PKS Admin Users to UAA
Complete the following steps to add the pks-admin and k8s-admin admin accounts to UAA.
To add the PKS Admin user to UAA, run the following commands:
uaac user add pks-admin --emails ADMIN-EMAIL-ADDRESS -p PASSWORD uaac member add pks.clusters.admin pks-admin
Where:
ADMIN-EMAIL-ADDRESS
is the email address for the admin user account.PASSWORD
is the password for the user account.
For example:
$ uaac user add pks-admin --emails pks-admin@example.com -p aZ1by2cX3d4! user account successfully added
$ uaac member add pks.clusters.admin pks-admin success
To add the PKS Manage user to UAA, run the following commands:
uaac user add k8s-admin --emails MANAGE-EMAIL-ADDRESS -p PASSWORD uaac member add pks.clusters.manage k8s-admin
Where:
MANAGE-EMAIL-ADDRESS
is the email address for the PKS Manage user account.PASSWORD
is the password for the user account.
For example:
$ uaac user add k8s-admin --emails k8s-admin@example.com -p aZ1by2cX3d4! user account successfully added
$ uaac member add pks.clusters.manage k8s-admin success
Log In to PKS
Validate the new UAA admin account by logging in to PKS.
To log in to PKS as
pks-admin
run the following command:pks login -a "PKS-HOSTNAME" -u "pks-admin" -p "PASSWORD" -k
Where:
PKS-HOSTNAME
is the PKS API Hostname.PASSWORD
is the password for the user account.
For example:
$ pks login -a "api.pks.vsphere.local" -u "pks-admin" -p "aE1ry2cJ3d2!" -k
API Endpoint: api.pks.vsphere.local User: pks-admin Login successful.To log in to PKS as
k8s-admin
run the following command:pks login -a "PKS-HOSTNAME" -u "k8s-admin" -p "PASSWORD" -k
Where:
PKS-HOSTNAME
is the PKS API Hostname.PASSWORD
is the password for the user account.
For example:
$ pks login -a "api.comet.pks.local" -u "k8s-admin" -p "rtXare1!" -k
API Endpoint: api.comet.pks.local User: k8s-admin Login successful.
Manage PKS Using BOSH
Create a BOSH Alias
To create a BOSH alias, complete one of the following:
To create a BOSH alias using the
alias
command, complete the following:- Go to BOSH Director tile > Credentials > Bosh Commandline Credentials.
- Click Link to Credential.
- Copy the
credential
string, including the surrounding double quotes. To create a BOSH alias run the following command:
alias bosh=CREDENTIAL-STRING
Where:
CREDENTIAL-STRING
is the entirecredential
string from the Bosh Commandline Credentials, including the surrounding double quotes.
For example:
$ alias bosh="BOSH_CLIENT=ops_manager BOSH_CLIENT_SECRET=sNc1itad BOSH_CA_CERT=/var/tempest/workspaces/default/root_ca_certificate BOSH_ENVIRONMENT=ip-from-bosh_commandline_credentials bosh"
To create a BOSH alias using the
source
command, complete the following:- Go to BOSH Director tile > Credentials > Bosh Commandline Credentials.
- Click Link to Credential.
- Copy the
credential
string content. Create a file named
bosh_evn.sh
with the following modifiedcredential
string content:export BOSH_CLIENT=BOSH-CLIENT export BOSH_CLIENT_SECRET=BOSH-CLIENT-SECRET export BOSH_CA_CERT=BOSH-CA-CERT export BOSH_ENVIRONMENT=BOSH-ENVIRONMENT-IP alias om=om-linux
Where:
BOSH-CLIENT
is theBOSH_CLIENT
value from the BOSH command line credentials.BOSH-CLIENT-SECRET
is theBOSH_CLIENT_SECRET
value from the BOSH command line credentials.BOSH-CA-CERT
is theBOSH_CA_CERT
path value from the BOSH command line credentials.IP-ADDRESS-OF-BOSH
is theBOSH_ENVIRONMENT
IP Address value from the BOSH command line credentials.
For example:
export BOSH_CLIENT=ops_manager export BOSH_CLIENT_SECRET=sgFGsM_cOPrw5IhrIeIX1JDFxPR6LcgR export BOSH_CA_CERT=/var/tempest/workspaces/default/root_ca_certificate export BOSH_ENVIRONMENT=10.0.16.5
Run the following command:
source bosh_env.sh
List BOSH Deployments
To retrieve a list of BOSH deployments, run the following command:
bosh deployments
List BOSH VMs
To retrieve a list of BOSH VMs, run the following command:
bosh vms
List BOSH Components
To retrieve a list of bosh components. run the following command:
bosh releases
Use BOSH to SSH to a Kubernetes Node
To SSH to a Kubernetes node, run the following command:
bosh ssh -d 'SERVICE-ID' DESTINATION-ID
Where:
SERVICE-ID
is your node’s service instance id.DESTINATION-ID
is your destination host’s ID. For example, you can provide an ID such as a master host, UAA host or Diego Cell ID as a destination. For more information aboutbosh ssh
, see SSH in the Cloud Foundry BOSH Commands documentation.
For example:
$ bosh ssh -d 'service-instance_9dd62330-fcea-469f-a50d-27c746622662' master/b1f99643-6905-4427-a01c-c26fc9101d40
Use BOSH to Delete a Kubernetes Cluster
If you want to delete a Kubernetes cluster that failed to deploy, and you cannot delete the cluster using the PKS CLI, you can delete the cluster using BOSH.
To delete a Kubernetes cluster, run the following command:
bosh -e pks delete-deployment -d SERVICE-ID
Where
SERVICE-ID
is your node’s service instance id.For example:
$ bosh -e pks delete-deployment -d service-instance_3bd9069f-aafd-4525-a145-f3e48570c055
Manage a Kubernetes Cluster using PKS and Kubernetes
You can manage your Kubernetes clusters using both PKS and Kubernetes command line commands.
Retrieve Cluster Credentials
To retrieve your Kubernetes cluster credentials, run the following:
pks get-credentials CLUSTER-NAME
Where
CLUSTER-NAME
is the name of your cluster.
Validate the Kubernetes Configuration File
To verify that the Kubernetes configuration file exists, run the following command:
ls -l ~/.kube/config
Connect to Kubernetes Cluster
To manage a Kubernetes cluster using Kubernetes CLI commands, you must first connect to the cluster.
- Retrieve your Kubernetes cluster credentials. For more information, see Retrieve Cluster Credentials above.
- Verify that the Kubernetes configuration file exists. For more information, see Validate the Kubernetes Configuration File above.
To create a DNS record for the Kubernetes cluster and master node IP address, complete the following steps:
Run the following command to get the information:
pks cluster CLUSTER-NAME
Locate the “Kubernetes Master Host” and “Kubernetes Master IP(s)” values.
To create a DNS record, create or modify the
/etc/hosts
file to have a new line using these values:MASTER-IP MASTER-HOST
Where:
MASTER-IP
is the Kubernetes Master IP(s) collected in the preceding step.MASTER-HOST
is the Kubernetes Master Host collected in the preceding step.
For example:
$ vi /etc/hosts 10.197.100.130 tea-cluster
To connect to the Kubernetes cluster, run the following command:
kubectl config use-context CLUSTER-NAME
To confirm your connection list your cluster nodes. For more information see List Kubernetes Cluster Nodes below.
List Kubernetes Cluster Nodes
To list the cluster nodes of the currently connected Kubernetes cluster, run the following command:
kubectl get nodes
Please send any feedback you have to pks-feedback@pivotal.io.