Upgrading Tanzu Kubernetes Grid Integrated Edition (NSX-T Networking)
Note: As of v1.8, Enterprise PKS has been renamed to VMware Tanzu Kubernetes Grid Integrated Edition. Some screenshots in this documentation do not yet reflect the change.
Page last updated:
This topic explains how to upgrade Tanzu Kubernetes Grid Integrated Edition (TKGI) from v1.7 to v1.8 on vSphere with NSX-T networking.
For instructions on upgrading TKGI with Flannel networking, see Upgrading Tanzu Kubernetes Grid Integrated Edition (Flannel Networking).
Warning: Do not manually upgrade your Kubernetes version. TKGI includes the compatible Kubernetes version.
Before you upgrade, follow the procedures in Prepare to Upgrade below to plan and prepare your upgrade.
After you complete the preparation steps, continue to the procedures in Perform the Upgrade below. These steps guide you through the process of upgrading Ops Manager and the TKGI tile, importing an updated stemcell, and applying the changes to your deployment.
After you complete the upgrade, follow the procedures in After the Upgrade below to verify that your upgraded TKGI deployment is running properly and to optionally upgrade NSX-T and vSphere.
TKGI v1.8 does not support NSX-T v2.4. You must upgrade to NSX-T v2.5 before upgrading to TKGI v1.8.
To see a list of NSX-T 2.5 versions compatible with TKGI v1.8, consult Product Snapshot in Release Notes for TKGI v1.8.
If you have not already, complete all of the steps in Upgrade Preparation Checklist for Tanzu Kubernetes Grid Integrated Edition v1.8.
This section describes the steps required to upgrade to TKGI v1.8:
- Upgrade Ops Manager
- Download and Import TKGI v1.8
- Download and Import the Stemcell if Needed
- Upgrade the TKGI Tile
Each version of TKGI is compatible with multiple versions of Ops Manager. See VMware Tanzu Network to determine if your Ops Manager version is compatible with TKGI v1.8.
To upgrade Ops Manager from v2.8.5 or later to v2.9.3 or later:
Log in to Ops Manager.
Click your username in the top right corner and navigate to Settings > Export Installation Settings.
Click Export Installation Settings.
- Ops Manager exports an encrypted archive of your current installation configuration.
- Later, you import this configuration into to your upgraded Ops Manager.
Log in to vCenter Server using the vSphere Client.
Shut down the Ops Manager VM.
Deploy the upgraded Ops Manager VM by following the first two steps of Deploying Ops Manager with NSX-T for TKGI:
Using a browser, navigate to the newly-deployed Ops Manager web interface.
On the welcome page, select Import Existing Installation.
Browse to and select the installation configuration archive you exported.
Log in to Ops Manager
Click Apply Changes.
Verify that the BOSH Director for vSphere tile shows the target, updated version.
When you upgrade TKGI, your configuration settings typically migrate to the new version automatically. To download and import a TKGI version:
Download the target version of the product from VMware Tanzu Network.
Import the target version of the TKGI tile to the Ops Manager Installation Dashboard.
Click Review Pending Changes.
Expand the Errands dropdown and enable or disable Upgrade all clusters errand
- See Deciding Between Full and Two-Phase Upgrade to decide whether to upgrade TKGI-provisioned Kubernetes clusters along with TKGI, or upgrade them later.
- VMware recommends that you upgrade Kubernetes clusters along with TKGI if possible.
- Enable the Upgrade all clusters errand to upgrade clusters along with TKGI.
Warning: Disabling the Upgrade all clusters errand causes the TKGI version tagged in your Kubernetes clusters to fall behind the TKGI tile version. If you disable the Upgrade all clusters errand when upgrading the TKGI tile, you must upgrade all your Kubernetes clusters before the next TKGI upgrade.
Note: If you are upgrading TKGI on NSX-T v2.5, you must select the Upgrade all clusters errand or plan to upgrade all clusters individually before upgrading to NSX-T v3.0.
Set the Run smoke tests errand to On. The errand uses the TKGI CLI to create a Kubernetes cluster and then delete it. If the creation or deletion fails, the errand fails and the installation of the TKGI tile is aborted.
TKGI requires a Xenial stemcell.
Because TKGI v1.8.0 is incompatible with 621.76+ stemcells, you must upgrade TKGI to v1.8.1 and update all clusters before upgrading stemcells to 621.76+.
If Ops Manager does not have the Xenial stemcell required for TKGI v1.8, the TKGI tile displays the message Missing stemcell. To download and import a new Xenial stemcell, follow the steps below:
On the TKGI tile, click the Missing stemcell link.
In the Stemcell Library, locate the TKGI tile and note the required stemcell version.
Navigate to the Stemcells (Ubuntu Xenial) page on VMware Tanzu Network and download the required stemcell version for your IaaS.
Return to the Installation Dashboard in Ops Manager and click Stemcell Library.
On the Stemcell Library page, click Import Stemcell and select the stemcell file you downloaded from VMware Tanzu Network.
Select the TKGI tile and click Apply Stemcell to Products.
Verify that Ops Manager successfully applied the stemcell. The stemcell version you imported and applied appears in the Staged column for TKGI.
Return to the Installation Dashboard.
To complete the upgrade of the TKGI tile:
Return to the Installation Dashboard in Ops Manager.
Click Review Pending Changes. For more information about this Ops Manager page, see Reviewing Pending Product Changes.
Click Apply Changes.
(Optional) If you enabled the Upgrade all clusters errand, you can use the BOSH CLI to monitor its progress:
- Log in to the BOSH Director by running
bosh -e MY-ENVIRONMENT log-infrom a VM that can access your TKGI deployment. For more information, see Using BOSH Diagnostic Commands in Tanzu Kubernetes Grid Integrated Edition.
bosh -e MY-ENVIRONMENT tasks.
- Locate the task number for the errand in the # column of the BOSH output.
bosh task TASK-NUMBER, replacing
TASK-NUMBERwith the task number you located in the previous step.
- Log in to the BOSH Director by running
Verify that the TKGI tile is updated to the target version.
After you complete the upgrade to TKGI v1.8, complete the following verifications and upgrades:
- Update the TKGI and Kubernetes CLIs
- Upgrade Kubernetes Clusters if Needed
- Verify TKGI Upgrade
- (Optional) Upgrade NSX-T Data Center v2.5 to v3.0
- (Optional) Upgrade to vSphere 7
Update the TKGI and Kubernetes CLIs on any local machine where you run commands that interact with your upgraded version of TKGI.
To update your CLIs, download and re-install the TKGI and Kubernetes CLI distributions that are provided with TKGI on VMware Tanzu Network.
For more information about installing the CLIs, see the following topics:
If you upgraded TKGI with the Upgrade all clusters errand disabled, the next step is to upgrade the Kubernetes clusters individually using the TKGI CLI.
Log in to the TKGI environment using the TKGI CLI.
Run the command
tkgi clustersto list all Kubernetes clusters with their current versions and status:
PKS Version Name k8s Version Plan Name UUID Status Action 1.7.0 pks-cluster-1-small 1.16.7 small 0bea03c8-af47-48e8-b249-814c0bc407b9 succeeded CREATE 1.7.0 pks-cluster-2-medium 1.16.7 medium 5d9f4501-70cb-460b-9d78-0afbc074cb8c succeeded CREATE 1.7.0 pks-cluster-3-large 1.16.7 large b448117a-bb6f-49de-bc9b-452588bd44ef succeeded CREATE
Update each cluster one-by-one using the command
tkgi upgrade-cluster CLUSTER-NAME.
- You do not have to wait for each upgrade to complete before upgrading the next one.
- The advantage of running each upgrade separately is that it makes troubleshooting easier. BOSH assigns a unique task ID to each cluster upgrade.
When the cluster upgrades are complete, run the command
tkgi clustersand verify that they list the target version:
PKS Version Name k8s Version Plan Name UUID Status Action 1.8.0 pks-cluster-1-small 1.17.5 small 0bea03c8-af47-48e8-b249-814c0bc407b9 succeeded UPGRADE 1.8.0 pks-cluster-2-medium 1.17.5 medium 5d9f4501-70cb-460b-9d78-0afbc074cb8c succeeded UPGRADE 1.8.0 pks-cluster-3-large 1.17.5 large b448117a-bb6f-49de-bc9b-452588bd44ef succeeded UPGRADE
To verify successful upgrade, create a test cluster:
pks create-cluster pks-cluster-4-test --external-hostname pks-cluster-test --plan medium --num-nodes 3
tkgi clustersto verify that the new cluster is created with the appropriate version of TKGI and Kubernetes:
$ tkgi clusters PKS Version Name k8s Version Plan Name UUID Status Action 1.8.0 pks-cluster-4-test 1.17.5 medium 5d9f4501-70cb-460b-9d78-0afbc074cb8c succeeded CREATE 1.8.0 pks-cluster-1-small 1.17.5 small 0bea03c8-af47-48e8-b249-814c0bc407b9 succeeded UPGRADE 1.8.0 pks-cluster-2-medium 1.17.5 medium 5d9f4501-70cb-460b-9d78-0afbc074cb8c succeeded UPGRADE 1.8.0 pks-cluster-3-large 1.17.5 large b448117a-bb6f-49de-bc9b-452588bd44ef succeeded UPGRADE
Warning: TKGI supports NSX-T v18.104.22.168 (v3.0.1 EP1) and later patch versions of NSX-T v3.0. It does not support NSX-T v3.0.0, or v3.0.1 without Express Patch EP1.
If you are using NSX-T v2.5, you can upgrade to NSX-T v3.0. For a list of NSX-T 2.5 and 3.0 versions compatible with TKGI v1.8, see Product Snapshot in Release Notes for TKGI v1.8.
Confirm that you have upgraded all TKGI-provisioned Kubernetes clusters to TKGI v1.8 using Upgrade all clusters errand in Ops Manager or using the TKGI CLI.
Note: This updates your Kubernetes clusters to the version of Kubernetes and version of NCP that are included with TKGI v1.8.
Confirm that your vSphere v6.5, v6.7, or v7.0 installation is on the supported version and patch for NSX-T v3.0.
- Refer to the VMware Product Interoperability Matrices.
- If necessary, update to the required vSphere version or patch before proceeding with the upgrade of NSX-T.
Upload the NSX-T upgrade bundle using the NSX-T Manager and proceed with the upgrade process by following the instructions in the UI.
Warning: TKGI only supports using single VDS for both vSphere v7 and NSX-T v3.0 traffic on new installations, with Ops Manager 2.9.5+, and with NSX-T v22.214.171.124 (v3.0.1 EP1) or later. Previous versions of TKGI required N-VDS, and you cannot migrate its existing NSX-T infrastructure from N-VDS to VDS.
For more information, refer to the NSX-T Data Center Upgrade Guide documentation.
If you made architectural changes to your NSX-T environment that affect TKGI, such as adding or updating a VIP address, or a load balancer for the NSX-T Management Cluster, update the BOSH Director and TKGI tiles with the new or updated IP addresses:
- In the BOSH Director tile > vCenter Configuration pane, update NSX Address and NSX CA Cert.
- In the TKGI tile > Networking pane, update NSX Manager hostname and NSX Manager CA Cert.
After upgrading TKGI and its Kubernetes clusters to v1.8 and NSX-T to v3.0, you can upgrade vSphere to v7. This upgrade includes upgrading the vCenter Server Appliance and each ESXi host, in that order.
- Upgrade vCenter. Refer to Upgrading the vCenter Server Appliance in the vCenter documentation.
- Upgrade each ESXi host, one at a time.
- Put the ESXi host into maintenance mode.
- Upgrade the ESXi host. Refer to Upgrading ESXi hosts in the vSphere documentation.
- Using the NSX-T Manager web interface for Transport Nodes, install the vSphere 7.0 VIBS onto the ESXi host.
- Using the NSX Manage web interface, verify that the ESXi host is in a “Success” state. If it is not, click the Resolve button.
- Remove the ESXi host from maintenance mode.
- Repeat the process for each ESXi host in your vCenter cluster that is part of your TKGI domain.
See Verifying Deployment Health for how to verify the health of your TKGI environment and gather information for troubleshooting cluster upgrades.
Please send any feedback you have to firstname.lastname@example.org.