Upgrading Enterprise PKS with NSX-T

Page last updated:

This topic explains how to upgrade VMware Enterprise PKS for environments using vSphere with NSX-T.

Warning: Do not manually upgrade your Kubernetes version. Enterprise PKS includes the compatible Kubernetes version.

Overview

The supported upgrade paths to Enterprise PKS v1.6.0 are from Enterprise PKS v1.5.0 and later. Enterprise PKS v1.6.0 is compatible with Ops Manager v2.6.16 and later or Ops Manager v2.7.6 and later.

Before you begin the upgrade, follow the procedures in the Prerequisites section to plan and prepare your upgrade.

After you complete the prerequisite steps, continue to the procedures in the During the Upgrade section. These steps guide you through the process of upgrading the Enterprise PKS tile, upgrading Ops Manager, importing an updated stemcell, and applying the changes to your deployment.

After you complete the upgrade, follow the procedures in the After the Upgrade section to verify that your upgraded Enterprise PKS deployment is running properly.

Prerequisites

Before you upgrade Enterprise PKS, you must do the following:

  1. Determine Your Upgrade Path
  2. Prepare to Upgrade

Determine Your Upgrade Path

Use the following table to determine your upgrade path to Enterprise PKS v1.6.

Target Ops Manager version Upgrade Path for NSX-T v2.4 Upgrade Path for NSX-T v2.5
v2.6
  1. Upgrade to NSX-T Data Center v2.4.3
  2. Upgrade to Ops Manager v2.6.16 and later.
  3. Upgrade to Enterprise PKS v1.6.
  4. Upgrade to NSX-T Data Center v2.5.
  1. Upgrade to Ops Manager v2.6.16 and later.
  2. Upgrade to Enterprise PKS v1.6.
v2.7
  1. Upgrade to NSX-T Data Center v2.4.3
  2. Upgrade to Ops Manager v2.7.6 and later.
  3. Upgrade to Enterprise PKS v1.6.
  4. Upgrade to NSX-T Data Center v2.5.
  1. Upgrade to Ops Manager v2.7.6 and later.
  2. Upgrade to Enterprise PKS v1.6.

Prepare to Upgrade

If you have not already, complete all of the steps in the Upgrade Preparation Checklist for Enterprise PKS v1.6.

Review new features that are available in Enterprise PKS v1.6 by reading the Release Notes. In particular, review the new networking-related features that are available for NSX-T deployments.

During the Upgrade

This section describes the steps required to upgrade to Enterprise PKS v1.6:

  1. Review the Release Notes for the supported software versions.
  2. Upgrade to Ops Manager v2.6.16+ or v2.7.6+
  3. Upgrade to Enterprise PKS v1.6
  4. Download and Import the Stemcell
  5. Verify Errand Configuration
  6. Apply Changes to the Enterprise PKS Tile
  7. Upgrade your Kubernetes clusters by selecting the Upgrade all clusters errand in Ops Manager, or by using the PKS CLI as described in Upgrading Clusters.

    Note: This operation updates the Kubernetes clusters to the version of Kubernetes that is included with with Enterprise PKS v1.6, and updates your Kubernetes clusters to the version of NCP that is included with Enterprise PKS v1.6.

  8. (Recommended) If you are running NSX-T v2.4.3, upgrade to v2.5.x.

Upgrade Ops Manager to v2.6.16+ or v2.7.6+

Before you upgrade to Enterprise PKS v1.6, you must upgrade to Ops Manager v2.6.16 and later or v2.7.6 and later.

To upgrade Ops Manager, do the following:

  1. Complete the procedures in Upgrade Ops Manager and Installed Products to v2.6 or Upgrade Ops Manager and Installed Products to v2.7.

  2. Verify that the PKS control plane remains functional by performing the following steps:

    1. Add more workloads and create an additional cluster. For more information, see About Workload Upgrades in Maintaining Workload Uptime and Creating Clusters.
    2. Monitor the PKS control plane VM by clicking the Enterprise PKS tile, selecting the Status tab, and reviewing the Pivotal Container Service VM’s data points. If any data points are at capacity, scale your deployment accordingly.

Upgrade Enterprise PKS to v1.6

To upgrade to Enterprise PKS v1.6, follow the same Ops Manager process that you use to install the tile for the first time.

Your configuration settings migrate to the new version automatically. Follow the steps below to perform an upgrade.

  1. Review the Release Notes for the version you are upgrading to.

  2. Download the desired version of the product from Pivotal Network.

  3. Navigate to the Ops Manager Installation Dashboard and click Import a Product to upload the product file.

  4. Under the Import a Product button, click + next to Enterprise PKS. This adds the tile to your staging area.

Download and Import the Stemcell

Enterprise PKS v1.6 requires a Xenial stemcell.

If Ops Manager does not have the Xenial stemcell required for Enterprise PKS, the Enterprise PKS tile displays the message Missing stemcell.

Note: If the Stemcell Library in Ops Manager already has a compatible Xenial stemcell, the Missing stemcell link does not appear. You do not need to download or import a new stemcell and can skip this step.

To download and import a new Xenial stemcell, follow the steps below:

  1. On the Enterprise PKS tile, click the Missing stemcell link.

    Verify stemcell assignment

  2. In the Stemcell Library, locate the Enterprise PKS tile and note the required stemcell version.

  3. Navigate to the Pivotal Stemcells (Ubuntu Xenial) page on Pivotal Network and download the required stemcell version for your IaaS.

  4. Return to the Installation Dashboard in Ops Manager and click Stemcell Library.

  5. On the Stemcell Library page, click Import Stemcell and select the stemcell file you downloaded from Pivotal Network.

  6. Select the Enterprise PKS tile and click Apply Stemcell to Products.

  7. Verify that Ops Manager successfully applied the stemcell. The stemcell version you imported and applied appears in the Staged column for Enterprise PKS.

  8. Select the Installation Dashboard link to return to the Installation Dashboard.

(Optional) Configure Host Monitoring

In Enterprise PKS v1.6, the Logging screen is renamed to Host Monitoring and adds options to configure the Telegraf agent to collect metrics from Node Exporter, etcd, and the Kubelet agent and send the metrics to a third-party monitoring service.

For information about configuring Host Monitoring, see the (Optional) Host Monitoring section of Installing Enterprise PKS on vSphere with NSX-T.

(Optional) Configure In-Cluster Monitoring

In Enterprise PKS v1.6, the Monitoring screen is renamed to In-Cluster Monitoring.

For information about configuring In-Cluster Monitoring, see the (Optional) In-Cluster Monitoring section of Installing Enterprise PKS on vSphere with NSX-T.

(Optional) Configure Tanzu Mission Control Integration

If you are a participant in the VMware Tanzu Mission Control beta program, you can choose to integrate your Enterprise PKS deployment with Tanzu Mission Control.

For information about configuring Tanzu Mission Control (Experimental), see the Tanzu Mission Control (Experimental) section of Installing Enterprise PKS on vSphere with NSX-T.

Verify Errand Configuration

To verify that errands are configured correctly in the Enterprise PKS tile, do the following:

  1. In the Enterprise PKS tile, click Errands.

  2. Under Post-Deploy Errands, configure Upgrade all clusters errand:

    • If you want to upgrade the Enterprise PKS tile and all your existing Kubernetes clusters simultaneously, verify that Upgrade all clusters errand is set to Default (On). The errand upgrades all clusters, one Kubernetes cluster at a time. Upgrading Enterprise PKS-deployed Kubernetes clusters can temporarily interrupt the service as described in Service Interruptions.

      Note: If you are upgrading Enterprise PKS on NSX-T v2.4.3, you must select the Upgrade all clusters errand or plan to upgrade all clusters individually prior to upgrading to NSX-T v2.5.

    • If you want to upgrade the Enterprise PKS tile only and then upgrade your existing Kubernetes clusters separately, disable Upgrade all clusters errand. For more information, see Upgrading Clusters.

      Warning: If you disable Upgrade all clusters errand when upgrading the Enterprise PKS tile or updating your tile configuration, you must upgrade all your Kubernetes clusters before the next Enterprise PKS upgrade or tile configuration update.

  3. Under Post-Deploy Errands, set the Run smoke tests errand to On. The errand uses the Enterprise PKS Command Line Interface (PKS CLI) to create a Kubernetes cluster and then delete it. If the creation or deletion fails, the errand fails and the installation of the Enterprise PKS tile is aborted.

  4. Click Save.

  5. Review the other configuration panes. Click Save on any panes where you make changes.

    Note: When you upgrade Enterprise PKS, you must place singleton jobs in the AZ you selected when you first installed the Enterprise PKS tile. You cannot move singleton jobs to another AZ.

Apply Changes to the Enterprise PKS Tile

Perform the following steps to complete the upgrade of the Enterprise PKS tile:

  1. Return to the Installation Dashboard in Ops Manager.

  2. Click Review Pending Changes. For more information about this Ops Manager page, see Reviewing Pending Product Changes.

  3. Click Apply Changes.

  4. (Optional) To monitor the progress of the Upgrade all clusters errand using the BOSH CLI, do the following:

    1. Log in to the BOSH Director by running bosh -e MY-ENVIRONMENT log-in from a VM that can access your Enterprise PKS deployment. For more information, see Using BOSH Diagnostic Commands in Enterprise PKS.
    2. Run bosh -e MY-ENVIRONMENT tasks.
    3. Locate the task number for the errand in the # column of the BOSH output.
    4. Run bosh task TASK-NUMBER, replacing TASK-NUMBER with the task number you located in the previous step.

After the Upgrade

After you complete the upgrade to Enterprise PKS v1.6, complete the following verifications and upgrades:

Update PKS and Kubernetes CLIs

Update the PKS and Kubernetes CLIs on any local machine where you run commands that interact with your upgraded version of Enterprise PKS.

To update your CLIs, download and re-install the PKS and Kubernetes CLI distributions that are provided with Enterprise PKS on Pivotal Network.

For more information about installing the CLIs, see the following topics:

Verify the Upgrade

After you apply changes to the Enterprise PKS tile and the upgrade is complete, perform the following steps:

  1. Verify that your Kubernetes environment is healthy. To verify the health of your Kubernetes environment, see Verifying Deployment Health.

  2. Verify that the PKS control plane remains functional by performing the following steps:

    1. Add more workloads and create an additional cluster. For more information, see About Workload Upgrades in Maintaining Workload Uptime and Creating Clusters.
    2. Monitor the PKS control plane VM by clicking the Enterprise PKS tile, selecting the Status tab, and reviewing the Pivotal Container Service VM’s data points. If any data points are at capacity, scale your deployment accordingly.

Upgrade NSX-T Data Center v2.4.3 to v2.5

If you are using NSX-T v2.4.3, Pivotal recommends that you upgrade to NSX-T v2.5.

  1. Confirm that you have upgraded all PKS-provisioned Kubernetes clusters to PKS v1.6 using Upgrade all clusters errand in Ops Manager or using the PKS CLI.

  2. Confirm that your vSphere v6.5 or v6.7 installation is on the supported version and patch for NSX-T v2.5.

  3. To upgrade NSX-T, refer to the NSX-T Data Center Upgrade Guide in the VMware documentation.

  4. If you made any architectural changes to your NSX-T environment that affect Enterprise PKS, such as adding a VIP address or load balancer for the NSX-T Management Cluster, update the PKS and BOSH Director tile with the new information, specifically:

    • In the BOSH Director tile, select the vCenter Configuration tab and update the following:
      • The NSX Address field
      • The NSX CA Cert field
      • Save the BOSH tile changes. Update BOSH with VIP and Cert
    • In the PKS tile, select the Networking tab and update the following:
      • The NSX Manager hostname field
      • The NSX Manager CA Cert field
      • Save the PKS tile changes Update Enterprise PKS with VIP and Cert
  5. If you made changes to the BOSH Director or PKS tiles:

    • Go to the Installation Dashboard in Ops Manager.
    • Click Review Pending Changes.
    • Expand the Errands list for Enterprise PKS.
    • Ensure that the Upgrade all clusters errand is selected.
    • Click Apply Changes. Upgrade all Kubernetes clusters
  6. Verify the upgrade of NSX-T Data Center.

After you apply changes to the Enterprise PKS tile and the upgrade is complete, verify that your Kubernetes environment is healthy. To verify the health of your Kubernetes environment, see Verifying Deployment Health.


Please send any feedback you have to pks-feedback@pivotal.io.