Upgrading PKS

Page last updated:

Warning: Pivotal Container Service (PKS) v1.3 is no longer supported because it has reached the End of General Support (EOGS) phase as defined by the Support Lifecycle Policy.
To stay up to date with the latest software and security updates, upgrade to a supported version.

This topic explains how to upgrade the Pivotal Container Service (PKS) tile and existing Kubernetes clusters.

Overview

The supported upgrade paths to PKS v1.3 are from PKS v1.2.7 and later patches. PKS v1.3 is compatible with Ops Manager v2.3.1 and later patches, and Ops Manager v2.4.

For conceptual information about upgrading the PKS tile and PKS-provisioned Kubernetes clusters, see What Happens During PKS Upgrades.

Note: Upgrading from PKS v1.2.5 and later patches to PKS v1.3 causes all certificates to be automatically regenerated. The new certificates are signed with a new certificate authority and are valid for four years. The old certificate authority, which remains trusted, has a validity of one year.

For information about upgrading PKS on vSphere with NSX-T integration, see Upgrading PKS with NSX-T.

WARNING: Do not manually upgrade your Kubernetes version. The PKS service includes the compatible Kubernetes version.

Before You Upgrade

This section describes the activities you must perform before upgrading PKS.

Determine Your Upgrade Path

Use the following table to determine your upgrade path to PKS v1.3.

WARNING: PKS v1.2.8 and earlier patches include a critical CVE. When upgrading to PKS v1.3 it is critical that you implement and maintain an ETCD certificate trust chain. Review the procedures in the PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) article in the Pivotal Support Knowledge Base before upgrading.

Note: PKS v1.3 on Azure is not supported by Ops Manager v2.3.10 and later patches and Ops Manager v2.4.4 and later patches. If you are deploying PKS v1.3 on Azure, install only Ops Manager v2.3.9 or earlier patches or Ops Manager v2.4.3 or earlier patches before deploying PKS v1.3.

If your current version of PKS is… Then use the following upgrade path:
v1.1.5 or later v1.1 patch
  1. Upgrade to Ops Manager v2.3.1 or later patch.
  2. Follow the procedures in Complete the CVE Upgrade Path below to upgrade to PKS v1.2.6.
  3. (Optional) Upgrade to Ops Manager v2.4.0 or later patch.
v1.2.0, v1.2.1, v1.2.2, v1.2.3, v1.2.4, or v1.2.5
  1. Upgrade to Ops Manager v2.3.1 or later patch.
  2. Follow the procedures in Complete the CVE Upgrade Path below to upgrade to PKS v1.2.6.
  3. (Optional) Upgrade to Ops Manager v2.4.0 or later patch.
v1.2.6
  1. Review the procedures in PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) in the Pivotal Support Knowledge Base.
  2. Follow the procedures in Complete the CVE Upgrade Path below to upgrade to PKS v1.2.7.
v1.2.7
  1. Review the procedures in PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) in the Pivotal Support Knowledge Base.
  2. Follow the procedures in Complete the CVE Upgrade Path below to upgrade to PKS v1.3.1.
v1.2.8 or later v1.2 patch
  1. Review the procedures in PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) in the Pivotal Support Knowledge Base.
  2. Follow the procedures in Complete the CVE Upgrade Path below to upgrade to PKS v1.3.4.
v1.3.0
  1. Review the procedures in PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) in the Pivotal Support Knowledge Base.
  2. Follow the procedures in Complete the CVE Upgrade Path below to upgrade to PKS v1.3.1.
v1.3.1, v1.3.2, or v1.3.3
  1. Review the procedures in PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) in the Pivotal Support Knowledge Base.
  2. Follow the procedures in Complete the CVE Upgrade Path below to upgrade to PKS v1.3.4.
v1.3.4 or later v1.3 patch
  1. Follow the procedures in Upgrade PKS below to upgrade to the most recent PKS v1.3 patch.
  2. (Optional) Upgrade to Ops Manager v2.4.0 or later patch.

Prepare to Upgrade

If you have not already, complete the steps in the Upgrade Preparation Checklist for PKS v1.3.

Complete the CVE Upgrade Path

It is critical that you upgrade your PKS v1.2 environment all the way through to PKS v1.3.2 or a later patch to remove Common Vulnerabilities and Exposures (CVE) 2019-3779.

WARNING: PKS v1.2.8 and earlier patches include a critical CVE. When upgrading from PKS v1.2.8 and earlier patches it is critical that you implement and maintain an ETCD certificate trust chain. Review the procedures in the PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) article in the Pivotal Support Knowledge Base before upgrading PKS v1.2.

The steps below describe the proper procedure for upgrading all PKS v1.2 patches to PKS v1.3:

  1. Upgrade to Ops Manager v2.3.1 or later v2.3 patch, or v2.4
  2. Upgrade to PKS v1.2.6
  3. Upgrade from PKS v1.2.6 to PKS v1.2.7
  4. Upgrade from PKS v1.2.7 to PKS v1.3.1
  5. Upgrade from PKS v1.2.8 or later v1.2 patches to PKS v1.3.4
  6. Upgrade from PKS v1.3.0 to PKS v1.3.1
  7. Upgrade from PKS v1.3.1, v1.3.2, or v1.3.3 to PKS v1.3.4

During an upgrade your existing configuration settings automatically migrate to the new version.

Upgrade to Ops Manager v2.3.1 or later v2.3 patch, or v2.4

Before you upgrade to PKS v1.3, you must upgrade to Ops Manager v2.3.1 or later v2.3 patch, or v2.4.

  1. Follow the procedures detailed in Upgrade Ops Manager and Installed Products to v2.3 or Upgrade Ops Manager and Installed Products to v2.4 in Upgrading Pivotal Cloud Foundry.

  2. Verify that the PKS control plane remains functional by performing the following steps:

    1. Add more workloads and create an additional cluster. For more information about performing those actions, see About Workload Upgrades in Maintaining Workload Uptime and Creating Clusters.
    2. Monitor the PKS control plane VM by clicking the Pivotal Container Service tile, selecting Status tab, and reviewing the Pivotal Container Service VM’s data points. If any data points are at capacity, scale your deployment accordingly.

Upgrade to PKS v1.2.6

Skip this step if you are already running PKS v1.2.6 or later version.

  1. To upgrade to PKS v1.2.6, perform the upgrade procedures described in Upgrade PKS, below. Note the following PKS v1.2.6-upgrade differences:
    • Download the PKS v1.2.6 tile.

Upgrade from PKS v1.2.6 to PKS v1.2.7

Skip this step if you are already running PKS v1.2.7 or later version.

You can upgrade PKS v1.2.6 directly to PKS v1.2.7.

  1. Determine your PKS migration path by reviewing PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) in the Pivotal Support Knowledge Base.
  2. To upgrade PKS v1.2.6 to PKS v1.2.7, perform the upgrade procedures described in Upgrade PKS, below. Note the following PKS v1.2.7-upgrade differences:
    • Download the PKS v1.2.7 tile.

Upgrade from PKS v1.2.7 to PKS v1.3.1

Skip this step if you are already running PKS v1.2.8 or later v1.2 patches.

You can upgrade PKS v1.2.7 directly to PKS v1.3.1.

  1. Review PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) in the Pivotal Support Knowledge Base.
  2. To upgrade to PKS v1.3.1, perform the upgrade procedures described in Upgrade PKS, below. Note the following PKS v1.3.1-upgrade differences:
    • Download the PKS v1.3.1 tile.

Upgrade from PKS v1.2.8 or later v1.2 patches to PKS v1.3.4

Skip this step if you are already running PKS v1.3.4 or later patches.

You can upgrade PKS v1.2.8 or later v1.2 patches directly to PKS v1.3.4.

  1. Review PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) in the Pivotal Support Knowledge Base.
  2. To upgrade to PKS v1.3.4, perform the upgrade procedures described in Upgrade PKS, below. Note the following PKS v1.3.4-upgrade differences:
    • Download the PKS v1.3.4 tile.

Upgrade from PKS v1.3.0 to PKS v1.3.1

Skip this step if you are already running PKS v1.3.1 or later v1.3 patches.

You can upgrade PKS v1.3.0 directly to PKS v1.3.1.

  1. Review PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) in the Pivotal Support Knowledge Base.
  2. To upgrade to PKS v1.3.1, perform the upgrade procedures described in Upgrade PKS, below. Note the following PKS v1.3.1-upgrade differences:
    • Download the PKS v1.3.1 tile.

Upgrade from PKS v1.3.1, v1.3.2, or v1.3.3 to PKS v1.3.4

Skip this step if you are already running PKS v1.3.4 or later patches.

You can upgrade PKS v1.3.1, v1.3.2 or v1.3.3 directly to PKS v1.3.4.

  1. Review PKS upgrade approach for CRITICAL CVE: 2019-3779 (67116) in the Pivotal Support Knowledge Base.
  2. To upgrade to PKS v1.3.4, perform the upgrade procedures described in Upgrade PKS, below. Note the following PKS v1.3.4-upgrade differences:
    • Download the PKS v1.3.4 tile.

Upgrade PKS

Upgrading PKS follows the same Ops Manager process that you used to install the PKS tile for the first time.

Note: Your configuration settings migrate to the new version automatically. Follow the steps below to perform an upgrade.

To upgrade between PKS versions, complete the following steps:

  1. Download and Import the PKS tile
  2. Download and Import the Stemcell
  3. Verify Errand Configuration
  4. Apply Changes to the PKS Tile

Download and Import the PKS tile

To download and import the PKS tile, complete the following steps:

  1. Review the Release Notes for the version you are upgrading to.

  2. Download the desired version of the product from Pivotal Network.

  3. Navigate to the Ops Manager Installation Dashboard and click Import a Product to upload the product file.

  4. Under the Import a Product button, click + next to Pivotal Container Service. This adds the tile to your staging area.

Download and Import the Stemcell

PKS v1.3.x uses a Xenial stemcell.

If Ops Manager does not have the Xenial stemcell required for PKS, the PKS tile displays the message Missing stemcell.

Note: If the Stemcell Library in Ops Manager already has a compatible Xenial stemcell, the Missing stemcell link does not appear. You do not need to download or import a new stemcell and can skip this step.

To download and import a new Xenial stemcell, follow the steps below:

  1. On the Pivotal Container Service tile, click on the Missing stemcell link.

    Verify stemcell assignment

  2. In the Stemcell Library, locate Pivotal Container Service and note the required stemcell version.

  3. Visit the Stemcells for PCF (Ubuntu Xenial) page on Pivotal Network, and download the required stemcell version appropriate for your IaaS.

  4. Return to the Installation Dashboard in Ops Manager, and click on Stemcell Library.

  5. On the Stemcell Library page, click Import Stemcell and select the stemcell file you downloaded from Pivotal Network.

  6. Select Pivotal Container Service and click Apply Stemcell to Products.

  7. Verify that Ops Manager successfully applied the stemcell. The stemcell version you imported and applied appears in the Staged column for Pivotal Container Service.

  8. Select the Installation Dashboard link to return to the Installation Dashboard.

Verify Errand Configuration

To verify that errands are configured correctly in the PKS tile, perform the following steps.

  1. Click the newly-added Pivotal Container Service tile.

  2. Click Errands.

  3. Under Post-Deploy Errands, verify that the Upgrade all clusters errand is set to Default (On). The errand upgrades a single Kubernetes cluster at a time. Upgrading PKS Kubernetes clusters can temporarily interrupt the service, as described in Service Interruptions.

    WARNING: If you are upgrading PKS, you must enable the Upgrade All Clusters errand.

  4. Under Post-Deploy Errands, set the Run smoke tests errand to On. The errand uses the PKS Command Line Interface (PKS CLI) to create a Kubernetes cluster and then delete it. If the creation or deletion fails, the errand fails and the installation of the PKS tile is aborted.

  5. Review the other configuration panes. Click Save on any panes where you make changes.

    Note: When you upgrade PKS, you must place singleton jobs in the AZ you selected when you first installed the PKS tile. You cannot move singleton jobs to another AZ.

Apply Changes to the PKS Tile

Perform the following steps to complete the upgrade to the PKS tile.

  1. Return to the Installation Dashboard in Ops Manager.

  2. Click Review Pending Changes. For more information about this Ops Manager page, see Reviewing Pending Product Changes.

  3. Click Apply Changes.

  4. (Optional) To monitor the progress of the Upgrade all clusters errand using the BOSH CLI, do the following:

    1. Log in to the BOSH Director by running bosh -e MY-ENVIRONMENT log-in from a VM that can access your PKS deployment. For more information, see Managing PKS Deployments with BOSH.
    2. Run bosh -e MY-ENVIRONMENT tasks.
    3. Locate the task number for the errand in the # column of the BOSH output.
    4. Run bosh task TASK-NUMBER, replacing TASK-NUMBER with the task number you located in the previous step.

After the Upgrade

After you complete the upgrade to PKS v1.3.x, complete the following verifications and upgrades.

Update PKS and Kubernetes CLIs

Update the PKS and Kubernetes CLIs on any local machine where you run commands that interact with your upgraded version of PKS.

To update your CLIs, download and re-install the PKS and Kubernetes CLI distributions that are provided with PKS on Pivotal Network.

For more information about installing the CLIs, see the following topics:

Verify the Upgrade

After you apply changes to the PKS tile and the upgrade is complete, perform the following steps:

  1. Verify that your Kubernetes environment is healthy. To verify the health of your Kubernetes environment, see Verify Kubernetes Health.

  2. Verify that the PKS control plane remains functional by performing the following steps:

    1. Add more workloads and create an additional cluster. For more information about performing those actions, see About Workload Upgrades in Maintaining Workload Uptime and Creating Clusters.
    2. Monitor the PKS control plane VM by clicking the Pivotal Container Service tile, selecting Status tab, and reviewing the Pivotal Container Service VM’s data points. If any data points are at capacity, scale your deployment accordingly.

(Optional) Upgrade vSphere

If you are deploying PKS on vSphere, consult the chart below, and upgrade vSphere if necessary.

Versions Editions
  • VMware vSphere 6.7 U1 EP06 (ESXi670-201901001) – for NSX-T 2.4
  • VMware vSphere 6.7 U1
  • VMware vSphere 6.7.0
  • VMware vSphere 6.5 U2 P03 (ESXi650-201811002) – for NSX-T 2.4
  • VMware vSphere 6.5 U2
  • VMware vSphere 6.5 U1
  • vSphere Enterprise Plus
  • vSphere with Operations Management Enterprise Plus

Note: VMware vSphere 6.7 is only supported with Ops Manager v2.3.1 or later.


Please send any feedback you have to pks-feedback@pivotal.io.