LATEST VERSION: v1.2 - RELEASE NOTES
Pivotal Container Service v1.1

What Happens During PKS Upgrades

This topic explains what happens to Kubernetes clusters provisioned by Pivotal Container Service (PKS) during PKS upgrades.

Introduction

PKS enables you to upgrade either the PKS tile and all PKS-provisioned Kubernetes clusters or only the PKS tile.

During an upgrade of the PKS tile, your configuration settings are automatically migrated to the new tile version. For upgrading instructions, see Upgrading PKS.

WARNING: If you upgrade the PKS tile from v1.0.x to v1.1, you must upgrade both the PKS tile and all PKS-provisioned Kubernetes clusters. This ensures existing clusters can run resize or delete commands after the upgrade.

Canary Instances and max_in_flight

The PKS tile is a BOSH deployment. When you deploy or upgrade a product using BOSH, two things that can affect the deployment are the number of canary instances and the value of the max_in_flight variable.

BOSH-deployed products can set a number of canary instances to upgrade first, before the rest of the deployment VMs. BOSH continues the upgrade only if the canary instance upgrade succeeds. If the canary instance encounters an error, the upgrade stops running and other VMs are not affected. The PKS tile uses one canary instance when deploying or upgrading PKS.

The max_in_flight variable limits how many instances of a component can restart simultaneously during updates or upgrades. This variable is set to 1 and is not configurable in PKS. Because the value is set to 1, only one component restarts at a time.

Upgrades of the PKS Tile and PKS-Provisioned Clusters

During an upgrade of the PKS tile and PKS-provisioned clusters, the following occurs:

  1. The PKS API server is recreated. For more information, see PKS API Server.

  2. Each of your Kubernetes clusters is recreated, one at a time. This includes the following stages for each cluster:

    1. Master nodes are recreated. For more information, see Master Nodes.
    2. Worker nodes are recreated. For more information, see Worker Nodes.

Note: When PKS is set to upgrade both the PKS tile and PKS-provisioned clusters, updating any stemcell in your deployment rolls every VM in each Kubernetes cluster. This ensures that all the VMs are patched. With the recommended resource configuration described above, no workload downtime is expected. For information about maintaining your Kubernetes workload uptime, see Maintaining Workload Uptime.

PKS API Server

When the PKS API server is recreated, you cannot interact with the PKS control plane or manage Kubernetes clusters. These restrictions prevent you from performing the following actions:

  • Logging in through the PKS CLI
  • Retrieving information about clusters
  • Creating and deleting clusters
  • Resizing clusters

Recreating the PKS API server does not affect deployed Kubernetes clusters and their workloads. You can still interact with them through the Kubernetes Command Line Interface, kubectl.

For more information about the PKS control plane, see PKS Control Plane Overview in PKS Cluster Management.

Master Nodes

When PKS recreates a single-master cluster during an upgrade, you cannot interact with your cluster, use kubectl, or push new workloads.

Note: To avoid this loss of functionality, Pivotal recommends using multi-master clusters.

Worker Nodes

When PKS recreates worker nodes, the upgrade runs on a single VM at a time. During the upgrade, the VM stops running containers. If your workloads run on a single VM, your apps will experience downtime.

Note: To avoid downtime for stateless workloads, Pivotal recommends using at least one worker node per availability zone (AZ). For stateful workloads, Pivotal recommends using a minimum of two worker nodes per AZ.

Upgrades of the PKS Tile Only

During an upgrade of the PKS tile only, the PKS API server is recreated.

When the PKS API server is recreated, you cannot interact with the PKS control plane or manage Kubernetes clusters. These restrictions prevent you from performing the following actions:

  • Logging in through the PKS CLI
  • Retrieving information about clusters
  • Creating and deleting clusters
  • Resizing clusters

Recreating the PKS API server does not affect deployed Kubernetes clusters and their workloads. You can still interact with them through the Kubernetes Command Line Interface, kubectl.

To upgrade the PKS tile only, set the Upgrade all clusters errand to Off before you begin the upgrade. For more information, see the Upgrade the PKS Tile section of Upgrading PKS.

For more information about the PKS control plane, see PKS Control Plane Overview in PKS Cluster Management.

Note: When PKS is set to upgrade only the PKS tile and not the clusters, the Kubernetes cluster version falls behind the PKS tile version. If the clusters fall more than one version behind the tile, PKS cannot upgrade the clusters. The clusters must be upgraded to match the PKS tile version before the next tile upgrade.


Please send any feedback you have to pks-feedback@pivotal.io.

Create a pull request or raise an issue on the source for this page in GitHub