PCF v2.1 Breaking Changes

This topic describes the breaking changes you need to be aware of when upgrading to Pivotal Cloud Foundry (PCF) v2.1. For more information about important preparation steps you must follow before beginning an upgrade, see Upgrading Pivotal Cloud Foundry.

Pivotal Cloud Foundry

Tile Patch Releases Use Xenial Stemcells

Ubuntu 14.04 LTS (Trusty) stemcells are reaching end-of-support in April 2019. Starting in September 2018, Pivotal and its partners will begin releasing product tile versions for PCF that require Ubuntu 16.04 LTS (Xenial) stemcells instead of Trusty stemcells. Using Xenial stemcells is necessary to avoid exposure to security vulnerabilities. A PCF deployment may include some products running on Trusty stemcells and others running on Xenial.

Before deploying any tiles that require a Xenial stemcell, you must manually download and import a new Xenial stemcell into the Ops Manager Stemcell Library. As a consequence of using Xenial stemcells, any automation you have set up to update the existing tile using Trusty stemcells may break.

Before you upgrade PCF, always verify that any tiles that you have installed, either from Pivotal or a partner, are compatible with the version of PCF you are deploying and its supported stemcells. For more information on checking PCF and stemcell compatibility, see Tile Compatibility in the Upgrade Checklist.

To find out which stemcell version is used by a product tile version, check the release notes or the Pivotal Network download page for the tile.

Pivotal Application Service (PAS)

Removal of Automated Backup Configuration for Internal MySQL

Starting in PCF v2.1, the PAS tile no longer includes the Automated Backups Configuration field. This option has been removed because it is not possible to restore the internal MySQL database from a full backup without degrading the Galera MySQL cluster.

To back up and restore the internal MySQL database, you must use BOSH Backup and Restore (BBR). See Backing Up and Restoring Pivotal Cloud Foundry for information on using BBR.

BBR provides the following advantages over the Automated Backup Configuration:

  • BBR locks the necessary APIs as part of the backup procedure. This release-level backup ensures correctness. See PAS Component Behavior During Backup.
  • BBR backs up the MySQL cluster and the blobstore together so that they are consistent.
  • BBR eliminates the need to manually remove the silk database table after restore.

Deploy Fails When NSX-T CNI Plugin Option Selected

Deploys of PAS 2.1.0 fail when configured to use the NSX-T CNI plugin. The output of the error is as follows:

Task 47 | 14:12:38 | Preparing deployment: Preparing deployment (00:00:01)
                   L Error: Colocated job 'cni' is already added to the instance group 'diego_cell'.
Task 47 | 14:12:39 | Error: Colocated job 'cni' is already added to the instance group 'diego_cell'.

Until a fix is available in a patch release of PAS 2.1, Pivotal recommends that you use PAS 2.0 if NSX-T CNI integration is a requirement.

PAS for Windows Requires Injector Step

Unlike other tiles, the PAS for Windows tile requires an additional step between downloading the product file from Pivotal Network and uploading it to Ops Manager.

After you download the product file, you need to download and run the PAS for Windows File System Injector tool on the file locally, to add the Windows Server container base image. See Installing and Configuring PAS for Windows for details. You can download the injector tool from the PAS for Windows page on Pivotal Network.

If you try to upload the PAS for Windows product file without having run the injector tool, Ops Manager may show the following error: is invalid: has an invalid release 'windows2016fs' specified for job type 'Windows Diego Cell'

Some Upgrades Require VM Re-Creation

Due to an update to garden-runc-release, Pivotal recommends that you re-create all VMs when you upgrade from the following older versions to current versions of PAS and PCF Isolation Segment (IST) and you do not also update your stemcell:

  • PAS, Small Footprint Runtime, and Elastic Runtime:

    • v2.0.0 - v2.0.6
    • v1.12.0 - v1.12.15
    • v1.11.0 - v1.11.27
    • v1.10.x, 1.9.x, 1.8.x
  • PCF Isolation Segment:

    • v2.0.0 - v2.0.5
    • v1.12.0 - v1.12.14
    • v1.11.0 - v1.11.25
    • v1.10.x

VMs re-create automatically when you update your stemcell. Otherwise, you can re-create all VMs for both PAS and IST by enabling the Recreate All VMs checkbox on the PAS tile > Director Config pane.

Read-only Volume Mounts

We back-ported a fix from NFS 1.3.1 to NFS 1.2.1 for an incompatibility between our NFS Volume release and Diego’s container runtime, garden. But, because the fix was in the NFS Service Broker, and service bindings created by old versions of this broker won’t get migrated during upgrade, existing NFS service bindings that specify read-only mounts will still exhibit the incompatibility.

As a result, customers upgrading from versions containing nfs-volume-release < 1.2.1 that have NFS services bound read-only to their applications will see that their applications crash after upgrade.

To fix this condition, customers should unbind the service, rebind it, and then restage the application.

Alternately, customers wishing to avoid application down time can temporarily re-bind their applications as read/write before upgrading, and then swtich to read-only afterwards.

Diego Cell Garden Healthcheck Fails and Becomes Unhealthy

You may observe the following error in the Diego Cell Garden logs:

cfnetworking: cni up failed: add network list failed: initialize net out: creating chain: iptables call: running [/var/vcap/packages/iptables/sbin/iptables -t filter -N netout--executor-healthcheck --wait]: exit status 1: iptables: Chain already exists.

This is a symptom of a failed Diego healthcheck, which can cause the Diego cell to become unhealthy. If you encounter this error, see the following Pivotal Knowledge Base article to understand and resolve the issue: Diego Cell Garden healthcheck fails and becomes unhealthy.

Ops Manager

Additional vSphere Permission Validation

As a bug fix, Ops Manager now validates vSphere vCenter permissions from the inventory service object. The following permission validations were previously missing and are now included:

  • InventoryService.Tagging.CreateTag
  • InventoryService.Tagging.DeleteTag
  • InventoryService.Tagging.EditTag

When you click Apply Changes with a vSphere deployment and you do not have these permissions configured, you may receive the following error:

Could not log in: Required Datacenter privileges not available: 
["Host.Inventory.EditCluster", "InventoryService.Tagging.CreateTag", 
"InventoryService.Tagging.DeleteTag", "InventoryService.Tagging.EditTag"]

Include the required datacenter privileges to continue. For more information about vSphere vCenter privileges, see vSphere Service Account Requirements.

Azure Stack Support is in Beta for Ops Manager

Operators can deploy Ops Manager v2.0 to Microsoft Azure in their own local datacenter using Azure Stack. Azure Stack support is in beta for Ops Manager v2.0 and should not be used in production.

Using Azure Stack in production deployments may result in unpredictable behavior.

PCF Isolation Segment

There are no known breaking changes in PCF Isolation Segment v2.1 at this time.

Create a pull request or raise an issue on the source for this page in GitHub