Upgrading Pivotal Cloud Foundry

Page last updated:

Note: Elastic Runtime has been renamed Pivotal Application Service.

This topic describes upgrading Pivotal Cloud Foundry (PCF) to v2.0. The upgrade procedure below describes upgrading Pivotal Cloud Foundry Operations Manager (Ops Manager), Pivotal Application Service (PAS), and product tiles.

The apps in your deployment continue to run during the upgrade. However, you cannot write to your deployment or make changes to apps during the upgrade.

WARNING: Read the Release Notes and Breaking Change for this release, including the Known Issues sections, before starting the upgrade process.

For more details about the impact of upgrading on individual PCF components, see Understanding the Effects of Single Components on a Pivotal Cloud Foundry Upgrade.

Before You Upgrade

This section contains important preparation steps, Preps, that you must follow before beginning an upgrade to PCF v2.0. Failure to follow these instructions may jeopardize your existing deployment data and cause your upgrade to fail.

Pivotal recommends that you back up your PCF deployment before upgrading. In case of failure, you can restore your deployment from the backup. To back up your PCF deployment, follow the instructions in Backing Up Pivotal Cloud Foundry with BBR.

Breaking Change: If you have the Single Sign-On (SSO) service tile installed, you must first upgrade the SSO tile to v1.5.3 and configure the Apps Manager errand before you can upgrade the PAS tile to v2.0. To properly upgrade your SSO tile, see Upgrade SSO Service Tile resource instance between PCF 1.12 and PCF 2.0 in the Pivotal Knowledge Base.

Prep 1: Review File Storage IOPS and Other Upgrade Limiting Factors

The PCF upgrade process moves a large quantity of data on disk.

To ensure a successful upgrade of PCF, verify that your underlying PAS file storage is performant enough to handle the upgrade. For more information about the configurations to evaluate, see Upgrade Considerations for Selecting Pivotal Cloud Foundry Storage.

In addition to file storage IOPS, other existing deployment factors can impact overall upgrade duration and performance.

Factor Impact
Network latency Network latency can contribute to how long it takes to move app instance data to new containers.
Number of ASGs A large number of Application Security Groups in your deployment can contribute to an increase in app instance container startup time.
Number of app instances and application growth A large increase in the number of app instances and average droplet size since the initial deployment can increase the upgrade impact on your system.

To review example upgrade-related performance measurements of an existing production Cloud Foundry deployment, see Pivotal Web Services Performance During Upgrade.

Prep 2: Verify App Usage Service Remedial Steps

If you want to upgrade from a PCF deployment that at one point included Elastic Runtime v1.7.16 or earlier, you must perform the remedial steps outlined in App Usage Data and Events Data Become Corrupted After Upgrade or Install before proceeding.

WARNING: If you fail to perform the remedial steps for this issue, the upgrade process may corrupt your existing usage data.

Prep 3: Check Certificate Authority Expiration Dates

Depending on the requirements of your deployment, you may need to rotate your Certificate Authority (CA) certificates. The non-configurable certificates in your deployment expire every two years. You must regenerate and rotate these certificates so that critical components do not face a complete outage.

Note: PCF uses SHA-2 certificates and hashes by default. You can convert existing SHA-1 hashes into SHA-2 hashes by rotating your Ops Manager certificates using the procedure described in Regenerating and Rotating Non-Configurable TLS/SSL Certificates.

To retrieve information about all the RSA and CA certificates for the BOSH Director and other products in your deployment, you can use the GET /api/v0/deployed/certificates?expires_within=TIME request of the Ops Manager API.

In this request, the expires_within parameter is optional. Valid values for the parameter are d for days, w for weeks, m for months, and y for years. For example, to search for certificates expiring within one month, replace TIME with 1m:

$ curl "https://OPS-MAN-FQDN/api/v0/deployed/certificates?expires_within=1m" \
 -X GET \
 -H "Authorization: Bearer UAA_ACCESS_TOKEN"

For information about regenerating and rotating CA certificates, see Managing Non-Configurable TLS/SSL Certificates.

Prep 4: Review Partner Service Tiles

Some partner service tiles may be incompatible with PCF v2.0. Pivotal works with partners to ensure their tiles are updated to work with the latest versions of PCF. For information about which partner service releases are currently compatible with PCF v2.0, review the appropriate partners services release documentation at http://docs.pivotal.io, or contact the partner organization that produces the service tile.

Prep 5: Download Upgrade Versions

To minimize disruptions to your deployment during the upgrade, and to satisfy any simultaneous upgrade requirements, download the version of the product files you wish to upgrade from Pivotal Network.

At the minimum, you must download PAS v2.0.x.

Prep 6: Prepare Your Environment

  1. Install the releases from your currently deployed version to the target version in sequential order. For example, if your deployment uses Ops Manager v1.8 and you are upgrading to v2.0, you must sequentially install v1.9, v1.10, v1.11, and v1.12 before proceeding with the upgrade to v2.0.

  2. If you have disabled lifecycle errands for any installed product to reduce deployment time, Pivotal recommends that you re-enable these errands before upgrading. For more information, see Adding and Deleting Products.

  3. Confirm that you have adequate disk space for your upgrades. You need at least 20 GB of free disk space to upgrade PCF Ops Manager and Pivotal Application Service (PAS). If you plan to upgrade other products, the amount of disk space required depends on how many tiles you plan to deploy to your upgraded PCF deployment.

    To check current persistent disk usage, select the Ops Manager Director tile from the Installation Dashboard. Select Status and review the value of the PERS. DISK column. If persistent disk usage is higher than 50%, select Settings > Resource Config, and increase your persistent disk space to handle the size of the resources. If you do not know how much disk space to allocate, set the value to at least 100 GB.

  4. If not already disabled, disable the VM Resurrector:

    1. From your Installation Dashboard, select the Ops Manager Director tile.
    2. Click Director Config.
    3. Clear the Enable VM resurrector plugin checkbox.
    4. Click Save.
    5. Return to the Installation Dashboard and click Apply Changes.
  5. If your original deployment was PCF v1.6 or earlier, rotate non-configurable Director certificates using the Ops Manager API. Follow Step 3: Regenerate Non-Configurable Certificates to Apply the New CA of the Managing Non-Configurable TLS/SSL Certificates topic to regenerate certificates, or begin with Step 1: Add a New CA to also update the certificate authority (CA).

  6. Check the required machine specifications for Ops Manager v2.0. These specifications depend on your IaaS. If these specifications do not match your existing Ops Manager, modify the values in your Ops Manager virtual machine (VM) instance. For example, if the boot disk of your existing Ops Manager is 50 GB and the new Ops Manager requires 100 GB, increase the size of your Ops Manager boot disk to 100 GB.

  7. If you are upgrading a vSphere environment, ensure that you have the following information about your existing environment before starting the upgrade:

    • IP Addresses: Record the following IP addresses, which you can find in the vSphere web client under Manage > Settings > vApp Options. This is the same information you entered at the end of deploying Ops Manager on vSphere.
      • IP Address of the Ops Manager
      • Netmask
      • Default Gateway
      • DNS Servers
      • NTP Servers
    • Hardware Information: Record the following VM hardware information so you can configure the new VM with similar settings. You can find this information in the vSphere web client under Manage > Settings > VM Hardware.
      • CPU
      • Memory
      • Hard Disk 1
      • Network Adapter 1. When you configure the new VM, ensure your network adapters are configured properly and are on the same network.

Prep 7: Upgrade MySQL for PCF

If your PCF deployment includes both PAS for Windows 2012R2 and MySQL for PCF v1.x, download and upgrade to MySQL for PCF v1.10.6 or later. For upgrade instructions, see the MySQL for PCF documentation.

Prep 8: Upgrade and Configure RabbitMQ for PCF

If your PCF deployment contains RabbitMQ for PCF, download and upgrade RabbitMQ for PCF to v1.11 or later. For upgrade instructions, see the RabbitMQ for PCF documentation. As part of the upgrade, ensure that the firewall rules on your Rabbit VM instances allow inbound traffic on port 8301.

Prep 9: Upgrade and Configure Redis for PCF

If your PCF deployment contains Redis for PCF, download and upgrade to Redis for PCF v1.10. Verify that you complete the following as part of the upgrade:

For upgrade instructions, see the Redis for PCF documentation.

Prep 10: Check OS Compatibility of PCF and BOSH-Managed Add-Ons

If both the following conditions are true, you must modify the add-on manifest to specify a compatible OS stemcell before upgrading to PCF v2.0:

  • You have deployed any PCF add-ons such as ClamAV for PCF, IPsec for PCF, or File Integrity Monitoring for PCF

  • You have deployed or are planning to deploy PAS for Windows 2012R2.

For example, if you deployed ClamAV for PCF and you plan to deploy PAS for Windows 2012R2, you must specify the target OS stemcell of ubuntu-trusty in the add-on manifest.

To update an add-on manifest, perform the following steps:

  1. Locate your existing add-on manifest file. For example, for ClamAV, locate the clamav.yml you uploaded to the Ops Manager VM.

  2. Modify the manifest to include following include directive:

      include:
        stemcell:
          - os: ubuntu-trusty
    
  3. Upload the manifest file to your PCF deployment. For example instructions, see Create the ClamAV Manifest.

If you use any other BOSH-managed add-ons in your deployment, you should verify OS compatibility for those component as well. For more information about configuring BOSH add-on manifests, see the BOSH documentation.

Prep 11: Check System Health Before Upgrade

  1. Run bosh cloud-check to confirm that the VMs in your deployment are healthy. For more information, see BOSH Cloud Check in Advanced Troubleshooting with the BOSH CLI.

  2. Check the system health of installed products. In the Installation Dashboard, select the Status tab for each service tile. Confirm that all jobs are healthy.

  3. (Optional) Check the logs for errors before proceeding with the upgrade. For more information, see Viewing Logs in the Command Line Interface in Application Logging in Cloud Foundry.

  4. Confirm no outstanding changes exist in Ops Manager or any other tile. All tiles should display green. Click Apply Changes if necessary.

  5. After applying changes, click Recent Install Logs to confirm that the changes completed cleanly. You should see the following message:

    Cleanup complete
    {"type": "step_finished", "id": "clean_up_bosh.cleaning_up"}
    Exited with 0.
    

Upgrade Ops Manager and Installed Products to v2.0

Step 1: Export Your Ops Manager

  1. In your Ops Manager v1.12.x Installation Dashboard, click the account drop-down menu and select Settings.

    Upgrade to 1.9

  2. On the Settings screen, select Export Installation Settings from the left menu, then click Export Installation Settings. This exports the current Ops Manager installation with all of its assets.

    Export install settings

When you export an installation, the export contains the base VM images and necessary packages, and references to the installation IP addresses. As a result, an exported file can be very large, 5 GB or more.

  • The export time depends on the size of the exported file.
  • Some browsers do not provide feedback on the status of the export process and might appear to hang.

Note: Some operating systems automatically unzip the exported installation. If this occurs, create a ZIP file of the unzipped export. Do not start compressing at the “installation” folder level. Instead, start compressing at the level containing the config.yml file:

Compress

Step 2: Deploy Ops Manager v2.0

  1. Download the Ops Manager VM Template v2.0.x from Pivotal Network.

  2. Record the FQDN of the existing Ops Manager VM.

  3. To avoid conflicts, power off the existing Ops Manager VM.

  4. Deploy the new Ops Manager VM by following the steps in one of these topics:

Step 3: Import Previous Ops Manager Installation

  1. When redirected to the Welcome to Ops Manager page, select Import Existing Installation.

    Welcome

  2. When prompted, enter the Decryption Passphrase for this Ops Manager installation. You set this passphrase during your initial installation of Ops Manager.

    Note: If lost, the Decryption Passphrase cannot be recovered.

  3. Click Choose File and browse to the installation ZIP file exported in Step 1 above.

    Decryption passphrase

  4. Click Import.

    Note: Some browsers do not provide feedback on the status of the import process, and might appear to hang.

  5. A “Successfully imported installation” message appears upon completion.

    Success

Step 4: Upgrade PAS and Product Tiles

After upgrading to Ops Manager v2.0, follow the step below to upgrade your PAS and product tiles.

  1. Import the product file to your Ops Manager Installation Dashboard.

  2. Hover over the product name in Available Products and click Add.

  3. Click the newly added tile to review any configurable options.

  4. (Optional) If you are using other service tiles, you can upgrade them following the same procedure. See Upgrading PAS and Other Pivotal Cloud Foundry Products for more information.

Step 4: Complete the Upgrade

  1. Navigate to the Ops Manager Installation Dashboard.

  2. Click Apply Changes. This immediately imports and applies upgrades to all tiles in a single transaction.

    WARNING: If the installation fails or returns errors, contact Support. Do not attempt to roll back the upgrade by restarting the previous (v1.12.x) Ops Manager VM.

  3. Click each service tile, select the Status tab, and confirm that all VMs appear and are in good health.

  4. After confirming that the new installation functions correctly, remove the previous (v1.12.x) Ops Manager VM.

After You Upgrade

Upgrade Cloud Foundry Command Line Interface

To use the experimental push commands, multiple buildpack support, and container networking commands in PCF v2.0, make sure you have the latest version of the Cloud Foundry Command Line Interface (cf CLI). For more information, see Using the cf CLI.

Create a pull request or raise an issue on the source for this page in GitHub