Pivotal Cloud Foundry v1.9

Upgrading Pivotal Cloud Foundry

Page last updated:

This topic describes upgrading Pivotal Cloud Foundry (PCF) to v1.9. The upgrade procedure below describes upgrading Pivotal Cloud Foundry Operations Manager (Ops Manager), Pivotal Elastic Runtime, and product tiles.

The apps in your deployment continue to run during the upgrade. However, you cannot write to your deployment or make changes to apps during the upgrade.

Important: Read the Release Notes for this release, including the Known Issues sections, before starting the upgrade process.

For more details about the impact of upgrading on individual PCF components, see the Understanding the Effects of Single Components on a Pivotal Cloud Foundry Upgrade topic.

Before You Upgrade

This section contains important preparation steps, Preps, that you must follow before beginning an upgrade to Ops Manager v1.9. Failure to follow these instructions may jeopardize your existing deployment data and cause your upgrade to fail.

Prep 1: Review File Storage IOPS and Other Upgrade Limiting Factors

During the PCF upgrade process, a large quantity of data is moved around on disk.

To ensure a successful upgrade of PCF, verify that your underlying Elastic Runtime file storage is performant enough to handle the upgrade. For more information on the configurations to evaluate, see Upgrade Considerations for Selecting Pivotal Cloud Foundry Storage.

In addition to file storage IOPS, consider additional existing deployment factors that can impact overall upgrade duration and performance:

Factor Impact
Network latency Network latency can contribute to how long it takes to move app instance data to new containers.
Number of ASGs A large number of Application Security Groups in your deployment can contribute to an increase in app instance container startup time.
Number of app instances and application growth A large increase in the number of app instances and average droplet size since the initial deployment can increase the upgrade impact on your system.

Prep 2: Upgrade to Elastic Runtime v1.8.33 or Later

To prevent potential TCP route data loss, Pivotal recommends that you upgrade to Elastic Runtime v1.8.33 or later before you upgrade to Elastic Runtime v1.9.x.

In addition, if you use an external database such as RDS on Amazon or CloudSQL on GCP, you must create the routing database and appropriate database user credentials before performing the upgrade to v1.8.33. Use the following MySQL command:

CREATE database routing;

For more information, see the Pivotal Elastic Runtime v1.8 Release Notes.

Prep 3: Verify App Usage Service Remedial Steps

If you are upgrading from a PCF deployment that at one point included Elastic Runtime v1.7.16 or earlier, you must perform the remedial steps outlined in App Usage Data and Events Data Become Corrupted After Upgrade or Install before proceeding with this upgrade.

WARNING: If you fail to perform the remedial steps for this issue, this upgrade process may corrupt your existing usage data.

Prep 4: Review Product Compatibility Prerequisites

Before upgrading to Ops Manager v1.9, you must be using the following product version:

  • Elastic Runtime v1.8.9 or later

If you are not currently using the correct version of the above product, download the correct version file from Pivotal Network and perform the update before proceeding with this upgrade procedure.

For information about how to upgrade product tiles in PCF v1.8, see the Upgrading Elastic Runtime and Other Pivotal Cloud Foundry Products topic.

Prep 5: Review RabbitMQ for PCF Version and Health Status

If you have RabbitMQ for PCF currently deployed in your PCF v1.8 environment, perform the following steps before upgrading to PCF v1.9.

  1. Review the version of RabbitMQ for Pivotal Cloud Foundry currently running in your PCF v1.8 deployment. Pivotal recommends that you upgrade to the latest patched version of RabbitMQ for PCF, v1.7.x, before proceeding with the platform upgrade.

  2. Examine the RabbitMQ for PCF management dashboard to ensure that the service status is green and that all nodes are healthy. For more information, see the RabbitMQ for Pivotal Cloud Foundry documentation.

Prep 6: Review and Remove Unsupported Products

If your deployment contains any of the following unsupported products, you must remove the associated product tile before upgrading. The following products are no longer maintained and are not compatible with PCF v1.9:

  • App Distribution for PCF
  • Data Sync for PCF
  • DataStax Enterprise for PCF
  • MongoDB for PCF
  • Neo4J for PCF
  • Pivotal HD
  • Session State Caching Powered by GemFire for PCF
  • Riak CS for PCF

Failure to remove these products prior to upgrade may cause issues with your deployment.

Prep 7: Review Partner Service Tiles

Some partner service tiles may be incompatible with PCF v1.9. Pivotal is working with partners to ensure their tiles are updated to work with the latest versions of PCF. For information about which partner service releases are currently compatible with PCF v1.9, review the appropriate partners services release documentation at, or contact the partner organization that produces the service tile.

Prep 8: Download Upgrade Versions

To minimize disruptions to your deployment during the upgrade, and to satisfy any simultaneous upgrade requirements, download the version of the product files you wish to upgrade from Pivotal Network.

At the minimum, you must download Elastic Runtime v1.9.x.

Prep 9: Prepare for CC API v3 Upgrade

The migration to PCF v1.9 removes all experimental Cloud Controller API v3 data. If you have experimented with the v3 API prior to this release, Pivotal recommends that you remove all existing v3 service bindings.

To determine if your deployment is using any v3 service bindings, perform the following steps:

  1. Log in as the admin user of your PCF deployment with the cf login command.

  2. Run the following command:

    $ cf curl /v3/service_bindings
    This command returns a JSON listing of all v3 service bindings and their GUIDs.

  3. If necessary, notify application owners of the pending deletions.

  4. To delete a service binding, run the following command:

    $ cf curl /v3/service_bindings/GUID -X DELETE
    In the above command, replace GUID with the service binding you want to delete.

  5. Repeat the deletion command for each v3 service binding until the cf curl /v3/service_bindings command returns zero results.

In addition, although existing v2 applications still run normally during the upgrade, some developer interactions with PCF such as pushing apps to PCF may fail. You may want to warn users about potential availability issues during the upgrade window.

Prep 10: Prepare Your Environment

  1. Install the releases from your currently deployed version to the target version in sequential order. For example, if your deployment uses Ops Manager v1.6 and you are upgrading to v1.9, you must sequentially install v1.7 and v1.8 before proceeding with the upgrade to v1.9.

  2. IMPORTANT: Back up all critical data prior to upgrading to Ops Manager v1.9. For example, to backup a v1.8 environment, follow the instructions in the v1.8 Backing Up Pivotal Cloud Foundry topic.

  3. If you have disabled lifecycle errands for any installed product to reduce deployment time, Pivotal recommends that you re-enable these errands before upgrading. For more information, see the Adding and Deleting Products topic.

  4. Confirm that you have adequate disk space for your upgrades. You need at least 20 GB of free disk space to upgrade PCF Ops Manager and Elastic Runtime. If you plan to upgrade other products, the amount of disk space required depends on how many tiles you plan to deploy to your upgraded PCF deployment.

    To check current persistent disk usage, select the Ops Manager Director tile from the Installation Dashboard. Select Status and review the value of the PERS. DISK column. If persistent disk usage is higher than 50%, select Settings > Resource Config, and increase your persistent disk space to handle the size of the resources. If you do not know how much disk space to allocate, set the value to at least 100 GB.

  5. If not already disabled, disable the VM Resurrector:

    1. From your Installation Dashboard, select the Ops Manager Director tile.
    2. Click Director Config.
    3. Clear the Enable VM resurrector plugin checkbox.
    4. Click Save.
    5. Return to the Installation Dashboard and click Apply Changes.
  6. If you are upgrading a vSphere environment, ensure that you have the following information about your existing environment before starting the upgrade:

    • Record the following IP addresses, which you can find in the vSphere web client under Manage > Settings > vApp Options. This is the same information you entered at the end of deploying Ops Manager on vSphere.
      • IP Address of the Ops Manager
      • Netmask
      • Default Gateway
      • DNS Servers
      • NTP Servers
    • Record the following VM hardware information so you can configure the new VM with similar settings. You can find this information in the vSphere web client under Manage > Settings > VM Hardware.
      • CPU
      • Memory
      • Hard Disk 1
      • Network Adapter 1. When you configure the new VM, ensure your network adapters are configured properly and are on the same network.

Prep 11: Check System Health Before Upgrade

  1. Run bosh cloudcheck to confirm that the VMs are healthy. For more information, see the BOSH Cloudcheck topic.

  2. Check the system health of installed products. In the Installation Dashboard, select the Status tab for each service tile. Confirm that all jobs are healthy.

  3. (Optional) Check the logs for errors before proceeding with the upgrade. For more information, see the Viewing Logs in the Command Line Interface topic.

  4. Confirm there are no outstanding changes in Ops Manager or any other tile. All tiles should be green. Click Apply Changes if necessary.

  5. After applying changes, click Recent Install Logs to confirm that the changes completed cleanly:

    Cleanup complete
    {"type": "step_finished", "id": "clean_up_bosh.cleaning_up"}
    Exited with 0.

Upgrade Ops Manager and Installed Products to v1.9

Step 1: Export Your Installation

  1. Make sure you have removed any tiles that are no longer supported. For the list of unsupported tiles, see the Review and Remove Unsupported Products section of this topic.

  2. In your Ops Manager v1.8.x Installation Dashboard, click the account dropdown and select Settings.

    Upgrade to 1.9

  3. On the Settings screen, select Export Installation Settings from the left menu, and click the Export Installation Settings button.

    Export install settings

    This exports the current PCF installation with all of its assets.

    When you export an installation, the export contains the base VM images and necessary packages, and references to the installation IP addresses. As a result, an exported file can be very large, 5 GB or more.

    • The export time depends on the size of the exported file.
    • Some browsers do not provide feedback on the status of the export process and might appear to hang.

    Note: Some operating systems automatically unzip the exported installation. If this occurs, create a ZIP file of the unzipped export. Do not start compressing at the “installation” folder level. Instead, start compressing at the level containing the config.yml file:


Step 2: Upgrade to Ops Manager v1.9

  1. Download the Ops Manager VM Template v1.9.x from the Pivotal Network site.

  2. Record the FQDN address of the existing Ops Manager VM.

  3. To avoid conflicts, power off the existing Ops Manager VM.

  4. Deploy the new Ops Manager VM by following the steps in one of these topics:

  5. When redirected to the Welcome to Ops Manager page, select Import Existing Installation.


  6. When prompted, enter a Decryption Passphrase.

    Note: Record and store your Decryption Passphrase in a safe location. If lost, the Decryption Passphrase cannot be recovered.

  7. Click Choose File and browse to the installation ZIP file exported in Step 1 above.

    Decryption passphrase

  8. Click Import.

    Note: Some browsers do not provide feedback on the status of the import process, and might appear to hang.

  9. A “Successfully imported installation” message appears upon completion.


Step 3: Upgrade Elastic Runtime and Product Tiles

  1. After upgrading to Ops Manager v1.9, upgrade your product versions.

  2. Import the product file to your Ops Manager Installation Dashboard.

  3. Hover over the product name in Available Products and click Add.

  4. Click the newly added tile to review any configurable options.

  5. (Optional) If you are using other service tiles, you can upgrade them following the same procedure. See the Upgrading Elastic Runtime and Other Pivotal Cloud Foundry Products topic for more information.

Step 4: Complete Your Installation

  1. Navigate to the Ops Manager Installation Dashboard.

  2. Click Apply Changes. This immediately imports and applies upgrades to all tiles in a single transaction.

    WARNING: If the installation fails or returns errors, contact Support. Do not attempt to roll back the upgrade by restarting the previous (v1.8.x) Ops Manager VM.

  3. Click each service tile, select the Status tab, and confirm that all VMs appear and are in good health.

  4. After confirming that the new installation functions correctly, remove the previous (v1.8.x) Ops Manager VM.

This completes the upgrade to Ops Manager v1.9.

After the Upgrade

The following post upgrade step only applies to users of PCF Metrics.

After the upgrade to PCF v1.9, LogSearch may fail to reconnect to BOSH when it restarts. As a result, after an upgrade to PCF v1.9, not all expected logs may appear in LogSearch.

To remedy this issue:

  1. In Kibana, search for *nats_to_syslog to verify that you have log data with a timestamp after the upgrade period.

  2. If you are missing log data after the upgrade, run the following command on the Ingestor VM to restart the nats_to_syslog job:

    $ monit restart nats_to_syslog

  3. Recheck Kibana for updated log data.

Create a pull request or raise an issue on the source for this page in GitHub