Preparing for Upgrading MySQL for PCF

Page last updated:

This topic explains how to prepare for upgrading the MySQL for Pivotal Cloud Foundry (PCF) service and existing service instances.

Overview

MySQL for PCF v2.5 requires service bindings to use BOSH DNS and does not support service bindings that use IP addresses. But apps that use older versions of the MySQL service may bind to their instances by IP address.

To avoid failure and app downtime when upgrading to MySQL for PCF to v2.5, all apps that bind to the service by IP address must be re-bound and restarted before you upgrade.

To give developers time to unbind, rebind, and restart their apps, perform the upgrade procedures on this page well in advance of upgrading the MySQL for PCF tile.

Check for Bindings with IP Addresses

To check for running app or service key bindings that use IP addresses, do the following:

  1. Run the following command and record the BOSH deployment name for your MySQL service broker:

    bosh deployments
    
  2. Run the following command, which lists bindings that use IP addresses or do not use TLS:

    bosh -d pivotal-mysql-GUID run-errand find-deprecated-bindings
    

    Where pivotal-mysql-GUID is the BOSH deployment name for your MySQL service broker.

    For example:

    $ bosh -d pivotal-mysql-e3ddd36247fe5b923caf run-errand find-deprecated-bindings
    Using environment '10.0.0.5' as client 'ops_manager'
    
    Using deployment 'pivotal-mysql-e3ddd36247fe5b923caf'
    
    Task 148
    
    Task 148 | 02:42:08 | Preparing deployment: Preparing deployment (00:00:01)
    Task 148 | 02:42:09 | Running errand: dedicated-mysql-broker/63f9ad1e-998e-451e-8c6e-f6211958f6fb (0) (00:00:03)
    Task 148 | 02:42:12 | Fetching logs for dedicated-mysql-broker/63f9ad1e-998e-451e-8c6e-f6211958f6fb (0): Finding and packing log files (00:00:01)
    
    Task 148 Started Tue Dec 18 02:42:08 UTC 2018
    Task 148 Finished Tue Dec 18 02:42:13 UTC 2018
    Task 148 Duration 00:00:05
    Task 148 done
    
    Instance   dedicated-mysql-broker/63f9ad1e-998e-451e-8c6e-f6211958f6fb
    Exit Code  0
    Stdout     +---------------------------+--------------------------------------+------------------------+--------------------------+--------------------+-------------------+-----------------------------+
               |          SERVICE          |             SERVICE GUI             |          ORG           |          SPACE           | APP OR SERVICE KEY |       TYPE        |           REASON            |
               +---------------------------+--------------------------------------+------------------------+--------------------------+--------------------+-------------------+-----------------------------+
               | tlsDB                     | a999db0b-176e-4ac8-8342-d72b338d1f0c | MYSQL-ORG-upgrade-test | MYSQL-SPACE-upgrade-test | user-cli           | ServiceKeyBinding | no tls                      |
               | tlsDB                     | a999db0b-176e-4ac8-8342-d72b338d1f0c | MYSQL-ORG-upgrade-test | MYSQL-SPACE-upgrade-test | user-cli           | ServiceKeyBinding | no dns: hostname="10.0.8.6" |
               | upgrade-outdated-instance | 34f26746-fb46-4f14-87bc-e1ddce26f340 | MYSQL-ORG-upgrade-test | MYSQL-SPACE-upgrade-test | cs-accept          | AppBinding        | no dns: hostname="10.0.8.5" |
               | tlsDB                     | a999db0b-176e-4ac8-8342-d72b338d1f0c | MYSQL-ORG-upgrade-test | MYSQL-SPACE-upgrade-test | cs-accept-tls      | AppBinding        | no dns: hostname="10.0.8.6" |
               +---------------------------+--------------------------------------+------------------------+--------------------------+--------------------+-------------------+-----------------------------+
    
  3. Perform the Remove IP-Based Bindings procedure below for any apps or service keys that list no dns in the REASON column of the output table.

    Note: You do not need to replace bindings listed as no tls.

Remove IP-Based Bindings

Some of the following steps require coordination with your app developers. To remove deprecated bindings, do the following:

  1. Have developers replace their IP-based service bindings with bindings that use BOSH DNS. See Re-Bind Apps to Use DNS below.
  2. Upgrade your service instances to be compatible with BOSH DNS. See Upgrade Service Instances below.

Re-Bind Apps to Use DNS

To replace your deprecated service bindings with new service bindings that use DNS hostnames, developers must do the following for each binding listed in the log error above:

  1. To unbind your app from your service instance, run the following command:

    cf unbind-service APP-NAME SERVICE-INSTANCE
    
  2. To re-bind your app to your service instance, run the following command:

    cf bind-service APP-NAME SERVICE-INSTANCE
    
  3. To restart your app, run the following command:

     cf restart APP-NAME
    

    After you restart all of your apps, your apps can connect to the database and resume service.

  4. To ensure that all deprecated service bindings have been replaced, run the following errand:

     bosh -d pivotal-mysql-GUID run-errand find-deprecated-bindings
    
  5. Ensure that no dns does not appear in the REASON column in the output table.

Upgrade Service Instances

To upgrade your service instances, operators must do the following:

  1. Notify the developers who own the apps listed by the find-deprecated-bindingsabove that you will be upgrading all service instances, which can impact app connectivity to the database.
  2. Log in to the BOSH Director by following the procedure in Log in to the BOSH Director VM.
  3. Locate the BOSH deployment name for your MySQL service broker, by running the following command:

    bosh deployments
    
  4. Record the BOSH deployment name for the MySQL service broker, which looks like pivotal-mysql-GUID.

  5. Upgrade all service instances by running the following command:

    bosh -d pivotal-mysql-GUID run-errand upgrade-all-service-instances
    

    Where pivotal-mysql-GUID is the value you recorded in the previous step.

  6. Notify your developers that, at this time, they will see that apps that do hostname validation cannot establish a TLS connection to the database. These apps will output an error similar the following example:

    failed to perform TLS handshake with host "10.0.8.5:3306": x509: certificate is valid for 127.0.0.1, not 10.0.8.5