LATEST VERSION: 2.5 - RELEASE NOTES

Installing and Configuring MySQL for PCF

Page last updated:

This topic provides instructions to operators of Pivotal Cloud Foundry (PCF) about how to install, configure, and deploy the MySQL for PCF v2.5 tile. The MySQL for PCF v2.5 service lets PCF developers create and use their own MySQL service instances on demand.

Role-Based Access in Ops Manager

Ops Manager administrators can use Role-Based Access Control (RBAC) to manage which operators can make deployment changes, view credentials, and manage user roles in Ops Manager. Therefore, your role permissions might not allow you to perform every procedure in this operator guide.

For more information about roles in Ops Manager, see Understand Roles in Ops Manager.

Prepare Your Ops Manager and PCF Installation for MySQL for PCF

Before you download and install the MySQL for PCF tile, complete the following procedures:

Create an Application Security Group for MySQL for PCF

To let apps running on Cloud Foundry communicate with the MySQL service network, you must create an Application Security Group (ASG).

The ASG allows smoke tests to run when you first install the MySQL for PCF service and allows apps to access the service after it is installed.

The example below follows this procedure to create an ASG.

To create an ASG for MySQL for PCF, do the following:

  1. Navigate to the Ops Manager Installation Dashboard and click the BOSH Director tile.

  2. Click Create Networks.

  3. Find the network that has Service Network checked, and find the CIDR that you can use in your ASGs.
    Reserved IP Ranges

  4. Using the CIDR that you found in the above step, create a JSON file mysql-asg.json with the configuration below:

    [
      {
        "protocol": "tcp",
        "destination": "CIDR",
        "ports": "3306"
      }
    ]
    

    If you want to store your service instance credentials in runtime CredHub, you must also open port 8844 in addition to performing the steps in Configure Secure Service Instance Credentials below. Modify the JSON file above to include "ports": "3306,8844".

  5. Use the CF CLI and the JSON file that you created to create an ASG called p.mysql:

    $ cf create-security-group p.mysql ./mysql-asg.json
    
  6. Bind the ASG to the appropriate space or, to give all started apps access, bind to the default-running ASG set:

    $ cf bind-running-security-group p.mysql
    

Enable the BOSH Resurrector

The BOSH Resurrector increases the availability of MySQL for PCF by restarting and resuming MySQL service in the following ways:

  • Reacts to hardware failure and network disruptions by restarting VMs on active, stable hosts
  • Detects operating system failures by continuously monitoring VMs and restarting them as required
  • Continuously monitors the BOSH Agent running on each service instance VM and restarts the VM as required

Pivotal recommends enabling the BOSH Resurrector when installing MySQL for PCF. To enable the BOSH Resurrector, do the following:

  1. Navigate to the Ops Manager Installation Dashboard and click the BOSH Director tile.

  2. Click Director Config.

  3. Select the Enable VM Resurrector Plugin checkbox.

  4. Click Save.

For general information about the BOSH Resurrector, see Using BOSH Resurrector

Download and Install the Tile

  1. Download the product file from Pivotal Network.

  2. Navigate to the Ops Manager Installation Dashboard and click Import a Product to upload the product file.

  3. Under the Import a Product button, click + next to the version number of MySQL for PCF. This adds the tile to your staging area.

  4. Click the newly-added MySQL for PCF tile to open its configuration panes.

    AZ and Network Assignments pane

Configure the Tile

Follow the steps below to configure the MySQL for PCF service. MySQL for PCF v2.5 has nine service plans that deploy dedicated MySQL service instances on demand.

Configure AZs and Networks

Follow the steps below to choose an availability zone (AZ) to run the service broker and to select networks.

  1. Click Assign AZs and Networks.

  2. Configure the fields as follows:

    FieldInstructions
    Place singleton jobs in Select the AZ that you want the MySQL broker VM to run in. The broker runs as a singleton job.
    Balance other jobs in Ignore; not used.
    Network Select a subnet for the MySQL broker. This is typically the same subnet that includes the PAS component VMs.
    This network is represented by the Default Network in this picture.
    Service Network Select the subnet for the on-demand service instances, the Services Network in this picture.
    If you are adding IPsec to encrypt MySQL communication, Pivotal recommends that you deploy MySQL to its own network to avoid conflicts with services that are not IPsec compatible.

    IMPORTANT: You cannot change the regions or networks after you click Apply Changes in the final step below.

  3. Click Save.

Configure Service Plans

Warning: Highly available (HA) cluster service plans are currently in beta. HA clusters are for advanced use cases only.

MySQL for PCF enables you to configure as many as nine service plans. Each service plan has a corresponding section in the tile configuration, such as Plan 1, Plan 2, and so on.

By default, plans 1–3 are active and plans 4–9 are inactive. The procedures below describe how to change these defaults.

WARNING: You must configure one of your service plans as a single node plan. This is because you can only restore to a single node plan. You cannot restore directly to a leader-follower plan. Ensure that the persistent disk in the single node plan is as least as large as the persistent disk of your largest leader-follower plan. For more information on restoring services instances, see Restore the Service Instance.

Perform the following steps for each plan that you want to use in your PCF deployment:

  1. Click the section for the plan. For example, Plan 1.

  2. Select the plan for your desired topology. Click the tabs below to show an example of each plan:

  3. Configure the fields as follows:

    FieldDescription
    Service Plan Access Select one of the following options:
    • Enable (Default)—Gives access to all orgs, and displays the service plan to all developers in the Marketplace.
    • Disable—Disables access to all orgs, and hides the service plan to all developers in the Marketplace. This disables creating new service instances of this plan.
    • Manual—Lets you manually control service access with the cf CLI. For more information, see Controlling Access to Service Plans by Org.
    Plan Name Accept the default or enter a name. This is the name that appears in the Marketplace for developers.
    Plan Description Accept the default or enter a description to help developers understand plan features. Pivotal recommends adding VM type details and disk size to this field.
    Plan Quota Enter the maximum number of service instances that can exist at one time. If the plan quota field is blank, the plan quota is set to the global quota by default. If you have selected the highly available cluster plan, the Plan Quota maximum is 5. This is because the highly available cluster plan is in beta.
    Paid Plan Check this box to indicate that this service plan is paid.
    MySQL VM Type Select a VM type for the MySQL nodes.
    Jumpbox VM Type Only for highly available cluster plans. Select a VM type for the MySQL jumpbox node. This VM is also called mysql-monitor.
    MySQL Persistent Disk Select a disk size. This disk stores the MySQL messages.
    • Single node or leader-follower plans: Select a disk size three times larger than the disk you intend to provide to developers.
    • Highly available cluster plans: Select a disk size that is the same size you intend to provide to developers.

    Note: For backups to work properly, follow these sizing recommendations. For more information, see Persistent Disk Usage.

    Jumpbox Persistent Disk Only for highly available cluster plans. Select a disk size. This disk stores backups. This must be twice the size of the MySQL persistent disk.
    MySQL Availability Zone(s) BOSH deploys your service instances to these availability zones (AZ). If more than one AZ is selected, BOSH will randomize which AZ to place each VM.
    • Single node plans: Select one AZ for service instances of this plan to be deployed.
    • Leader-follower plans: You must select at least two AZs in order to be resilient against AZ failures.
    • Highly available cluster plans: You must select at least three AZs for this plan to be HA.

  4. Click Save.

WARNING: If you expect your developers to upgrade from one plan to another, do not place the plans in separate AZs. For example, if you create Plan 1 in AZ1 and Plan 2 in AZ2, developers receive an error and cannot continue if they try to upgrade from Plan 1 to Plan 2. This prevents them from losing their data by orphaning their disk in AZ1.

To learn how to manually migrate the data from one AZ to another, see Migrating Data in MySQL for PCF.

(Optional) Deactivate Service Plan

Follow the steps below to deactivate a service plan:

  1. If the service plan has existing service instances, perform the following steps:
    1. Click the section for the plan. For example, Plan 2.
    2. Under Service Plan Access, select Disable.
    3. Click Save.
    4. Return to the Ops Manager Installation Dashboard and click Apply Changes to redeploy.
    5. When the PCF deployment has redeployed, use the cf CLI or Apps Manager to delete all existing service instances on the service plan.
    6. Return to the MySQL for PCF tile configuration.
  2. Click the section for the plan. For example, Plan 2.
  3. Click Inactive.
  4. Click Save.

Configure Global Settings

Follow the steps below to determine if service instances are assigned public IP addresses and to set the total number of service instances allowed across all plans.

  1. Click Settings.

    global-settings

  2. Configure the fields as follows:

    FieldInstructions
    Assign Public IP addresses for all Service VMs Select this checkbox:
    • If the service instances need an external backup, blobstore, or syslog storage
    • If you have configured BOSH to use an external blobstore.
    Maximum service instances Enter the global quota for all on-demand instances summed across every on-demand plan. For information about determining global quotas, see Service Plan Recommended Usage and Limitations.
    Email address Enter an email address to send MySQL monitoring notifications to.

  3. Click Save.

Configure MySQL

Follow the steps below to set MySQL defaults and enable developers to customize their instances:

  1. Click Mysql Configuration

    mysql-config

  2. Configure the fields as follows:

FieldInstructions
Enable Lower Case Table Names Select this checkbox to store all table names in lowercase. Selecting this sets the MySQL server system variable lower_case_table_names to 1 on all MySQL for PCF instances by default. To allow developers to override this default, see the checkbox below. For more information, see lower_case_table_names in the MySQL documentation.
Allow Developers To Override Lower Case Table Names Select this checkbox to allow developers to override the default Enable Lower Case Table Names value set above, when they are creating a new service instance. For more information, see Optional Parameters for the MySQL for PCF Service Instances.
Enable Local Infile Select this checkbox to enable data downloading from the client’s local file system. For more information about local data loading capability, see Security Issues with LOAD DATA LOCAL in the MySQL 5.7 Reference Manual.
Wait Timeout Set the time in seconds that MySQL waits to close inactive connections. For more information, see wait_timeout in the MySQL 5.7 Reference Manual.

Configure Backups

Follow the steps below to configure automated backups:

Note: As of v2.2.0, the MySQL for PCF tile no longer includes the option to disable automated backups. You must configure automated backups.

  1. Click Backups.

    backups

  2. Consult About Automated Backups in Backing Up and Restoring to learn about how automated backups work.

  3. Select a Backup configuration and perform the steps in the appropriate section of Backing Up and Restoring:

    • Option 1: Back up with SCP—MySQL for PCF runs an SCP command that secure-copies backups to a VM or physical machine operating outside of PCF. SCP stands for secure copy protocol, and offers a way to securely transfer files between two hosts. The operator provisions the backup machine separately from their PCF installation. This is the fastest option.
    • Option 2: Back up to Ceph or S3—MySQL for PCF runs an Amazon S3 client that saves backups to an S3 bucket, a Ceph storage cluster, or another S3-compatible endpoint certified by Pivotal.
    • Option 3: Back up to GCS—MySQL for PCF runs an GCS SDK that saves backups to an Google Cloud Storage bucket.
    • Option 4: Back up to Azure Storage—MySQL for PCF runs an Azure SDK that saves backups to an Azure storage account.

Configure Security

Note: If you want to enable TLS in MySQL for PCF, you must follow the procedures in Preparing for TLS before configuring security.

Follow the steps below to configure the security settings for the MySQL service. Do one or both of the following:

security

Configure TLS

To enable TLS for the MySQL service, perform the following steps:

  1. Ensure that you have performed the procedures in Preparing for TLS.
  2. Click Security.
  3. Under TLS Options, select one of the following:
  4. Optional: This enables developers to configure their MySQL service VMs to use TLS.
  5. Required: This enables developers to configure their MySQL service VMs to use TLS, and requires all MySQL service VMs to only accept secure connections.

    WARNING: Selecting Required breaks any apps that are not currently connecting over TLS.

  6. Click Save.
  7. After deploying the tile, notify your developers that they must enable TLS for their service instances and activate TLS for their apps. See Using TLS.

Configure Secure Service Instance Credentials

If you want to store your service instance credentials in runtime CredHub instead of the Cloud Controller Database (CCDB), perform the following steps:

  1. Ensure that you have configured the PAS tile to support securing service instance credentials in runtime CredHub. See Step 1: Configure the PAS Tile.
  2. Ensure that you have created an ASG that opens port 8884. See Create an Application Security Group for MySQL for PCF above.
  3. Click Security.
  4. Select the Enable Secure Service Instance Credentials checkbox.
  5. Click Save.
  6. After deploying the tile, notify the developers that they must unbind and rebind any existing service instances bindings if they want to use secure service instance credentials. Perform the following steps:

    1. Unbind the service instance from the app. For example:
      $ cf unbind-service my-app my-service-instance
    2. Rebind the service instance to the app. For example:
      $ cf bind-service my-app my-service-instance
    3. Restart the app to apply the new service instance binding. For example:
      $ cf restart my-app
    4. Print the VCAP_SERVICES environment variable to verify that the new service instance binding includes CredHub pointers. For example:
      $ cf env my-app
      Getting env variables for app my-app in org system / space example as admin...
      OK
      System-Provided:
      {
      "VCAP_SERVICES": {
      "p.mysql": [
      {
      "credentials": {
       "credhub-ref": "/c/548966e5-e333-4d65-8773-7b4e3bb6ca97/4a246b0b-83bb-46d0-b8ac-35a93374ae67/caf6e32e-7361-4869-9a57-54ab8ae67b3f/credentials"
      },
      [...]
      

Configure Monitoring

Follow the steps below to enable different types of monitoring and logging available in the MySQL service.

  1. Click Monitoring.

    monitoring

  2. Configure the fields as follows:

    FieldInstructions
    Metrics Polling Interval Set this time, in seconds, to determine the frequency that the monitor polls for metrics. All service instances emit metrics about the health and status of the MySQL server.
    Enable User Statistics Logging Select this checkbox to better understand server activity and identify sources of load on a MySQL server. For more information about user statistics, see User Statistics Documentation.
    Enable Server Activity Logging Select this checkbox to record who connects to the servers and what queries are processed using the Percona Audit Log Plugin. For more information, see the Percona Documentation.
    Enable Read Only Admin User Select this checkbox to create a read-only admin user, roadmin, on each service instance. This user can be used for auditing and monitoring without risking mutating or changing any data, because the roadmin user cannot make changes.

    To retrieve the credentials for the roadmin user, see Retrieve Admin and Read-Only Admin Credentials for a Service Instance. The read-only admin user is always roadmin, but the password varies by service instance.

  3. Click Save.

Configure System Logging

Follow the steps below to enable system logging for the MySQL broker and service instance VMs. Logs use RFC5424 format.

  1. Click Syslog.

  2. Click Yes.

    syslog

  3. Configure the fields as follows:

    FieldInstructions
    Address Enter the address or host of the syslog server for sending logs, for example, logmanager.example.com.
    Port Enter the port of the syslog server for sending logs, for example, 29279.
    Transport Protocol Select the protocol over which you want system logs. Pivotal recommends using TCP.
    Enable TLS If you select TCP, you can also select to send logs encrypted over TLS.
    Permitted Peer Enter either the accepted fingerprint, in SHA1, or the name of the remote peer, for example, *.example.com
    SSL Certificate Enter the SSL Certificate(s) for the syslog server. This ensures the logs are transported securely.

    IMPORTANT: If your syslog server is external to PCF, you might need to select Provide public IP addresses to all Service VMs on the Settings page.

  4. Click Save.

Configure Service Instance Upgrades

Follow the steps below to configure service instance upgrades. This section configures the upgrade-all-service-instances errand, which MySQL for PCF uses to upgrade service instances.

  1. Click the Service Instance Upgrades tab.

    Service Instance Upgrades section

  2. Configure the fields as follows:

    FieldInstructions
    Number of simultaneous upgrades Enter the maximum number of service instances that will be in an upgrading state at the same time. The minimum value is 0 and the maximum is 1 less than the number of BOSH workers.

    Note: To determine the number of BOSH workers, navigate to the BOSH Director tile, click Director Config and locate the Director Workers field.

    Number of upgrade canary instances Enter the number of service instances to upgrade first before upgrading the rest of the instances.
    BOSH Upgrade Timeout Enter the amount of time in seconds to wait for BOSH to respond before timing out when upgrading service instances.

  3. Click Save.

(Optional) Review Errands

You do not need to change the default configuration for errands in the Errands tab.

IMPORTANT: In order to re-define plans later, you must set the Delete All Service Instances and Deregister Broker errand to On.

Errands are scripts that run to do various tasks. MySQL for PCF can run errands to manage the broker and service instances at the following points:

  • Post-Deploy Errands: Run when you click Apply Changes.
  • Pre-Delete Errands: Run before you delete the MySQL for PCF tile.

Errands

By default, MySQL for PCF runs the following errands:

Post-Deploy Errands
  • Register On-demand MySQL Broker: Registers a broker with the Cloud Controller and lists it in the Marketplace.
  • Smoke Tests: Validates basic MySQL operations.
  • Validate no IP-based bindings in use before upgrade-all-service-instances: Checks if service instances have app bindings or service keys using IP addresses, or have a TLS certificate that is signed with an IP address. If true, the installation will fail.
  • Upgrade all On-demand MySQL Service Instances: Upgrades existing instances of a service to its latest installed version.
Pre-Delete Errand
  • Delete All Service Instances and Deregister Broker: Deletes all service instances and deregisters the broker.

WARNING: The Delete All Service Instances and Deregister Broker errand does necessary cleanup tasks when you delete the MySQL for PCF tile. Setting this errand to Off can cause problems when attempting to reinstall the tile. Pivotal recommends that you do not set this errand to Off.

MySQL for PCF also uses errands to configure leader-follower service instances. For more information about leader-follower errands, see Errands Used in Leader-Follower Failover.

You can use errands when troubleshooting the broker or service instances. For more information about using errands for troubleshooting, see Run Service Broker Errands to Manage Brokers and Instances.

Verify Stemcell Version and Apply All Changes

MySQL for PCF v2.4.0 and later requires a Xenial stemcell.

Follow the steps below to verify your stemcell version and apply all changes:

  1. Click Stemcell Library. For more information about using the Stemcell Library, see Importing and Managing Stemcells.

  2. Verify and, if necessary, import a new stemcell version.

  3. In the Ops Manager Dashboard, do the following to complete the installation:

    1. Click Review Pending Changes. For more information about this Ops Manager page, see Reviewing Pending Product Changes.

    2. Click Apply Changes.
Create a pull request or raise an issue on the source for this page in GitHub