Managing Certificates

Page last updated:

This topic describes how to check expiration dates and rotate certificates used by MySQL for PCF.

Overview

MySQL for PCF uses the following types of certificates:

  • TLS certificates managed by the operator: If you are using TLS for MySQL for PCF, you provided a CA certificate to BOSH CredHub when you did the procedures in Preparing for TLS. Operators must rotate this TLS CA certificate before the certificate expires or if it is compromised.
  • Internal certificates managed by MySQL for PCF: In MySQL for PCF v2.5, you cannot rotate internal certificates and CA certificates. To rotate these certificates, you must upgrade to MySQL for PCF v2.6. To check the expiration date of your internal certificates, see Check Expiration Date below.

Warning: If your internal certificates expire, service bindings and backups can fail.

To rotate the TLS certificates, do the following:

  1. Access BOSH CredHub
  2. Check Expiration Dates
  3. Rotate TLS Certificates

Warning: Do not attempt to rotate a CA certificate on your own. Contact Pivotal Support and perform the following procedures with their help.

MySQL for PCF Certificates

The following table lists MySQL for PCF certificates:

Certificate
TLS Certificates
/services/tls_ca
mysql_tls
mysql_server_tls
Internal Certificates
/pivotal_mysql_GUID/agent_ca
/pivotal_mysql_GUID/agent_client_ssl
/pivotal_mysql_GUID/agent_server_ssl
/service_instance_GUID/agent_ca
/service_instance_GUID/lf_agent_client_ssl
/service_instance_GUID/lf_agent_server_ssl
/service_instance_GUID/agent_client_ssl
/service_instance_GUID/agent_server_ssl
/service_instance_GUID/streaming_backup_ca
/service_instance_GUID/streaming_backup_server_cert
/service_instance_GUID/pxc_tls_ca
/service_instance_GUID/pxc_tls_server
/service_instance_GUID/pxc_internal

In the above table, GUID is the GUID for the service instance. To find the GUID of your service instance, do the procedure in Find Information about Your Service Instance.

Access BOSH CredHub

Pivotal Cloud Foundry (PCF) stores some of its internal CA and non-CA certificates in the BOSH CredHub credentials store. For more information, see BOSH CredHub.

To access the BOSH CredHub credentials store, you must retrieve credentials from BOSH Director and then use the credentials to log in to CredHub from the Ops Manager VM.

To access the BOSH CredHub, do the following:

  1. If the CredHub CLI is not installed on your local machine, install the CredHub CLI by doing the instructions in credhub-cli on GitHub.

  2. In the Ops Manager Installation Dashboard, click the BOSH Director tile.

  3. Click the Credentials tab.

  4. In the BOSH Director section, click the link to the BOSH Commandline Credentials. CredHub Credentials

  5. Record the values for BOSH_CLIENT and BOSH_CLIENT_SECRET.
    For example:

    {"credential":"BOSH_CLIENT=ops_manager
    BOSH_CLIENT_SECRET=abCdE1FgHIjkL2m3n-3PqrsT4EUVwXy5
    BOSH_CA_CERT=/var/tempest/workspaces/default/root_ca_certificate
    BOSH_ENVIRONMENT=10.0.0.5 bosh "}
    

    The BOSH_CLIENT is the BOSH CredHub client name and the BOSH_CLIENT_SECRET is the BOSH CredHub client secret.

  6. Record the information needed to log in to the BOSH Director VM by doing the procedure in Gather Credential and IP Address Information.

  7. Log in to the Ops Manager VM by doing the procedure in Log in to the Ops Manager VM with SSH.

  8. From the Ops Manager VM, set the API target of the CredHub CLI to your BOSH CredHub server by running the following command:

    credhub api https://BOSH-DIRECTOR:8844 --ca-cert=/var/tempest/workspaces/default/root_ca_certificate
    

    Where BOSH-DIRECTOR is the IP address of the BOSH Director VM you recorded above.

    For example:

    $ credhub api https://10.0.0.5:8844 --ca-cert=/var/tempest/workspaces/default/root_ca_certificate
    

  9. Log in to CredHub by running the following command:

    credhub login --client-name=CREDHUB-CLIENT-NAME --client-secret=CREDHUB-CLIENT-SECRET
    

    Where:

    • CREDHUB-CLIENT-NAME is the value you recorded for BOSH_CLIENT in step 5 of this procedure.
    • CREDHUB-CLIENT-SECRET is the value you recorded for BOSH_CLIENT_SECRET in step 5 of this procedure.

    For example:

    $ credhub login \
    --client-name=credhub \
    --client-secret=abcdefghijklm123456789
    

Check Expiration Dates

You can check the expiration date for TLS and internal certificates. However, you can only rotate TLS certificates in MySQL for PCF v2.5. See Rotate TLS Certificates below.

You cannot rotate internal certificates. If your internal certificates are about to expire, you must upgrade to MySQL for PCF v2.6 to rotate them.

Warning: If your internal certificates expire, service bindings and backups can fail.

To check certificate expiration dates, do the following:

  1. Log in to CredHub by doing the procedure in Access BOSH CredHub above.

  2. Run the following command:

    credhub get -n CERTIFICATE-NAME -j | jq -r .value.ca | openssl x509 -text -noout | grep -A 2 "Validity"
    

    Where CERTIFICATE-NAME is the name of the certificate you are checking.

    For example:

    $ credhub get -n /services/tls_ca -j | jq -r .value.ca | openssl x509 -text -noout | grep -A 2 "Validity"
    Validity
              Not Before: Jan 23 19:04:58 2019 GMT
              Not After : Jan 23 19:04:58 2020 GMT
    

    The above command retrieves the certificate from CredHub, parses its JSON structure, decrypts it, and finds its Validity properties. The expiration date for the certificate is the value of Not After.

    For a list of MySQL for PCF certificates, see MySQL for PCF Certificates above.

Rotate TLS Certificates

To rotate CA certificates, you must do the following:

  1. Add a New CA Certificate
  2. Rebind Your App
  3. Remove the Old CA Certificate

Add a New CA Certificate

To add a new CA Certificate to Ops Manager, do the following:

Warning: This procedure involves restarting all of the VMs in your PCF deployment to apply a CA certificate. The operation can take a long time to complete.

  1. Log in to CredHub by doing the procedure in Access BOSH CredHub above.

  2. Set the CA certificate by doing one of the following:

    Note: Your PCF deployment can have multiple CA certificates. Pivotal recommends using a dedicated CA certificate for each service.

    • If you do not have a CA certificate: generate a new certificate by running the following command:

      credhub regenerate --name="/services/tls_ca"
      
    • If you have an existing certificate: create a new root.pem file with the contents of the certificate. Then, run the following command:

      credhub set \
          --name="/services/tls_ca" \
          --type="certificate" \
          --certificate=PEM-PATH/root.pem \
          --private=CERT-KEY...
      

      Where:

      • PEM-PATH is the path to the root.pem file.
      • CERT-KEY is the private key for the certificate.
  3. Extract and output the certificate portion from the CA certificate by running the following command:

    bosh int <(credhub get \
    --name=/services/tls_ca) \
    --path /value/certificate
    
  4. Record the output.

  5. Navigate to the Ops Manager Installation Dashboard and click the BOSH Director tile.

  6. Click Security.

  7. Append the contents of the new CA certificate to the old CA certificate under Trusted Certificates. Do not remove the old CA certificate.

  8. Click Save.

  9. Return to the Installation Dashboard in Ops Manager and click Apply Changes.

Rebind Your App

If your apps do not use the trusted store to validate server certificates, developers must re-bind apps to the service instance to receive the updated CA certificate. Apps not written in Java and Spring do not use the trusted store.

If you are rotating your CA certificate because it was compromised, you must remove your old CA certificate before rebinding your app. If you have not removed your old CA certificate, do the procedure in Remove the Old CA Certificate below.

To re-bind your app, do the following:

  1. Stop your app by running the following command:

    cf stop YOUR-APP
    

    Where YOUR-APP is the name of your app.

  2. Unbind your app from the service instance by running the following command:

    cf unbind-service YOUR-APP YOUR-SERVICE-INSTANCE
    

    Where:

    • YOUR-APP is the name of your app.
    • YOUR-SERVICE-INSTANCE is the name of your service instance.
  3. Re-bind your app to the service instance by running the following command:

    cf bind-service YOUR-APP YOUR-SERVICE-INSTANCE
    

    Where:

    • YOUR-APP is the name of your app.
    • YOUR-SERVICE-INSTANCE is the name of your service instance.
  4. Restage your app by running the following command:

    cf restage YOUR-APP
    

    Where YOUR-APP is the name of your app.

Remove the Old CA Certificate

Warning: This procedure involves restarting all of the VMs in your PCF deployment to apply a CA certificate. The operation can take a long time to complete.

After your apps have reconnected to service instances with certificates generated by the new CA, do the following to remove the old CA certificate:

  1. Navigate to the Installation Dashboard in Ops Manager and click the BOSH Director tile.

  2. Click Security.

  3. Delete the old CA certificate in Trusted Certificates.

  4. Click Save.

  5. Return to the Installation Dashboard in Ops Manager and click Apply Changes. This restarts all the VMs in your deployment.

  6. If you are rotating your CA certificate because it was compromised, you must rebind your apps to update the CA certificate. See Rebind Your App above.