Troubleshooting On-Demand Instances
Note: Pivotal Platform is now part of VMware Tanzu. In v2.8 and later, MySQL for Pivotal Platform is named MySQL for VMware Tanzu.
Page last updated:
This topic provides techniques that app developers can use to begin troubleshooting on-demand instances.
Troubleshoot Errors
Start here if you are responding to a specific error or error messages.
Common Service Errors
No Metrics from Log Cache |
|
---|---|
Symptom | You receive no metrics when running the cf tail command. |
Cause | This might be because the Firehose is disabled in the TAS for VMs tile. |
Solution | Ask your operator to ensure that the V2 Firehose checkbox is enabled, and the Enable Log Cache syslog ingestion checkbox is disabled in the TAS for VMs tile. For more information about configuring these checkboxes, see Enable Syslog Forwarding. |
If Instances or Database are Inaccessible
You might experience the following in a leader-follower or Multi‑Site Replication topology, or during upgrades:
- Temporary Outages
- Apps Cannot Write to the Database
- Apps Are Inoperable
- Apps Cannot Connect to the Database
- MySQL Connector/J v5.1.41 or Earlier
- Mutual TLS
Temporary Outages |
|
---|---|
Symptom | MySQL for VMware Tanzu service instances can become temporarily inaccessible during upgrades and VM or network failures. |
Solution | For more information, see Service Interruptions. |
Apps Cannot Write to the Database |
|
---|---|
Symptom | You have a leader-follower or Multi‑Site Replication topology, and your apps can no longer write to the database. |
Cause | If you have a leader-follower topology, the leader VM might be read-only. If you can no longer read to the database as well, your persistent disk might be full. If you have a Multi‑Site Replication topology, your leader VM might be down. |
Solution | If you have a leader-follower topology and the leader VM is read-only, for how to troubleshoot this problem, see Both Leader and Follower Instances Are Read-Only. If your apps can no longer read to the database as well, your persistent disk might be full. For more information about troubleshooting inoperable apps, see Apps are Inoperable. If you have a Multi‑Site Replication topology and your leader VM is down, to resolve this issue, you can trigger a failover to the follower VM. For more information about troubleshooting this problem, see Triggering Multi-Site Replication Failover and Switchover. |
Apps Are Inoperable |
|
---|---|
Symptom | Your apps become inoperable. Read, write, and cf CLI operations do not work. |
Cause | Your persistent disk might be full |
Solution | Contact your operator to check if your persistent disk is full. For more information about troubleshooting this problem, see Persistent Disk is Full. |
Apps Cannot Connect to the Database |
|
---|---|
Symptom | Apps can fail to connect to the database. |
Cause |
|
Solution |
|
MySQL Connector/J v5.1.41 or Earlier |
|
---|---|
Symptom | Apps cannot connect to the database when TLS is enabled and the apps are using MySQL Connector/J v5.1.41 or earlier. |
Cause | You see errors about certificates. For example: Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[na:1.8.0_152] |
Solution | If you cannot update the MySQL Connector/J, do the workaround in How to disable KeyManager and TrustManager in Container Security Provider Framework in the Javanbuildpack in the Support knowledge base. |
Mutual TLS |
|
---|---|
Symptom | Apps cannot connect to the database when TLS is enabled and your apps uses mTLS. |
Cause | You see network errors in your app logs. |
Solution | To resolve this issue disable mTLS in your apps. |
Techniques for Troubleshooting
See the following sections for troubleshooting techniques when using the Cloud Foundry Command-Line Interface (cf CLI) to perform basic operations on a MySQL for VMware Tanzu service instance.
Basic cf CLI operations include create
, update
, bind
, unbind
, and delete
.
Understand a Cloud Foundry (CF) Error Message
Failed operations (create, update, bind, unbind, delete) result in an error message.
You can retrieve the error message later by running the cf CLI command cf service INSTANCE-NAME
.
$ cf service myservice Service instance: myservice Service: super-db Bound apps: Tags: Plan: dedicated-vm Description: Dedicated Instance Documentation url: Dashboard: Last Operation Status: create failed Message: Instance provisioning failed: There was a problem completing your request. Please contact your operations team providing the following information: service: redis-acceptance, service-instance-guid: ae9e232c-0bd5-4684-af27-1b08b0c70089, broker-request-id: 63da3a35-24aa-4183-aec6-db8294506bac, task-id: 442, operation: create Started: 2017-03-13T10:16:55Z Updated: 2017-03-13T10:17:58Z
Use the information in the Message
field to debug further.
Provide this information to Support when filing a ticket.
The task-id
field maps to the BOSH task ID.
For more information on a failed BOSH task, use the bosh task TASK-ID
.
The broker-request-guid
maps to the portion of the On-Demand Broker log
containing the failed step.
Access the broker log through your syslog aggregator, or access BOSH logs for
the broker by typing bosh logs broker 0
.
If you have more than one broker instance, repeat this process for each instance.
Find Information about Your Service Instance
You might need to find the name, GUID, or other information about a service instance. To find this information, do the following:
Log into the space containing the instance or failed instance.
$ cf login
If you do not know the name of the service instance, run
cf services
to see a listing of all service instances in the space. The service instances are listed in thename
column.$ cf services Getting services in org my-org / space my-space as user@example.com... OK name service plan bound apps last operation my-instance p.mysql db-small create succeeded
To retrieve more information about a specific instance, run
cf service SERVICE-INSTANCE-NAME
To retrieve the GUID of the instance, run
cf service SERVICE-INSTANCE-NAME --guid
The GUID is useful for debugging.
Use the Knowledge Base (Community)
Find the answer to your question and browse product discussions and solutions by searching the VMware Tanzu Knowledge Base.
File a Support Ticket
You can file a support ticket here.
Be sure to provide the error message from cf service YOUR-SERVICE-INSTANCE
.
To expedite troubleshooting, if possible, provide your service broker logs, service instance logs, and BOSH task output. Your cloud operator should be able to obtain these from your error message.