Troubleshooting Spring Cloud® Services for Pivotal Cloud Foundry
See below for information about problems related to your Pivotal Cloud Foundry (PCF) platform configuration or Spring Cloud Services or other product installation.
p-circuit-breaker-dashboard service in Spring Cloud Services requires that RabbitMQ for PCF is configured to deploy the Pre-provisioned broker. If the
run_tests errand or create and update service fails with a message similar to following:
Showing status of last operation from service test-p-circuit-breaker-dashboard-*********... status: create failed message: Error creating service instance: java.lang.IllegalArgumentException: Service plan standard not found
Enable the Pre-provisioned broker as described in the RabbitMQ for PCF documentation.
If the broker fails to provision a service instance and the logs for the
spring-cloud-broker-worker app (see Read Broker Application Logs) contain the following:
java.util.concurrent.TimeoutException: Execution took longer than 180000 MILLISECONDS
This is due to a problem encountered during app deployments to your PCF foundation. Verify that you can deploy apps without issues to your PCF foundation (see “Starting or staging an application results in an InsufficientResources error” in the PCF Knowledge Base).
If you encounter a message similar to the following when attempting to access a Service Registry dashboard:
with the following text:
Authentication Failed: Could not obtain access token
If your PCF foundation is using a self-signed certificate, ensure that it includes all of the domains listed in Requirements. If you update the certificate, you will need to either restart the service instance’s backing app(s) or recreate the service instance. Otherwise you may continue to see this message and be unable to access the Service Registry dashboard.
If you encounter an exception message similar to the following:
org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://p-spring-cloud-services.uaa.example.com/oauth/token": java.security.cert.CertificateException: No subject alternative DNS name matching p-spring-cloud-services.uaa.example.com found.; nested exception is javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative DNS name matching p-spring-cloud-services.uaa.example.com found.
Ensure that your Pivotal Application Service (PAS) or Elastic Runtime SSL certificate is configured as described in Requirements. Be sure that the certificate includes all wildcards listed in that section and has a separate wildcard for each subdomain.
The Cloud Foundry Command Line Interface tool (cf CLI)
create-service command may return an error similar to the following:
Server error, status code: 502, error code: 10001, message: Service broker error: javax.net.ssl.SSLException: Received fatal alert: protocol_version
Spring Cloud Services requires the RabbitMQ for PCF product. If you have provided SSL certificates and keys to RabbitMQ for PCF, you must also enable its TLS 1.0 support in order to use it with Spring Cloud Services. See Requirements.
“com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN”
If you encounter the following exception message in Spring Cloud Services broker logs:
org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
The credentials used by the Spring Cloud Services broker to access its RabbitMQ for PCF service instance may be invalid. This can happen after restoring a backup of Pivotal Application Service (or Elastic Runtime), as a restored RabbitMQ for PCF service instance may not accept the restored credentials used by the Spring Cloud Services broker. You can update the Spring Cloud Services broker’s credentials by unbinding the RabbitMQ service instance from the broker apps (the
spring-cloud-broker-worker apps) and then rebinding.
To unbind the RabbitMQ service instance from the Spring Cloud Services broker apps and rebind it, run the following cf CLI commands:
$ cf target -o system -s p-spring-cloud-services $ cf unbind-service spring-cloud-broker spring-cloud-broker-rmq $ cf unbind-service spring-cloud-broker-worker spring-cloud-broker-rmq $ cf bind-service spring-cloud-broker spring-cloud-broker-rmq $ cf restage spring-cloud-broker $ cf bind-service spring-cloud-broker-worker spring-cloud-broker-rmq $ cf restage spring-cloud-broker-worker
If the PCF foundation is newly restored and the RabbitMQ service instance’s queues are empty, you may also unbind the RabbitMQ service instance from the broker apps, delete it, create a new instance, and then bind that instance to the Spring Cloud Services broker apps.
WARNING: Deleting and replacing the RabbitMQ service instance will destroy any data in the instance’s queues. If the RabbitMQ service instance contains messages and is on a production system, unbind and rebind it instead.
To delete and replace the RabbitMQ service instance used by the Spring Cloud Services broker apps, run the following cf CLI commands:
$ cf target -o system -s p-spring-cloud-services $ cf unbind-service spring-cloud-broker spring-cloud-broker-rmq $ cf unbind-service spring-cloud-broker-worker spring-cloud-broker-rmq $ cf delete-service -f spring-cloud-broker-rmq $ cf create-service p-rabbitmq standard spring-cloud-broker-rmq $ cf bind-service spring-cloud-broker spring-cloud-broker-rmq $ cf restage spring-cloud-broker $ cf bind-service spring-cloud-broker-worker spring-cloud-broker-rmq $ cf restage spring-cloud-broker-worker