Troubleshooting ECS Service Broker for Pivotal Platform

This document describes troubleshooting steps and common issues when running the ECS Service Broker for Pivotal Platform.

Collecting Logs

The ECS Service Broker is installed through a tile in Operations Manager, so when an installation fails, collecting logs from Ops Manager is helpful. To do this, navigate to the Ops Manager Installation Dashboard and select Change Log. Find the entry for the ECS Pivotal Platform Service Broker and click Logs. The logs will be displayed for the following BOSH errands:

  • Installing Dell EMC ECS Broker
  • Running errand deploy-service-broker for DELL EMC ECS Broker
  • Running errand register-broker for DELL EMC ECS Broker

If you encounter errors during run time, or more detailed logs are required, you can also collect logs from the deployed application. To find these logs, navigate to the Pivotal Platform Apps Manager tile. Find the organization and space for the ECS Broker, which is typically ecs-broker-service-org and ecs-broker-service-space. From there, you can view detailed application logs.

General Troubleshooting Steps

For the most part, issues with the broker are a result of incorrect configuration. Here are some general things to check, which commonly resolve issues:

  1. Make sure you have HTTP connectivity to both the ECS management API, which typically runs on port 4443, and ECS S3 service, which will run on port 9020 (HTTP) and port 9021 (HTTPS). If you are accessing ECS through a load-balancer, these port numbers may be different. To test access to these ports, use the CURL command from a Linux or Mac OS host:
    $ curl -v -k https://:4443/login

$ curl -v -k http://:9020/

$ curl -v -k https://:9021/ You might see an “Unauthorized” error. This is not a problem, since connectivity has been established.

  1. Make sure you have Management User credentials configured for the ECS Admin Username. ECS has support for two types of users: object users and management. The broker will require a management user to be configured. Test authentication
    $ curl -k -u : https:///login

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>USERNAME

  1. Make sure that the ECS certificate is configured properly. A production ECS installation will typically have a signed certificate installed, which does not require any special confguration. If this is the case, ensure that the tile prompt, “Is the ECS SSL certificate is signed by a trust authority?” is marked as true. If using a self-signed certificate, you must import that cert into the tile configuration. You can read the certificate from the host using the openssl command from a Linux or Mac OS:
    $ openssl s_client -showcerts -connect .com:4443 < /dev/null | \
    openssl x509 -text | sed -n -e '/BEGIN\ CERTIFICATE/,/END\ CERTIFICATE/ p'
    

Common Issues

Below is a list of error messages and their resolutions.

Access is denied

You may see an error matching the following in the broker stack trace:

Error: Unauthorized (401): Access is denied due to invalid or expired credentials

This is likely an error with the management username or password.

Unable to render template for job ‘deploy-service-broker’

You may see an error in the BOSH logs that match the following:

Error filling in template 'application.yml.erb' (line 38: undefined method `plan_guid' for nil:NilClass)

If this error appears, check the Catalog Service pages of the ECS Broker tile, and ensure that at least one service is selected for repository service and one of that service’s plans is selected for repository plan.

These enable the broker to create an internal bucket to use for its own metadata.