Diagnosing Problems in Pivotal Platform
Page last updated:
This topic provides guidance for diagnosing issues encountered during a Pivotal Platform installation.
Besides whether products install successfully or not, an important consideration when diagnosing issues is communication between VMs deployed by Pivotal Platform. Depending on what products you install, communication takes the form of messaging, routing, or both. If they go wrong, an installation can fail. For example, in an Pivotal Application Service (PAS) installation the Pivotal Platform VM tries to push a test app to the cloud during post-installation testing. The installation fails if the resulting traffic cannot be routed to the HA Proxy load balancer.
The debug endpoint is a web page that provides information useful for diagnostics. If you have superuser privileges and can view the Ops Manager Installation Dashboard, you can access the debug endpoint.
In a browser, open the URL:
OPS-MANAGER-FQDN is the fully-qualified domain name (FQDN) of your Ops Manager installation.
The debug endpoint offers three links:
Files allows you to view the YAML files that Ops Manager uses to configure products that you install. The most important YAML file,
installation.yml, provides networking settings and describes
microbosh. In this case,
microboshis the VM whose BOSH Director component is used by Ops Manager to perform installations and updates of PAS and other products.
Components describes the components in detail.
Rails log shows errors thrown by the VM where the Ops Manager web app, such as a Rails app, is running, as recorded in the
production.logfile. To explore other logs, see Logging Tips.
This section contains general tips for locating where a particular problem is called out in the log files. For guidance regarding specific logs, such as those for PAS components, see the sections below.
- Start with the largest and most recently updated files in the job log.
- Identify logs that contain
errin the name.
- Scan the file contents for a “failed” or “error” string.
To troubleshoot specific PAS components by viewing their log files:
Navigate to your Ops Manager Installation Dashboard and click on the PAS tile.
Select the Status tab.
In the Job column, locate the component you want to troubleshoot.
In the Logs column for the component, click the download icon.
Select the Logs tab.
Once the ZIP file corresponding to the component moves to the Downloaded list, click the linked file path to download the ZIP file.
Once the download completes, unzip the file.
The contents of the log directory vary depending on which component you view. For example, the Diego Cell log directory contains subdirectories for the
garden processes. To view the standard error stream for
garden, download the Diego Cell logs and open
diego.0.job > garden > garden.stderr.log.
You can obtain diagnostic information from Ops Manager by logging in to the VM where it is running. To log in to the Ops Manager VM, you need:
The IP address of the Pivotal Platform VM shown in the Settings tab of the Ops Manager Director tile.
Your import credentials. Import credentials are the username and password used to import the Pivotal Platform
.ovffile into your virtualization system.
To log in to the Ops Manager VM:
Open a terminal window.
To connect to the Pivotal Platform installation VM, run:
IMPORT-USERNAMEis the username you used to import the
.ovffile into your virtualization system.
VM-IP-ADDRESSis the IP address of the Pivotal Platform installation VM.
Enter your import password when prompted.
Go to the home directory of the web app by running:
You are now in a position to explore whether things are as they should be within the web app. You can also verify that the
microboshcomponent is successfully installed. A successful MicroBOSH installation is required to install PAS and any products like databases and messaging services.
Change directories to the BOSH installation log home by running:
You may want to begin by running a tail command on the
If you are unable to resolve an issue by viewing configurations, exploring logs, or reviewing common problems, you can troubleshoot further by running BOSH diagnostic commands with the BOSH Command Line Interface (CLI).
Note: Do not manually modify the deployment manifest. Ops Manager overwrites manual changes to this manifest. In addition, manually changing the manifest may cause future deployments to fail.
To view the VMs in your Pivotal Platform deployment, follow the procedure specific to your IaaS.
To view the VMs in your AWS deployment:
Log in to the AWS Console.
Navigate to the EC2 Dashboard.
Click Running Instances.
Click the gear icon in the upper-right corner.
Select job, deployment, director, and index.
To view the VMs in your OpenStack deployment:
Install the novaclient from the python-novaclient repository on GitHub.
Point novaclient to your OpenStack installation and tenant by exporting the following environment variables:
export OS\_AUTH\_URL= YOUR\_KEYSTONE\_AUTH\_ENDPOINT export OS\_TENANT\_NAME = TENANT\_NAME export OS\_USERNAME = USERNAME export OS\_PASSWORD = PASSWORD
List your VMs by running:
nova list --fields metadata
To view the VMs in your vSphere deployment:
Log in to vCenter.
Select Hosts and Clusters.
Select the top level object that contains your Pivotal Platform deployment. For example, select Cluster, Datastore, or Resource Pool.
In the top tab, click Related Objects.
Select Virtual Machines.
Right-click on the Table heading and select Show/Hide Columns.
Select the job, deployment, director, and index boxes.
Apps Manager provides a graphical user interface to help manage organizations, users, apps, and spaces. For more information about Apps Manager, see Getting Started with Apps Manager
When troubleshooting Apps Manager performance, you might want to view the Apps Manager app logs. To view the Apps Manager app logs:
From a command line, log in to Pivotal Platform by running:
cf login -a api.YOUR-SYSTEM-DOMAIN -u admin
YOUR-SYSTEM-DOMAINis your system domain.
When prompted, enter your UAA Administrator credentials. To obtain these credentials, see Credentials tab in the PAS tile.
systemorg and the
apps-managerspace by running:
cf target -o system -s apps-manager
Tail the Apps Manager logs by running:
cf logs apps-manager
Apps Manager recognizes the
LOG_LEVEL environment variable. The
LOG_LEVEL environment variable allows you to filter the messages reported in Apps Manager log files by severity level. Apps Manager defines severity levels using the Ruby standard library Logger class.
By default, the Apps Manager
LOG_LEVEL environment variable is set to
info. The logs show more verbose messaging when you set the
To change the Apps Manager
LOG_LEVEL environment variable, run:
cf set-env apps-manager LOG_LEVEL LEVEL
LEVEL is the desired severity level.
You can set
LOG_LEVEL to one of the six severity levels defined by the Ruby Logger class:
- Level 5:
unknown– An unknown message that should always be logged.
- Level 4:
fatal– An unhandleable error that results in a program crash.
- Level 3:
error– A handleable error condition.
- Level 2:
warn– A warning.
- Level 1:
info– General information about system operation.
- Level 0:
debug– Low-level information for developers.
Once set, Apps Manager log files only include messages at the set severity level and above. For example, if you set
fatal, the log includes
unknown level messages only.
To obtain disk usage statistics by Diego Cell VMs and containers, see Examining GrootFS Disk Usage.