Monitoring and Troubleshooting
This topic broadly outlines techniques for troubleshooting your Concourse for PCF installation.
To get help on all
fly commands, run:
The following selected
fly commands give you useful information to help you troubleshoot Concourse for PCF environments and pipelines.
You can use the following
fly commands to troubleshoot possible environment problems.
|Lists active containers. This confirms which container or task is placed on which worker.|
|Lists registered workers. This helps you verify that the number of containers does not exceed the maximum number allowable.|
|Removes a non-running worker. Stops Concourse for PCF from tracking an out-of-commission worker.|
|Lists active volumes. Checks disk usage across workers.|
You can use the following
fly commands to troubleshoot possible pipeline problems.
|Lists configured pipelines.|
|Shows build history.
This is useful for listing build IDs of one-off tasks ran previously using |
|Validates a pipeline’s configuration without calling |
|Checks for new versions. This is useful when developing a new resource.|
|Shows logs of in-progress builds.|
|Displays build steps for a running or recent build.|
|Submits local tasks. This is useful for spinning up a task quickly to test it before putting it in a job.|
The following shows some common problems and solutions.
|The worker is out of disk space||An error states there is an inability to create a storage volume and may state that permissions are denied.||Increase persistent disk size for the worker or increase the number of worker VMs.|
|Container limit is reached||Cannot create container: limit of 250 containers reached
This error state is unlikely to appear.
|Job does not start||This error may present as the build getting stuck in Pending state.||Restart the ATC job:
|Build fails when updating Concourse for PCF from a job on the same instance||When a build fails after BOSH deploys a Concourse for PCF update from a job running on that Concourse for PCF instance, typically the job fails with a “worker for container not found” error. This is expected behavior; the BOSH Director will recreate the worker VM.||Run the job again.|
|BOSH cannot finish worker upgrade while tasks are running||BOSH is not able to restart the worker job to finalize the upgrade until all work is completed.||If you have a long-running task, wait for the task to be completed.
If you need to upgrade quickly, cancel running tasks and jobs.
You might need to contact Pivotal Support for help identifying a problem. In that case, support might ask you to send job log files.
The following links provide other troubleshooting resources.
|Enabling syslog forwarding and getting logs for other Concourse components||VM Logs|
|Common BOSH issues||BOSH tips|