LATEST VERSION: v1.0 - CHANGELOG
Pivotal Container Service v1.0

Troubleshooting

Page last updated:

PKS API is Slow or Times Out

Symptom

When you run PKS CLI commands, the PKS API times out or is slow to respond.

Explanation

The PKS API control plane VM requires more resources.

Solution

  1. Navigate to https://YOUR-OPS-MANAGER-FQDN/ in a browser to log in to the Ops Manager Installation Dashboard.

  2. Select the Pivotal Container Service tile.

  3. Select the Resource Config page.

  4. For the Pivotal Container Service job, select a VM Type with greater CPU and memory resources.

  5. Click Save.

  6. Click the Installation Dashboard link to return to the Installation Dashboard.

  7. Click Apply Changes.


Cluster Creation Fails

Symptom

When creating a cluster, you run pks cluster CLUSTER-NAME to monitor the cluster creation status. In the command output, the value for Last Action State is error.

Explanation

There was an error creating the cluster.

Diagnostics

  1. Log in to the BOSH Director and run bosh tasks. The output from bosh tasks provides details about the tasks that the BOSH Director has run. See Manage PKS Deployments with BOSH for more information about logging in to the BOSH Director.

  2. In the BOSH command output, locate the task that attempted to create the cluster.

  3. Find more information about the task by running bosh -e MY-ENVIRONMENT tasks TASK-NUMBER. For example:

    $ bosh -e pks tasks 23

For more information about troubleshooting failed BOSH tasks, see Tasks.


Cannot Access Add-On Features or Functions

Symptom

You cannot access a feature or function provided by a Kubernetes add-on.

Examples include the following:

  • You cannot access the Kubernetes Web UI (Dashboard) in a browser or using the kubectl command-line tool.
  • Heapster does not start.
  • Pods cannot resolve DNS names, and error messages report the service kube-dns is invalid. If kube-dns is not deployed, the cluster typically fails to start.

Explanation

The Kubernetes features and functions listed above are provided by the following PKS add-ons:

  • Kubernetes Dashboard kubernetes-dashboard
  • Heapster: heapster
  • DNS Resolution: kube-dns

To enable these add-ons, Ops Manager must run scripts after deploying PKS. You must configure Ops Manager to automatically run these post-deploy scripts.

Solution

Perform the following steps to configure Ops Manager to run post-deploy scripts to deploy the missing add-ons to your cluster.

  1. Navigate to https://YOUR-OPS-MANAGER-FQDN/ in a browser to log in to the Ops Manager Installation Dashboard.

  2. Click the Ops Manager v2.0 tile.

  3. Select Director Config.

  4. Select Enable Post Deploy Scripts.

    Note: This setting enables post-deploy scripts for all tiles in your Ops Manager installation.

  5. Click Save.

  6. Click the Installation Dashboard link to return to the Installation Dashboard.

  7. Click Apply Changes.

  8. After Ops Manager finishes applying changes, enter pks delete-cluster on the command line to delete the cluster. For more information, see Delete a Cluster in Using PKS.

  9. On the command line, enter pks create-cluster to recreate the cluster. For more information, see Create a Cluster in Using PKS.


Error: Failed Jobs

Symptom

In stdout or log files, you see an error message referencing post-start scripts failed or Failed Jobs.

Explanation

After deploying PKS, Ops Manager runs scripts to start a number of jobs. You must configure Ops Manager to automatically run these post-deploy scripts.

Solution

Perform the following steps to configure Ops Manager to run post-deploy scripts.

  1. Navigate to https://YOUR-OPS-MANAGER-FQDN/ in a browser to log in to the Ops Manager Installation Dashboard.

  2. Click the Ops Manager v2.0 tile.

  3. Select Director Config.

  4. Select Enable Post Deploy Scripts.

    Note: This setting enables post-deploy scripts for all tiles in your Ops Manager installation.

  5. Click Save.

  6. Click the Installation Dashboard link to return to the Installation Dashboard.

  7. Click Apply Changes.

  8. After Ops Manager finishes applying changes, enter pks delete-cluster on the command line to delete the cluster. For more information, see Delete a Cluster.

  9. On the command line, enter pks create-cluster to recreate the cluster. For more information, see Create a Cluster.


Error: No Such Host

Symptom

In stdout or log files, you see an error message that includes lookup vm-WORKER-NODE-GUID on IP-ADDRESS: no such host.

Explanation

This error occurs on GCP when the Ops Manager Director tile uses 8.8.8.8 as the DNS server. When this IP range is in use, the master node cannot locate the route to the worker nodes.

Solution

Use the Google internal DNS range, 169.254.169.254, as the DNS server.


Error: FailedMount

Symptom

In Kubernetes log files, you see a Warning event from kubelet with FailedMount as the reason.

Explanation

A persistent volume fails to connect to the Kubernetes cluster worker VM.

Diagnostics

  • In your cloud provider console, verify that volumes are being created and attached to nodes.
  • From the Kubernetes cluster master node, check the controller manager logs for errors attaching persistent volumes.
  • From the Kubernetes cluster worker node, check kubelet for errors attaching persistent volumes.

Please send any feedback you have to pks-feedback@pivotal.io.

Create a pull request or raise an issue on the source for this page in GitHub