Troubleshooting Windows Cells
This topic describes how to troubleshoot Windows cells deployed by Pivotal Cloud Foundry (PCF) Runtime for Windows.
Perform the following steps to retrieve the logs for the Windows cell:
- Navigate to the Ops Manager Installation Dashboard.
- Click the PCF Runtime for Windows tile.
- Click the Status tab.
- Under the Logs column, click the download icon for the Windows cell you want to retrieve logs from.
- Click the Logs tab.
- When the logs are ready, click the filename to download them.
- Unzip the file to examine the contents. Each component on the cell has its own logs directory:
Hakim is a diagnostic tool that reveals common configuration issues with Windows cells. Because PCF Runtime for Windows runs Hakim as a BOSH job, the logs may include Hakim error messages. Refer to the section below for a list of Hakim error messages and their possible solutions.
The following processes are not running
This usually indicates a failed deployment. Try redeploying the PCF Runtime for Windows tile.
There was an error detecting ntp synchronization on your machine.
An accurate system clock is essential for internal Cloud Foundry metric reports.
Please configure your NTP settings, if not already done.
We recommend that your firewall have outbound rules set for UDP on port 123.
In addition, ensure that your 'DnsCache' service is running
If NTP is not configured, clock skew with other PCF components can occur. Clock skew can result in odd errors, such as not receiving any metrics from apps running on the affected machine. Ensure that you are using the same NTP server on your Windows cell as the rest of your PCF deployment.
Windows firewall service is not enabled. The Windows firewall is required in order to enforce Application Security Group rules. Running without the firewall is possible, but strongly not recommended.
Garden Windows enforces PCF security group settings for apps running on the Windows cell through the Windows firewall. Apps can run without this, but security groups do not work correctly and apps have unrestricted network access.
To resolve this error, enable the Windows firewall. Perform the following steps in your RDP session to access the Windows firewall configuration:
- Open the Server Manager from the task bar.
- Click Tools in the upper right and select Windows Firewall with Advanced Security.
- Configure and enable the Windows firewall.
Fair Share CPU Scheduling must be disabled
You must disable Fair Share CPU scheduling for your Windows cell to function properly. Perform the following steps in your RDP session:
- Open the Registry Editor at
- Navigate to
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Quota System\.
- Double-click the
- Change the Value data from
- Click OK.
Failed to create container
This usually indicates an issue with the Windows containerization service. Contact Pivotal Support and provide the full output of this error.
Failed to resolve consul host
This usually indicates interference with DNS resolution on your Windows cell. To resolve this error, perform the following steps in your RDP session to set
127.0.0.1 as the primary DNS server for the active network adapter:
- Open the Control Panel.
- Click Network and Internet
- Click Network and Sharing Center.
- Click Change adapter settings on the left.
- Double-click your active network adapter.
- Click Properties.
- Select Internet Protocol Version 4 (TCP/IPv4).
- Click Properties.
- Ensure that Use the following DNS server addresses is selected and enter
127.0.0.1for Preferred DNS server.
- Click OK.
Perform the following steps to connect to your Windows cell to run diagnostics:
Download and install a Remote Desktop Protocol (RDP) client.
Follow the steps in the Log into BOSH section of the Advanced Troubleshooting with the BOSH CLI topic to target and log in to your BOSH Director. The steps vary slightly depending on whether your PCF deployment uses internal authentication or an external user store.
Retrieve the IP address of your Windows cell using one of the following methods.
If you use BOSH CLI v1, run the following command:
$ bosh vms garden-windows Acting as user 'director' on deployment 'garden-windows' on 'p-bosh-1170e9b438cb29ff7c63' Director task 274 Task 274 done +-------------------------------------------------------+---------+---------+---------+--------------+ | VM | State | AZ | VM Type | IPs | +-------------------------------------------------------+---------+---------+---------+--------------+ | cell_windows/0 (03e221b3-3222-5e1e-eedd-b92221ff88e1) | running | default | xlarge | 198.51.100.1 | +-------------------------------------------------------+---------+---------+---------+--------------+
VMs total: 1
If you use BOSH CLI v2, run the following command, replacing
MY-ENVwith the alias you assigned to your BOSH Director:
$ bosh -e MY-ENV -d garden-windows Using environment 'DIRECTOR-IP' as client 'admin'
Name Release(s) Stemcell(s) Team(s) Cloud Config garden-windows ... ... - latest
Retrieve the Administrator password for your Windows cell by following the steps for your IaaS:
- On vSphere, this is the value of
consumer-vars.ymlfile you used to previously build a stemcell.
- On Amazon Web Services (AWS), navigate to the AWS EC2 console. Right-click on your Windows cell and select Get Windows Password from the drop-down menu. Provide the local path to the
ops_mgr.pemprivate key file you used when installing Ops Manager and click Decrypt password to obtain the Administrator password for your Windows cell.
- On Google Cloud Platform (GCP), navigate to the Compute Engine Dashboard. Under VM Instances, select the instance of the Windows VM. At the top of the page, click on Create or reset Windows password. When prompted, enter “Administrator” under Username and click Set. You will receive a one-time password for the Windows cell.
- You cannot RDP into Windows cells on Azure.
- On vSphere, this is the value of
Open your RDP client. The examples below use the Microsoft Remote Desktop app.
Click New and enter your connection information:
- Connection name: Enter a name for this connection.
- PC name: Enter the IP address of your Windows cell.
- User name: Enter
- Password: Enter the password of your Windows cell that you obtained above.
To mount a directory on your local machine as a drive in the Windows cell, perform the following steps:
- From the same Edit Remote Desktops window as above, click Redirection.
- Click the plus icon at the bottom left.
- For Name, enter the name of the drive as it will appear in the Windows cell. For Path, enter the path of the local directory.
- Click OK.
Close the Edit Remote Desktops window and double-click the newly added connection under My Desktops to open a RDP connection to the Windows cell.
In the RDP session, you can use the Consul CLI to diagnose problems with your Windows cell.
Perform the following steps to use the Consul CLI on your Windows cell to diagnose problems with your Consul cluster:
- In your RDP session, open a PowerShell window.
- Change into the directory that contains the Consul CLI binary:
PS C:\Users\Administrator> cd C:\var\vcap\packages\consul-windows\bin\
- Use the Consul CLI to list the members of your Consul cluster:
PS C:\Users\Administrator\var\vcap\packages\consul-windows\bin> .\consul.exe members Node Address Status Type Build Protocol DC cell-windows-0 10.0.0.111:8301 alive client 0.6.4 2 dc1 cloud-controller-0 10.0.0.94:8301 alive client 0.6.4 2 dc1 cloud-controller-worker-0 10.0.0.99:8301 alive client 0.6.4 2 dc1 consul-server-0 10.0.0.96:8301 alive server 0.6.4 2 dc1 diego-brain-0 10.0.0.109:8301 alive client 0.6.4 2 dc1 diego-cell-0 10.0.0.103:8301 alive client 0.6.4 2 dc1 diego-cell-1 10.0.0.104:8301 alive client 0.6.4 2 dc1 diego-cell-2 10.0.0.107:8301 alive client 0.6.4 2 dc1 diego-database-0 10.0.0.92:8301 alive client 0.6.4 2 dc1 ha-proxy-0 10.0.0.254:8301 alive client 0.6.4 2 dc1 nfs-server-0 10.0.0.100:8301 alive client 0.6.4 2 dc1 router-0 10.0.0.105:8301 alive client 0.6.4 2 dc1 uaa-0 10.0.0.93:8301 alive client 0.6.4 2 dc1
- Examine the output to ensure that the
cell-windows-0service is registered in the Consul cluster and is
alive. Otherwise, your Windows cell cannot communicate with your PCF deployment and developers cannot push .NET apps to the Windows cell. Check the configuration of your Consul cluster, and ensure that your certificates are not missing or misconfigured.