Shutting Down and Restarting Enterprise PKS
-
Shutdown Sequence and Tasks
- Step 1: Disable BOSH Resurrection
- Step 2: Shut Down Customer Apps
- Step 3: Shut Down Kubernetes Clusters
- Step 4: Shut Down Enterprise PKS Control Plane
- Step 5: Shut Down VMware Harbor Registry
- Step 6: Shut Down BOSH Director
- Step 7: Shut Down Ops Manager
- Step 8: Shut Down NSX-T Components
- Step 9: Shut Down vCenter Server
- Step 10: Shut Down ESXi Hosts
-
Startup Sequence and Tasks
- Step 1: Start ESXi Hosts
- Step 2: Start vCenter
- Step 3: Start NSX-T Components
- Step 4: Start Ops Manager
- Step 5: Start the BOSH Director
- Step 6: Start the Enterprise PKS Control Plane
- Step 7: Start Harbor Registry
- Step 8: Start the Kubernetes Clusters
- Step 9: Start Customer Apps
- Step 10: Re-enable BOSH Resurrection
Page last updated:
Warning: VMware Enterprise PKS v1.6 is no longer supported because it has reached the End of General Support (EOGS) phase as defined by the Support Lifecycle Policy. To stay up to date with the latest software and security updates, upgrade to a supported version.
This topic lists and describes the shutdown and startup sequence for VMware Enterprise PKS including Enterprise PKS-provisioned Kubernetes cluster nodes, PKS control plane components, and vSphere hosts.
Shutdown Sequence and Tasks
To perform a graceful shutdown of all Kubernetes, Enterprise PKS, and infrastructure components, complete the following tasks in sequence.
Step 1: Disable BOSH Resurrection
If you have the Enable VM Resurrector Plugin checkbox selected in the BOSH Director tile > Director Config pane, you must turn BOSH resurrection off before restarting PKS, to prevent BOSH from recreating VMs.
To do this, run the command bosh update-resurrection off
.
Step 2: Shut Down Customer Apps
Shut down all customer apps running on Enterprise PKS-provisioned Kubernetes clusters.
Note: This task is optional. Perform it after considering the types of apps you have deployed. For example, stateful, stateless, or legacy apps.
Step 3: Shut Down Kubernetes Clusters
Shut down all Enterprise PKS-provisioned Kubernetes clusters following the procedure defined in the How to shutdown and startup a Multi Master PKS cluster knowledge base article.
For each Kubernetes cluster that you intend to shut down, do the following:
Using the BOSH CLI, retrieve the BOSH deployment name of your Enterprise PKS clusters by running the following command:
bosh deployments
Kubernetes cluster deployment names begin with
service-instance_
and include a unique BOSH-generated hash.Using the BOSH CLI, stop the Kubernetes worker nodes by running the following command:
bosh -d service-instance_CLUSTER-UUID stop worker
Where
CLUSTER-UUID
is the BOSH deployment name of your Enterprise PKS cluster. For example:$ bosh -d service-instance_aa1234567bc8de9f0a1c stop worker
Note: When you use the BOSH
stop
command, all processes on the Kubernetes node are stopped. BOSH marks them stopped so that when the VM is powered back on, the processes do not start automatically.Using the BOSH CLI, stop the Kubernetes master nodes by running the following command:
bosh -d service-instance_CLUSTER-UUID stop master
Where
CLUSTER-UUID
is the BOSH deployment name of your Enterprise PKS cluster. For example:$ bosh -d service-instance_aa1234567bc8de9f0a1c stop master
Using vCenter, shut down all Kubernetes node VMs. To do this, perform the following steps:
- Verify the node type by checking the “job” name in the Custom Attributes pane.
Perform a graceful shutdown by right-clicking the target VM and selecting Power > Shut Down Guest OS.
Step 4: Shut Down Enterprise PKS Control Plane
To shut down the PKS control plane VM, do the following:
Using the BOSH CLI, retrieve the BOSH deployment ID of your Enterprise PKS deployment by running the following command:
bosh deployments
Enterprise PKS deployment names begin with
pivotal-container-service
and include a unique BOSH-generated hash.Using the BOSH CLI, stop the PKS control plane VM by running the following command:
bosh -d pivotal-container-service-DEPLOYMENT-ID stop
Where
DEPLOYMENT-ID
is the BOSH-generated ID of your Enterprise PKS deployment. For example:$ bosh -d pivotal-container-service-1bf7b02738056cdc37e6 stop
Using vCenter, locate and gracefully shut down the PKS control plane VM.
Step 5: Shut Down VMware Harbor Registry
To shut down the Harbor Registry VM, do the following:
Using the BOSH CLI, retrieve the BOSH deployment ID of your Harbor Registry deployment by running the following command:
bosh deployments
Harbor Registry deployment names begin with
harbor-container-registry
and include a unique BOSH-generated hash.Using the BOSH CLI, stop the Harbor Registry VM by running the following command:
bosh -d harbor-container-registry-DEPLOYMENT-ID stop
Where
DEPLOYMENT-ID
is the BOSH-generated ID of your Harbor Registry deployment. For example:$ bosh -d harbor-container-registry-b4023f6857207b237399 stop
Using vCenter, locate and gracefully shut down the Harbor Registry VM.
Step 6: Shut Down BOSH Director
Using vCenter, locate and gracefully shut down the BOSH Director VM.
View a larger version of this image.
Step 7: Shut Down Ops Manager
Using vCenter, locate and gracefully shut down the Ops Manager VM.
View a larger version of this image.
Step 8: Shut Down NSX-T Components
Using vCenter, gracefully shut down all NSX-T VMs in the following order:
- NSX-T Manager
- NSX-T Controllers
- NSX-T Edge Nodes
View a larger version of this image.
Step 9: Shut Down vCenter Server
To shut down the vCenter Server VM, do the following:
- Navigate to the vCenter Appliance Management Interface at
https://YOUR-VCENTER-HOSTNAME-OR-IP-ADDRESS>:5480
, whereYOUR-VCENTER-HOSTNAME-OR-IP-ADDRESS
is the hostname or IP address that you use to connect to vCenter through the vSphere Web Client. - Log in as root.
- Select Actions > Shutdown from the menu and confirm the operation.
For more information about how to shut down the vCenter Server VM, see Reboot or Shut Down the vCenter Server Appliance in the vSphere documentation and the How to stop, start, or restart vCenter Server 6.x services KB article.
Note: After you shut down this vCenter VM, the vSphere Web Client will not be available.
View a larger version of this image
Step 10: Shut Down ESXi Hosts
To shut down each ESXi host in the vSphere cluster, do the following:
Put the ESXi host into maintenance mode by doing the following:
- Using a browser, navigate to the HTTPS IP address of the ESXi host, for example: https://10.196.146.20/.
- Log in using vSphere administrative credentials.
- Put the ESXi host in maintenance mode by selecting Actions > Enter maintenance mode.
View a larger version of this image.
Power off the ESXi host. To do this, you have two options:
- Use the EXSi web interface and select Actions > Shut down.
- Use the remote management console for the host, such as Dell IDRAC or HP iLO.
View a larger version of this image.
Startup Sequence and Tasks
To restart all Kubernetes, Enterprise PKS, and infrastructure components, complete the following tasks in the sequence presented.
Step 1: Start ESXi Hosts
To start the ESXi hosts, do the following:
- Using the remote management console, such as Dell IDRAC or HP iLO, power on each ESXi host.
- Connect to the web interface of each ESXi host and exit maintenance mode.
Step 2: Start vCenter
Connect to the web interface of the ESXi server that hosts the vCenter VM. Select the vCenter VM, and click Power On.
Step 3: Start NSX-T Components
To start the NSX-T components, perform the following steps:
- Log into vCenter using the vSphere Client.
- Power on the following VMs in the following order:
- NSX-T Manager
- NSX-T Controllers
- NSX-T Edge Nodes
Step 4: Start Ops Manager
- Using vCenter, power on the Ops Manager VM.
- Using a browser, go to the Ops Manager URL.
- Enter the Ops Manager passphrase.
- Log in to the Ops Manager UI.
Step 5: Start the BOSH Director
Using vCenter, power on the BOSH Director VM.
Note: BOSH is aware that all the VMs under its control were stopped. BOSH does not attempt to resurrect any VMs, which is the desired behavior.
It may take approximately 90 minutes for BOSH to start properly.
To speed up the BOSH startup process:
- Obtain the BOSH Director VM Credentials from Ops Manager. For information about doing this, see Retrieving Credentials from Your Deployment in the Pivotal documentation.
- SSH to the BOSH Director VM.
On the BOSH Director VM, run the following commands:
sudo -i monit summary
If you see messages such as
Process uaa Connection failed
andProcess credhub not monitored
, then run the following command:monit restart uaa
After a few minutes, run the following command again:
monit summary
You should see that the
uaa
andcredhub
processes are now running. At this point, the BOSH Director should be fully up and running.
Step 6: Start the Enterprise PKS Control Plane
To start the PKS control plane, do the following:
- Using vCenter, power on the PKS control plane VM.
Using the BOSH CLI, start the Enterprise PKS process on the VM by running the following command:
bosh -d pivotal-container-service-DEPLOYMENT-ID start
Where
DEPLOYMENT-ID
is the BOSH-generated ID of the Enterprise PKS deployment. For example:$ bosh -d pivotal-container-service-1bf7b02738056cdc37e6 start
Note: Because you stopped the Enterprise PKS process using BOSH, you must restart it using BOSH.
Step 7: Start Harbor Registry
To start Harbor Registry, do the following:
- Using vCenter, power on the Harbor VM.
Using the BOSH CLI, start the Harbor process on the VM by running the following command:
bosh -d harbor-container-registry-DEPLOYMENT-ID start
Where
DEPLOYMENT-ID
is the BOSH-generated ID of your Harbor Registry deployment. For example:$ bosh -d harbor-container-registry-b4023f6857207b237399 start
Step 8: Start the Kubernetes Clusters
For each Kubernetes cluster that you intend to start up, start the Kubernetes nodes in the following order:
- Using the BOSH CLL, run
ssh
to access the first Enterprise PKS master node and start etcd. - Using the BOSH CLI, start the next Enterprise PKS master node.
- Using the BOSH CLI, start all remaining Enterprise PKS master nodes including the master where you started etcd.
- Using the BOSH CLI, start all Enterprise PKS worker nodes.
For exact Kubernetes node startup instructions, refer to the How to shutdown and startup a Multi Master PKS cluster knowledge base article.
Step 9: Start Customer Apps
Start all apps running on the Enterprise PKS-provisioned Kubernetes clusters.
Step 10: Re-enable BOSH Resurrection
Turn BOSH resurrection back on by running the command bosh update-resurrection on
.
Please send any feedback you have to pks-feedback@pivotal.io.