Shutting Down and Starting Up Enterprise PKS
-
Shutdown Sequence and Tasks
- Step 1: Shut Down Customer Apps
- Step 2: Shut Down Kubernetes Clusters
- Step 3: Shut Down Enterprise PKS Control Plane
- Step 4: Shut Down VMware Harbor Registry
- Step 5: Shut Down BOSH Director
- Step 6: Shut Down Ops Manager
- Step 7: Shut Down NSX-T Components
- Step 8: Shut Down vCenter Server
- Step 9: Shut Down ESXi Hosts
- Startup Sequence and Tasks
Page last updated:
This topic lists and describes the shutdown and startup sequence for Enterprise Pivotal Container Service (Enterprise PKS) including Enterprise PKS-provisioned Kubernetes cluster nodes, PKS control plane components, and vSphere hosts.
Shutdown Sequence and Tasks
To perform a graceful shutdown of all Kubernetes, Enterprise PKS, and infrastructure components, complete the following tasks in sequence.
Step 1: Shut Down Customer Apps
Shut down all customer apps running on Enterprise PKS-provisioned Kubernetes clusters.
Note: This task is optional. Perform it after considering the types of apps you have deployed. For example, stateful, stateless, or legacy apps.
Step 2: Shut Down Kubernetes Clusters
Shut down all Enterprise PKS-provisioned Kubernetes clusters following the procedure defined in the How to shutdown and startup a Multi Master PKS cluster knowledge base article.
For each Kubernetes cluster that you intend to shut down, do the following:
Using the BOSH CLI, retrieve the BOSH deployment name of your Enterprise PKS clusters by running the following command:
bosh deployments
Kubernetes cluster deployment names begin with
service-instance_
and include a unique BOSH-generated hash.Using the BOSH CLI, stop the Kubernetes worker nodes by running the following command:
bosh -d service-instance_CLUSTER-UUID stop worker
Where
CLUSTER-UUID
is the BOSH deployment name of your Enterprise PKS cluster. For example:$ bosh -d service-instance_aa1234567bc8de9f0a1c stop worker
Note: When you use the BOSH
stop
command, all processes on the Kubernetes node are stopped. BOSH marks them stopped so that when the VM is powered back on, the processes do not start automatically.Using the BOSH CLI, stop the Kubernetes master nodes by running the following command:
bosh -d service-instance_CLUSTER-UUID stop master
Where
CLUSTER-UUID
is the BOSH deployment name of your Enterprise PKS cluster. For example:$ bosh -d service-instance_aa1234567bc8de9f0a1c stop master
Using vCenter, shut down all Kubernetes node VMs. To do this, perform the following steps:
- Verify the node type by checking the “job” name in the the Custom Attributes pane.
Perform a graceful shutdown by right-clicking the target VM and selecting Power > Shut Down Guest OS.
Step 3: Shut Down Enterprise PKS Control Plane
To shut down the PKS control plane VM, do the following:
Using the BOSH CLI, retrieve the BOSH deployment ID of your Enterprise PKS deployment by running the following command:
bosh deployments
Enterprise PKS deployment names begin with
pivotal-container-service
and include a unique BOSH-generated hash.Using the BOSH CLI, stop the PKS control plane VM by running the following command:
bosh -d pivotal-container-service-DEPLOYMENT-ID stop
Where
DEPLOYMENT-ID
is the BOSH-generated ID of your Enterprise PKS deployment. For example:$ bosh -d pivotal-container-service-1bf7b02738056cdc37e6 stop
Using vCenter, locate and gracefully shut down the PKS control plane VM.
Step 4: Shut Down VMware Harbor Registry
To shut down the Harbor Registry VM, do the following:
Using the BOSH CLI, retrieve the BOSH deployment ID of your Harbor Registry deployment by running the following command:
bosh deployments
Harbor Registry deployment names begin with
harbor-container-registry
and include a unique BOSH-generated hash.Using the BOSH CLI, stop the Harbor Registry VM by running the following command:
bosh -d harbor-container-registry-DEPLOYMENT-ID stop
Where
DEPLOYMENT-ID
is the BOSH-generated ID of your Harbor Registry deployment. For example:$ bosh -d harbor-container-registry-b4023f6857207b237399 stop
Using vCenter, locate and gracefully shut down the Harbor Registry VM.
Step 5: Shut Down BOSH Director
Using vCenter, locate and gracefully shut down the BOSH Director VM.
View a larger version of this image.
Step 6: Shut Down Ops Manager
Using vCenter, locate and gracefully shut down the Ops Manager VM.
View a larger version of this image.
Step 7: Shut Down NSX-T Components
Using vCenter, gracefully shut down all NSX-T VMs in the following order:
- NSX-T Manager
- NSX-T Controllers
- NSX-T Edge Nodes
View a larger version of this image.
Step 8: Shut Down vCenter Server
To shut down the vCenter Server VM, do the following:
- Navigate to the vCenter Appliance Management Interface at
https://YOUR-VCENTER-HOSTNAME-OR-IP-ADDRESS>:5480
, whereYOUR-VCENTER-HOSTNAME-OR-IP-ADDRESS
is the hostname or IP address that you use to connect to vCenter through the vSphere Web Client. - Log in as root.
- Select Actions > Shutdown from the menu and confirm the operation.
For more information about how to shut down the vCenter Server VM, see Reboot or Shut Down the vCenter Server Appliance in the vSphere documentation and the How to stop, start, or restart vCenter Server 6.x services KB article.
Note: After you shut down this vCenter VM, the vSphere Web Client will not be available.
View a larger version of this image
Step 9: Shut Down ESXi Hosts
To shut down each ESXi host in the vSphere cluster, do the following:
Put the ESXi host into maintenance mode by doing the following:
- Using a browser, navigate to the HTTPS IP address of the ESXi host, for example: https://10.196.146.20/.
- Log in using vSphere administrative credentials.
- Put the ESXi host in maintenance mode by selecting Actions > Enter maintenance mode.
View a larger version of this image.
Power off the ESXi host. To do this, you have two options:
- Use the EXSi web interface and select Actions > Shut down.
- Use the remote management console for the host, such as Dell IDRAC or HP iLO.
View a larger version of this image.
Startup Sequence and Tasks
To restart all Kubernetes, Enterprise PKS, and infrastructure components, complete the following tasks in the sequence presented.
Step 1: Start ESXi Hosts
To start the ESXi hosts, do the following:
- Using the remote management console, such as Dell IDRAC or HP iLO, power on each ESXi host.
- Connect to the web interface of each ESXi host and exit maintenance mode.
Step 2: Start vCenter
Connect to the web interface of the ESXi server that hosts the vCenter VM. Select the vCenter VM, and click Power On.
Step 3: Start NSX-T Components
To start the NSX-T components, perform the following steps:
- Log into vCenter using the vSphere Client.
- Power on the following VMs in the following order:
- NSX-T Manager
- NSX-T Controllers
- NSX-T Edge Nodes
Step 4: Start Ops Manager
- Using vCenter, power on the Ops Manager VM.
- Using a browser, go to the Ops Manager URL.
- Enter the Ops Manager passphrase.
- Log in to the Ops Manager UI.
Step 5: Start the BOSH Director
Using vCenter, power on the BOSH Director VM.
Note: BOSH is aware that all the VMs under its control were stopped. BOSH does not attempt to resurrect any VMs, which is the desired behavior.
It may take approximately 90 minutes for BOSH to start properly.
To speed up the BOSH startup process:
- Obtain the BOSH Director VM Credentials from Ops Manager. For information about doing this, see Retrieving Credentials from Your Deployment in the Pivotal documentation.
- SSH to the BOSH Director VM.
On the BOSH Director VM, run the following commands:
sudo -i monit summary
If you see messages such as
Process uaa Connection failed
andProcess credhub not monitored
, then run the following command:monit restart uaa
After a few minutes, run the following command again:
monit summary
You should see that the
uaa
andcredhub
processes are now running. At this point, the BOSH Director should be fully up and running.
Step 6: Start the Enterprise PKS Control Plane
To start the PKS control plane, do the following:
- Using vCenter, power on the PKS control plane VM.
Using the BOSH CLI, start the Enterprise PKS process on the VM by running the following command:
bosh -d pivotal-container-service-DEPLOYMENT-ID start
Where
DEPLOYMENT-ID
is the BOSH-generated ID of the Enterprise PKS deployment. For example:$ bosh -d pivotal-container-service-1bf7b02738056cdc37e6 start
Note: Because you stopped the Enterprise PKS process using BOSH, you must restart it using BOSH.
Step 7: Start Harbor Registry
To start Harbor Registry, do the following:
- Using vCenter, power on the Harbor VM.
Using the BOSH CLI, start the Harbor process on the VM by running the following command:
bosh -d harbor-container-registry-DEPLOYMENT-ID start
Where
DEPLOYMENT-ID
is the BOSH-generated ID of your Harbor Registry deployment. For example:$ bosh -d harbor-container-registry-b4023f6857207b237399 start
Step 8: Start the Kubernetes Clusters
For each Kubernetes cluster that you intend to start up, start the Kubernetes nodes in the following order:
- Using the BOSH CLL, run
ssh
to access the first Enterprise PKS master node and start etcd. - Using the BOSH CLI, start the next Enterprise PKS master node.
- Using the BOSH CLI, start all remaining Enterprise PKS master nodes including the master where you started etcd.
- Using the BOSH CLI, start all Enterprise PKS worker nodes.
For exact Kubernetes node startup instructions, refer to the How to shutdown and startup a Multi Master PKS cluster knowledge base article.
Step 9: Start Customer Apps
Start all apps running on the Enterprise PKS-provisioned Kubernetes clusters.
Please send any feedback you have to pks-feedback@pivotal.io.