LATEST VERSION: v1.4 - RELEASE NOTES
Pivotal Container Service v1.3

vSphere with NSX-T Cluster Objects

Page last updated:

This topic lists and describes the vSphere VMs and NSX-T objects that Pivotal Container Service (PKS) creates when you create a Kubernetes cluster. When you delete a Kubernetes cluster, PKS removes these objects.

For information about creating a Kubernetes cluster using PKS, see Creating Clusters. For information about deleting a Kubernetes cluster using PKS, see Deleting Clusters.

vSphere Virtual Machines

When a new Kubernetes cluster is created, PKS creates the following virtual machines (VMs) in the designated vSphere cluster:

Object Number Object Description
1 or 3 Kubernetes master nodes. The number depends on the plan used to create the cluster.
1 or more Kubernetes worker nodes. The number depends on the plan used to create the cluster, or the number specified during cluster creation.

Note: For production clusters, three master nodes are required, and a minimum of three worker nodes are required. See Requirements for PKS on vSphere with NSX-T for more information.

NSX-T Logical Switches

When a new Kubernetes cluster is created, PKS creates the following NSX-T logical switches:

Object Number Object Description
1 Logical switch for Kubernetes master and worker nodes.
1 Logical switch for each Kubernetes namespace: default, kube-public, kube-system, pks-infrastructure.
1 Logical switch for the NSX-T load balancer associated with the Kubernetes cluster.

NSX-T Tier-1 Logical Routers

When a new Kubernetes cluster is created, PKS creates the following NSX-T Tier-1 logical routers:

Object Number Object Description
1 Tier-1 router for Kubernetes master and worker nodes. Name: cluster-router.
1 Tier-1 router for each Kubernetes namespace: default, kube-public, kube-system, pks-infrastructure.
1 Tier-1 router for the NSX-T load balancer associated with the Kubernetes cluster.

NSX-T Load Balancers

For each Kubernetes cluster created, PKS creates a single instance of a small NSX-T load balancer. This load balancer contains the objects listed in the following table:

Object Number Object Description
1 Virtual Server (VS) to access Kubernetes control plane API on port 8443.
1 Server Pool containing the 3 Kubernetes master nodes.
1 VS for HTTP Ingress Controller.
1 VS for HTTPS Ingress Controller.

The IP address allocated to each VS is derived from the Floating IP Pool that was created for use with PKS. The VS for the HTTP Ingress Controller and the VS for the HTTPS Ingress Controller use the same IP address.

NSX-T DDI/IPAM

For each Kubernetes cluster created, PKS extracts and allocates the following NSX-T subnets from the IP blocks created in preparation for installing PKS with NSX-T:

Object Number Object Description
1 A /24 subnet from the Nodes IP Block will be extracted and allocated for the Kubernetes master and worker nodes.
1 A /24 subnet from the Pods IP Block will be extracted and allocated for each Kubernetes namespace: default, kube-public, kube-system, pks-infrastructure.

NSX-T Tier-0 Logical Routers

For each Kubernetes cluster created, PKS defines the following NSX-T NAT rules on the Tier-0 logical router:

Object Number Object Description
1 SNAT rule created for each Kubernetes namespace: default, kube-public, kube-system, pks-infrastructure using 1 IP from the Floating IP Pool as translated IP address.
1 (NAT topology only) SNAT rule created for each Kubernetes cluster using 1 IP from the Floating IP Pool as translated IP address. The Kubernetes cluster subnet is derived from the Nodes IP Block using a /24 netmask.

NSX-T Distributed Firewall (DFW) Rules

For each Kubernetes cluster created, PKS defines the following NSX-T distributed firewall rules:

Object Amount Object Description
1 DFW rule for kubernetes-dashboard: Source=Kubernetes worker node (hosting the Dashboard Pod); Destination=Dashboard Pod IP; Port: TCP/8443; Action: allow
1 DFW rule for kube-dns: Source=Kubernetes worker node (hosting the DNS Pod); Destination=DNS Pod IP; Port: TCP/8081 and TCP/10054; Action: allow

Please send any feedback you have to pks-feedback@pivotal.io.

Create a pull request or raise an issue on the source for this page in GitHub