Creating the Enterprise PKS Management Plane
Page last updated:
Warning: Pivotal Container Service (PKS)
v1.5 is no longer supported because it has reached the End
of General Support (EOGS) phase as defined by the
Support Lifecycle Policy.
To stay up to date with the latest software and security updates, upgrade to a supported version.
Prepare the vSphere and NSX-T infrastructure for the Enterprise PKS Management Plane where the PKS, Ops Manager, BOSH Director, and Harbor Registry VMs are deployed.
Before you begin this procedure, make sure you have completed the following prerequisites for installing Enterprise PKS on vSphere with NSX-T:
- Preparing to Install Enterprise PKS on vSphere with NSX-T Data Center
- Installing and Configuring NSX-T for Enterprise PKS
The Enterprise PKS Management Plane is the network for PKS Management Plane components, including the PKS API Server, Ops Manager, BOSH Director, and Harbor Registry. The Enterprise PKS Management Plane includes a vSphere resource pool for Management Plane components, as well as a NSX Tier-1 Logical Switch, Tier-1 Logical Router, and Router Port, and NSX-T NAT rules on the Tier-0 Router.
For all types of NSX-T deployment topologies, create a Tier-1 (T1) Logical Switch and a Tier-1 Logical Router and Port. Enable route advertisement for the T1 Logical Router and advertise All NSX connected routes for the Management Plane VMs. Lastly, link the Tier-1 Router to the Tier-0 Router.
If you are using the NAT Topology, you will also need to create the following NAT rules on the Tier-0 Router:
- Source NAT (SNAT) rule to allow the PKS Management VMs to communicate with your vCenter and NSX Manager environments. For example, an SNAT rule that maps
10.172.1.1is a routable IP address from your PKS MANAGEMENT CIDR.
- Destination NAT (DNAT) rule that maps an external IP address from the PKS MANAGEMENT CIDR to the IP where you deploy Ops Manager on the Management Plane logical switch. For example, a DNAT rule that maps
172.31.0.2is the IP address you assign to Ops Manager when connected to
- Destination NAT (DNAT) rule that maps an external IP address from the PKS MANAGEMENT CIDR to the IP where you deploy Harbor on the Management Plane logical switch. For example, a DNAT rule that maps
172.31.0.3is the IP address you assign to Harbor when connected to
Lastly, if you want to provide users with remote access to the PKS CLI, you will need to define a DNAT rule that maps an external IP address from the PKS MANAGEMENT CIDR to the IP where you deploy the PKS API Server on the Management Plane logical switch. This rule lets developers use the PKS CLI remotely from their workstations or laptops. Such a rule is needed for both NAT and No-NAT topologies.
- Log in to vCenter for your vSphere environment.
- Select Compute Cluster > New Resource Pool.
- Name the resource pool, such as
- Click OK.
- Verify resource pool creation.
- In NSX Manager, select Switching > Add.
- Create a new logical switch.
- Click Add.
- Verify logical switch creation.
Defining a Tier-1 Router involves creating the router and attaching it to the logical switch, creating a router port, and advertising the routes.
Create Tier-1 Router
In NSX Manager, select Routing > Add > Tier-1 Router.
Configure the Tier-1 Router.
Verify Tier-1 Router creation.
Create Tier-1 Router Port
Select the Tier-1 Router you created.
Select Configuration > Router Ports.
Click Add and configure the Tier-1 Router Port.
- Logical Switch: Select
LS-MGMT-PKSfrom the menu list
- IP Address/mask:
Verify Tier-1 Router Port creation.
Advertise the Tier-1 Routes
- Select the Tier-1 Router > Routing > Route Advertisement.
- Advertise the Tier-1 routes as follows:
- Status: enabled
- Advertise all NSX connected routes: yes
- Click Save.
- Verify route advertisement.
Connect the Tier-1 router to the Tier-0 router and verify router-to-router connectivity.
Select the Tier-1 Router > Overview screen and verify the configuration of the Tier-1 Router up to this point.
At the Tier-1 Router > Overview screen, select the option Tier-0 Connection > Connect.
At the Connect to Tier-0 Router, select the Tier-0 Router and click Connect.
Verify connectivity between the Tier-1 and Tier-0 Routers.
Select the Tier-1 Router > Configuration > Router ports.
The Tier-1 Router created for the Management Plane should have 2 port connections: one connected to the Tier-0 router, and a second port connected to the logical switch defined for the Management Plane. This second port is the default gateway for all VMs connected to this logical switch.
Create a Source NAT (SNAT) rule on the Tier-0 Router for Enterprise PKS management components to access vCenter and NSX manager. The SNAT rule on the Tier-0 Router allows the Management Plane VMs to communicate with the vCenter and NSX-T Management environments. For example, create a SNAT rule that maps
10.172.1.1 is a routable IP address from your PKS MANAGEMENT CIDR.
Note: Limit the Destination CIDR for the SNAT rules to the subnets that contain your vCenter and NSX Manager IP addresses.
- Select Tier-0 Router > Services > NAT.
Click ADD and configure the SNAT rule:
- Destination IP:
- Translated IP:
Verify SNAT rule creation.
Create a DNAT rule on the T0 Router to access the Ops Manager Web UI, which is required to deploy Enterprise PKS.
The Destination NAT (DNAT) rule on the Tier-0 Router maps an external IP address from the PKS MANAGEMENT CIDR to the IP where you deploy Ops Manager on the Management Plane logical switch. For example, a DNAT rule that maps
172.31.0.2 is the IP address you assign to Ops Manager when connected to
To create a DNAT rule for Ops Manager:
- In the NSX-T Manager interface, select Routing > Routers.
- Select the T0 Router > Services > NAT.
- Add and configure a DNAT rule with the routable IP address as the destination and the internal IP address for Ops Manager as the translated IP:
- Action: DNAT (for example)
- Destination IP:
- Translated IP:
- Click Add.
- Verify the DNAT rule you defined.
If you are using VMware Harbor Registry with Enterprise PKS, create a DNAT rule on the Tier-0 router to access the Harbor Web UI. This DNAT rule maps the private Harbor IP address to a routable IP address from the floating IP pool on the Enterprise PKS Management network.
See Create DNAT Rule in the VMware Harbor Registry documentation for instructions.
This DNAT rule is optional depending on whether or not you need to provide external access to the PKS CLI. If you do need to provide external access, this rule is needed for both NAT and no-NAT modes.
Note: You cannot create this rule until after Enterprise PKS is installed and the PKS API VM has an IP address.
- When the Enterprise PKS installation is completed, retrieve the Enterprise PKS endpoint by performing the following steps:
- From the Ops Manager Installation Dashboard, click the Enterprise PKS tile.
- Click the Status tab and record the IP address assigned to the Pivotal Container Service job.
- Create a DNAT rule on the shared Tier-0 router to map an external IP from the PKS MANAGEMENT CIDR to the Enterprise PKS endpoint. For example, a DNAT rule that maps
172.31.0.4is the Enterprise PKS endpoint IP address on the
ls-pks-mgmtNSX-T Logical Switch.
Note: Ensure that you have no overlapping NAT rules. If your NAT rules overlap, you cannot reach Enterprise PKS Management Plane from VMs in the vCenter network.
After you complete this procedure, follow the instructions in Creating the Enterprise PKS Compute Plane.
Please send any feedback you have to firstname.lastname@example.org.