Defining Network Profiles for Shared Tier-1 Router

Page last updated:

Warning: Pivotal Container Service (PKS) v1.5 is no longer supported because it has reached the End of General Support (EOGS) phase as defined by the Support Lifecycle Policy.
To stay up to date with the latest software and security updates, upgrade to a supported version.

This topic describes how to define network profiles to enable Kubernetes clusters to use a shared Tier-1 Router.

About the Shared Tier-1 Topology

A Shared Tier-1 network topology provides an alternative approach for deploying Kubernetes clusters.

Note: The Shared Tier-1 topology requires NSX-T Data Center v2.5 or later.

By default when a Kubernetes cluster is provisioned by Enterprise PKS, the system creates the following NSX-T objects:

  • 1 Logical Switch and Tier-1 Router for each Kubernetes Nodes subnet
  • 1 Logical Switch and Tier-1 Router for each Kubernetes namespace
  • 1 Logical Switch and Tier-1 Router each NSX-T Load Balancer that is allocated for the Kubernetes cluster

As depicted, the result is that a given Kubernetes cluster will run several Tier-1 switches and routers in its topology.

An alternative topology is to use a single, shared Tier-1 switch and router for each Kubernetes cluster. As depicted, the shared Tier-1 model only uses one Tier-1 router and multiple logical switches connected to the shared Tier-1 to connect all Kubernetes cluster components, including:

  • Kubernetes Nodes Networks
  • Kubernetes Namespaces
  • NSX-T load balancer instances allocated for the Kubernetes cluster

The shared Tier-1 model configures any necessary NAT rules (if using NAT mode) on the single Tier-1 router directly. The Tier-0 router is no longer used for any NAT configuration. As a result, the Tier-0 router can operate in Active/Active mode if all Kubernetes clusters are deployed using the Shared Tier-1 model. The Shared Tier-1 model enables higher scale numbers for PKS as number of NSX-T objects allocated per Kubernetes cluster is drastically reduced. The advantage of the shared Tier-1 topology is that you can increase the number of NSX-T objects that can be supported in a given cluster.

Implementing a Shared Tier-1 Topology

To implement a shared Tier-1 topology, you define a network profile that enables the single_tier_topology key. Shown below is an example network profile to enable a Shared Tier-1 Router for Kubernetes clusters:

{
  "name": "example-network-profile-shared-t1",
  "description": "Shared-Tier-1 topology network profile",
  "parameters": {
    "single_tier_topology": true
  }
}

To create a Shared Tier-1 network profile, see Create Network Profile.

To create a cluster using a Shared Tier-1 network profile, see Create a Cluster with a Network Profile.

Implementing a Shared Tier-1 Topology in a Multi-Tier-0 Environment

In a Shared Tier-1 Router topology, all Kubernetes cluster traffic is automatically NATed in the single Tier-1 router that services that cluster. However, in a Multi-Tier-0 environment, traffic from Kubernetes Node Networks to the Shared Tier-0 Router cannot be NATed.

To implement a Shared Tier-1 topology in a Multi-Tier-0 environment, use the infrastructure_networks field in the network profile and include the subnets where your infrastructure is running. During Kubernetes cluster creation, Enterprise PKS will add a NO_SNAT rule from the Node Network to subnets specified in the infrastructure_networks field.

In the following example network profile, the infrastructure-networks field includes three subnets for which NO_SNAT rules will be created. These subnets map to the PKS Control Plane (30.0.0.0/24), vCenter and NSX-T VMs (192.168.111.0/24), and the Nodes DNS server (192.168.115.1).

{
  "name": "tenant-A-shared-T1",
  "description": "Example Network Profile for Tenant A Shared Tier-1 Router Topology",
  "parameters": {
    "t0_router_id": "a6addd27-24ce-469a-979e-cf742a19ef5c",
    "fip_pool_ids": [
      "a8b7f715-42f0-46bf-a4f2-1599c55058b6" ],
    "pod_ip_block_ids": [
      "edd59bf6-ff04-420c-88de-2c43d47f7130" ],
    "infrastructure_networks": [
      "30.0.0.0/24",
      "192.168.111.0/24",
      "192.168.115.1"
    ],
    "single_tier_topology": true
  }
}

Please send any feedback you have to pks-feedback@pivotal.io.