Defining Network Profiles for Edge Router Selection

Page last updated:

This topic describes how to define network profiles for Kubernetes clusters provisioned with Enterprise Pivotal Container Service (Enterprise PKS) on vSphere with NSX-T.

Edge Router Selection

Using Enterprise PKS on vSphere with NSX-T, you can deploy Kubernetes clusters on dedicated Tier-0 routers, creating a multi-tenant environment for each Kubernetes cluster. As shown in the diagram below, with this configuration a shared Tier-0 router hosts the PKS control plane and connects to each customer Tier-0 router using BGP. To support multi-tenancy, configure firewall rules and security settings in NSX Manager.

Cluster Isolation Using Multiple T0 Routers

To deploy Kubernetes clusters on tenancy-based Tier-0 router(s), follow the steps below:

  1. For each Kubernetes tenant, create a dedicated Tier-0 router, and configure static routes, BGP, NAT and Edge Firewall security rules as required by each tenant. For instructions, see Configuring Multiple Tier-0 Routers for Tenant Isolation.
  2. Define a network profile per tenant that references the Tier-0 router UUID provisioned for that tenant. For example, the following network profiles define two tenant Tier-0 routers with a NATed topology.

    np_customer_A-NAT.json
    {
      "description": "network profile for Customer A",
      "name": "network-profile-Customer-A",
      "parameters": {
        "lb_size": "medium",
        "t0_router_id": "82e766f7-67f1-45b2-8023-30e2725600ba",
        "fip_pool_ids": ["8ec655f-009a-79b7-ac22-40d37598c0ff"],
        "pod_ip_block_ids": ["fce766f7-aaf1-49b2-d023-90e272e600ba"]
      }
    }
    
    np_customer_B-NAT.json
    {
      "description": "network profile for Customer B",
      "name": "network-profile-Customer-B",
      "parameters": {
        "lb_size": "small",
        "t0_router_id": "a4e766cc-87ff-15bd-9052-a0e2425612b7",
        "fip_pool_ids": ["4ec625f-b09b-29b4-dc24-10d37598c0d1"],
        "pod_ip_block_ids": ["91e7a3a1-c5f1-4912-d023-90e272260090"]
      }
    }
    

    The following network profiles define two customer Tier-0 routers for a no-NAT topology:

    np_customer_A.json
    {
      "description": "network profile for Customer A",
      "name": "network-profile-Customer-A",
      "parameters": {
        "lb_size": "medium",
        "t0_router_id": "82e766f7-67f1-45b2-8023-30e2725600ba",
        "fip_pool_ids": [
            "8ec655f-009a-79b7-ac22-40d37598c0ff",
            "7ec625f-b09b-29b4-dc24-10d37598c0e0"
        ],
        "pod_routable": "true",
        "pod_ip_block_ids": [
            "fce766f7-aaf1-49b2-d023-90e272e600ba",
            "6faf46fd-ccce-4332-92d2-d918adcccce0"
        ]
      }
    }
    
    np_customer_B.json
    {
      "description": "network profile for Customer B",
      "name": "network-profile-Customer-B",
      "parameters": {
        "lb_size": "small",
        "t0_router_id": "a4e766cc-87ff-15bd-9052-a0e2425612b7",
        "fip_pool_ids": [
            "4ec625f-b09b-29b4-dc24-10d37598c0d1",
            "6ec625f-b09b-29b4-dc24-10d37598dDd1"
        ],
        "pod_routable": "true",
        "pod_ip_block_ids": [
            "91e7a3a1-c5f1-4912-d023-90e272260090",
            "6faf46fd-ccce-4332-92d2-d918adcccce0"
        ]
      }
    }
    

    Note: The pod_routable parameter controls the routing behavior of a tenant Tier-0 router. If the parameter is set to true, the custom Pods IP Block subnet is routable and NAT is not used. If pod_routable is not present or is set to false, the custom Pods IP Block is not routable and the tenant Tier-0 is deployed in NAT mode.


Please send any feedback you have to pks-feedback@pivotal.io.