Defining Network Profile for the HTTP/S Ingress Controller

Page last updated:

Warning: VMware Enterprise PKS v1.6 is no longer supported because it has reached the End of General Support (EOGS) phase as defined by the Support Lifecycle Policy. To stay up to date with the latest software and security updates, upgrade to a supported version.

This topic describes how to define network profiles for Kubernetes clusters provisioned with VMware Enterprise PKS on vSphere with NSX-T.

About the NCP Ingress Controller

The NSX-T Load Balancer is a logical load balancer that handles a number of functions using virtual servers and pools.

By default the following NSX-T virtual servers are deployed by Enterprise PKS:

  • 1 per cluster TCP layer 4 virtual server for the Kubernetes API server
  • 1 TCP layer 4 virtual server for each Kubernetes service of type: LoadBalancer
  • 2 per cluster layer 7 virtual servers for the Kubernetes Ingress Controller (HTTP and HTTPS)

The TCP layer 4 virtual server that fronts the Kubernetes API server is always created, and it is not controlled by the parameters exposed in the network profile.

The nsx_lb parameter is used to control the TCP layer 4 virtual server that is provisioned for each Kubernetes service of type: LoadBalancer.

The nsx_ingress_controller parameter is used to control if NCP is used as the Ingress Controller for the Kubernetes cluster. By default when you define an ingress resource for a Kubernetes cluster, NCP instructs the NSX-T load balancer to provision 2 layer 7 virtual services (HTTP and HTTPS) as the Ingress Controller.

The nsx_ingress_controller parameter is subject to the nsx_lb parameter as described in the following table.

nsx_lb setting “nsx_ingress_controller”: true (default) “nsx_ingress_controller”: false
“nsx_lb”: true (default) This means you want to use the NSX-T Layer 4 Load Balancer and the NCP-provisioned Layer 7 Ingress Controller. This means you want to use the NSX-T Layer 4 Load Balancer and a third-party Ingress Controller, such as NGINX.
“nsx_lb”: false This is an invalid configuration because you cannot disable the NSX-T Load Balancer and use NCP as the Ingress Controller. The network profile will fail validation. This means you want to use a third-party load balancer and a third-party ingress controller.

Ingress Parameters

The following table shows the ingress parameters for configuring NCP:








Parameter Type Description
name String User-defined name of the network profile.
description String User-defined description for the network profile.
parameters Map One or more name-value pairs.
cni_configurations Map Map containing two key-value pairs: type and parameters.
type Constant String Only nsxt is accepted.
parameters Map Map containing one or more key-value pairs for NCP settings.
nsx_ingress_controller Boolean Flag to control if NCP is used as the Ingress Controller for the Kubernetes cluster.
ingress_ip String IP address to use for ingress controller load balancer.
ingress_persistence_settings Map Holds parameters for customizing Layer 7 persistence.
persistence_type String Valid values are cookie or source_ip. An empty value is not accepted.
persistence_timeout Integer Value that is equal to 1 or larger. Empty value is not accepted.

Example Network Profile for Ingress

The following network profile is an example that illustrates the parameters exposed for configuring the NSX Ingress Controller.

{
  "name": "ncp_network_profile",
  "description": "Example network profile for ingress controller",
  "parameters" : {
     "cni_configurations": {
         "type": "nsxt",
         "parameters": {
           "nsx_ingress_controller": false,        
           "ingress_ip": "192.168.160.212",
           "ingress_persistence_settings": {
             "persistence_type": "cookie",
             "persistence_timeout": 1 
             }
           }
        }
    }
}

NSX Ingress Controller

NCP depends on the NSX-T Load Balancer to fulfill its role as an Ingress Controller. To use a third-party ingress controller, such as the NGINX Ingress Controller, set the nsx_ingress_controller to false.

For example:

{
  "name": "example_network_profile",
  "description": "Use the nsx_lb with a 3rd party ingress controller",
  "parameters": {
    "cni_configurations": {
      "type": "nsxt",
      "parameters": {
         "nsx_lb": true,
         "nsx_ingress_controller": false
      }
    }
  }
}

The following network profile uses a third party load balancer and a third-party ingress controller.

{
  "name": "example_network_profile",
  "description": "Use the nsx_lb with a 3rd party ingress controller",
  "parameters": {
    "cni_configurations": {
      "type": "nsxt",
      "parameters": {
         "nsx_lb": false,
         "nsx_ingress_controller": false
      }
    }
  }
}

The following network profile is invalid; you cannot disable the nsx_lb and use the NCP Ingress Controller.

{
  "name": "example_network_profile",
  "description": "Use the nsx_lb with a 3rd party ingress controller",
  "parameters": {
    "cni_configurations": {
      "type": "nsxt",
      "parameters": {
         "nsx_lb": false,
         "nsx_ingress_controller": true
      }
    }
  }
}

Ingress IP

The ingress_ip parameter instructs NCP to create an ingress virtual server with the given IP address.

The ingress_ip parameter type is a string that accepts any valid IP address. Missing entry is accepted.

Example network profile for ingress_ip:

{
  "name": "example-network-profile",
  "description": "ingress_ip",
  "parameters" : {
     "cni_configurations": {
         "type": "nsxt",
         "parameters": {
           "ingress_ip": "192.168.160.212"
         }
     }
  }
}

An invalid IP address is rejected with an invalid parameter value error. For example, the following network profile parameters cannot be parsed becase the ‘ingress_ip’: '192.168.460.212’ address is an invalid IP address.

{
  "name": "example-network-profile",
  "description": "ingress_ip-ERROR",
  "parameters" : {
     "cni_configurations": {
         "type": "nsxt",
         "parameters": {
           "ingress_ip": "192.168.460.212"
         }
     }
  }
}

The following network profile cannot be parsed because the JOSN input is invalid (not a string).

{
  "name": "example-network-profile",
  "description": "ingress_ip-ERROR",
  "parameters" : {
     "cni_configurations": {
         "type": "nsxt",
         "parameters": {
           "ingress_ip": 192.168.160.212
         }
     }
  }
}

Ingress Persistence Settings

The ingress_persistence parameter lets you customize layer 7 persistence for Kubernetes services.

The ingress_persistence_settings parameter is a map that supports two keys: persistence_type and persistence_timeout. These two keys are correlated and must be set/unset at the same time. If persistence_type and persistence_timeout are not both specified, the network profile fails validation.

Parameter Data type Description
persistence_type String Valid values are cookie or source_ip. An empty value is not accepted.
persistence_timeout Integer Value that is equal to 1 or larger. Empty value is not accepted.

Example 1: network profile for ingress_persistence_settings:

{
  "name": "example_network_profile",
  "description": "ingress_persistence_settings",
  "parameters" : {
     "cni_configurations": {
         "type": "nsxt",
         "parameters": {
           "ingress_ip": "192.168.160.212"       
           "ingress_persistence_settings": {
             "persistence_type": "cookie",
             "persistence_timeout": 1
            }
         }
     }
  }
}

Example 2: network profile for ingress_persistence_settings:

{
  "name": "example_network_profile",
  "description": "ingress_persistence_settings",
  "parameters" : {
     "cni_configurations": {
         "type": "nsxt",
         "parameters": {
           "ingress_ip": "192.168.160.212"         
           "ingress_persistence_settings": {
             "persistence_type": "source_ip",
             "persistence_timeout": 100
            }
         }
     }
  }
}

Please send any feedback you have to pks-feedback@pivotal.io.