Defining Network Profiles for the Layer 4 Load Balancer

Page last updated:

This topic describes how to define network profile to configure the NSX-T Load Balancer for Enterprise Pivotal Container Service (Enterprise PKS).

About the NSX-T Load Balancer

The NSX-T Load Balancer is a logical load balancer that handles a number of functions using virtual servers and pools.

By default the following NSX-T virtual servers are deployed by Enterprise PKS:

  • 1 per cluster TCP layer 4 virtual server for the Kubernetes API server
  • 1 TCP layer 4 virtual server for each Kubernetes service of type: LoadBalancer
  • 2 per cluster layer 7 virtual servers for the Kubernetes Ingress Controller (HTTP and HTTPS)

The TCP layer 4 virtual server that fronts the Kubernetes API server is always created, and it is not controlled by the parameters exposed in the network profile.

The nsx_lb parameter is used to control the TCP layer 4 virtual server that is provisioned for each Kubernetes service of type: LoadBalancer.

The nsx_ingress_controller parameter is used to control if NCP is used as the Ingress Controller for the Kubernetes cluster. By default when you define an ingress resource for a Kubernetes cluster, NCP instructs the NSX-T load balancer to provision 2 layer 7 virtual services (HTTP and HTTPS) as the Ingress Controller.

The nsx_ingress_controller parameter is subject to the nsx_lb parameter as described in the following table.

nsx_lb setting nsx_ingress_controller: true nsx_ingress_controller: false
nsx_lb: true Use the NSX-T Layer 4 LoadBalancer and the NCP-provisioned Layer 7 Ingress Controller. Use the NSX-T Layer 4 LoadBalancer and a third-party Ingress Controller, such as NGINX.
nsx_lb: false Invalid configuration. You cannot disable the NSX-T Load Balancer and use NCP as the Ingress Controller. The network profile will fail validation. Use a third-party load balancer and a third-party ingress controller.

NSX-T Load Balancer Parameters

The following table shows the parameters for configuring the NSX-T Load Balancer:







cni_configurations Map Map containing two key-value pairs: type and parameters.
type Constant String Only nsxt is accepted.
parameters Map Map containing one or more key-value pairs for NCP settings.
nsx_lb Boolean Flag to control if the NSX-T Load Balancer is used for the Kubernetes cluster.
x_forwarded_for String Sets the original client source IP in the request header. Default is none. Accepts "insert" and "replace".
max_l4_lb_service Integer Limit the maximum number of layer 4 virtual servers per cluster. Minimum is 1.
l4_persistence_type String Connection stickiness based on source_ip (only accepted value).
l4_lb_algorithm String Specify the layer 4 load balancer behavior. Accepts "round_robin" (default), "least_connection", "ip_hash", "weighted_round_robin".

Example Network Profile for NSX-T Load Balancer Configuration

The following network profile is an example that illustrates the parameters exposed for configuring ncp.ini.

{
  "name": "network_profile",
  "description": "Example network profile for nsxt load balancer",
  "parameters" : {
     "cni_configurations": {
         "type": "nsxt",
         "parameters": {
           "nsx_lb": true,        
           "x_forwarded_for": "replace",
           "max_l4_lb_service": 10,
           "l4_persistence_type": "source_ip",
           "l4_algorithm": "weighted_round_robin"           
           }
         }
    }
}

NSX Load Balancer

The nsx_lb flag gives you the choice to of deploying the NSX-T Load Balancer or a third-party load balancer, such as Nginx. The nsx_lb parameter maps to the use_native_loadbalancer parameter in NCP.ini. The default setting is true. To use a third party load balancer, set this parameter to false.

For example:

{
  "name": "example_network_profile",
  "description": "nsx_lb and nsx_ingress_controller are enabled",
  "parameters": {
    "cni_configurations": {
      "type": "nsxt",
      "parameters": {
         "nsx_lb": false,
         "nsx_ingress_controller": false
      }
    }
  }
}

Since the nsx_ingress_controller is a component of the NSX-T Load Balancer, if you choose to use a third-party load balancer (nsx_lb: false), you must also explicitly disable the nsx_ingress_controller. See Defining Network Profile for the HTTP/S Ingress Controller.

Set Client Source IP Address

The X-Forwarded-For HTTP header field is used to identify the originating IP address of a client connecting to a web server through an HTTP proxy or load balancer.

In network profile, the x_forwarded_for parameter can be enabled to ensure that the client IP address will be set in the HTTP request header. The x_forwarded_for parameter is useful in situations where it is important to know the source IP address of the client request, such as for auditing purposes.

The x_forwarded_for parameter type is String that accepts "insert" and "replace". Any other type will be rejected. Missing entry is accepted.

If set to “insert”, the client source IP will be appended (comma separated) to the existing set of client source IP addresses. If set to “replace”, any existing client source IP address will be replaced with the current client source IP address.

In the following example network profile, the source IP address of the client will be appended to the existing set of client source IP addresses.

{
  "name": "example-network-profile",
  "description": "x_forwarded_for insert",
  "parameters" : {
     "cni_configurations": {
         "type": "nsxt",
         "parameters": {
           "x_forwarded_for": "insert"
      }
    }
  }
}

In the following example network, the existing source IP address of the client will replace all other client source IP entries.

{
  "name": "example-network-profile",
  "description": "x_forwarded_for replace",
  "parameters" : {
     "cni_configurations": {
         "type": "nsxt",
         "parameters": {
           "x_forwarded_for": "replace"
      }
    }
  }
}

Max Number of Layer 4 Load Balancer Virtual Servers

The default NSX-T Load Balancer behavior is that auto-scaling is unlimited. This means that the number of layer 4 virtual servers that can be deployed is governed only by the capacity of the Edge Cluster where the load balancer service is deployed. As a result, in theory it is possible for a single Kubernetes cluster to use up all of the layer 4 virtual servers that the Edge Cluster can support.

The max_l4_service parameter sets the upper limit for the number of virtual servers that can be used by a Kubernetes cluster. You can use this parameter to limit the number of virtual servers that can be created per Kubernetes cluster.

The max_l4_lb_service data type is an integer. The value must be larger or equal to 1. Missing entry is accepted.

For example, the following network profile uses the max_l4_lb_service parameter to limit the number of layer 4 virtual servers to 100 per cluster:

{
  "name": "example_network_profile",
  "description": "max_l4_lb_service",
  "parameters" : {
     "cni_configurations": {
         "type": "nsxt",
         "parameters": {
           "max_l4_lb_service": 100
         }
     }
  }
}

Layer 4 Persistence Type

The l4_persistence_type is used to set connection stickiness based on source_ip.

The l4_persistence_type data type is string. The only accepted value is source_ip.

{
  "name": "example_network_profile",
  "description": "l4_persistence_type",
  "parameters": {
    "cni_configurations": {
      "type": "nsxt",
      "parameters": {
        "l4_persistence_type": "source_ip"
      }
    }
  }
}

Layer 4 Load Balancer Algorithm

The l4_lb_algorithm is used to set the algorithm type for the layer 4 NSX-T Load Balancer service.

The l4_lb_algorithm data type is string enumeration that accepts the following values: "round_robin" (default), "least_connection", "ip_hash", and "weighted_round_robin".

For example, the following network profile specifies the weighted_round_robin as the load balancer algorithm.

{
  "name": "example_network_profile",
  "description": "l4_lb_algorithm",
  "parameters": {
    "cni_configurations": {
      "type": "nsxt",
      "parameters": {
        "l4_lb_algorithm": "weighted_round_robin"
      }
    }
  }
}

Please send any feedback you have to pks-feedback@pivotal.io.