Load Balancers in Enterprise PKS

Page last updated:

This topic describes the types of load balancers that are used in Enterprise Pivotal Container Service (Enterprise PKS) deployments. Load balancers differ by the type of deployment.

Load Balancers in Enterprise PKS Deployments without NSX‑T

For Enterprise PKS deployments on GCP, AWS, or vSphere without NSX-T, you can configure load balancers for the following:

  • PKS API: Configuring this load balancer enables you to run PKS Command Line Interface (PKS CLI) commands from your local workstation.
  • Kubernetes Clusters: Configuring a load balancer for each new cluster enables you to run Kubernetes CLI (kubectl) commands on the cluster.
  • Workloads: Configuring a load balancer for your application workloads enables external access to the services that run on your cluster.

The following diagram, applicable to GCP, AWS, and vSphere without NSX-T, shows where each of the above load balancers can be used within your Enterprise PKS deployment. PKS diagram including all load balancer options for PKS deployments without NSX-T

If you use either vSphere without NSX-T or GCP, you are expected to create your own load balancers within your cloud provider console. If your cloud provider does not offer load balancing, you can use any external TCP or HTTPS load balancer of your choice.

About the PKS API Load Balancer

The PKS API load balancer enables you to access the PKS API from outside the network on Enterprise PKS deployments on GCP, AWS, and on vSphere without NSX-T. For example, configuring a load balancer for the PKS API enables you to run PKS CLI commands from your local workstation.

For information about configuring the PKS API load balancer on vSphere without NSX-T, see Configuring PKS API Load Balancer.

About Kubernetes Cluster Load Balancers

When you create an Enterprise PKS cluster on GCP, AWS, and on vSphere without NSX-T, you must configure external access to the cluster by creating an external TCP or HTTPS load balancer. The load balancer enables the Kubernetes CLI to communicate with the cluster.

If you create a cluster in a non-production environment, you can choose not to use a load balancer. To enable kubectl to access the cluster without a load balancer, you can do one of the following:

  • Create a DNS entry that points to the cluster’s master VM. For example:
    my-cluster.example.com    A    10.0.0.5
  • On the workstation where you run kubectl commands, add the master IP address of your cluster and kubo.internal to the /etc/hosts file. For example:
    10.0.0.5 kubo.internal

For more information about configuring a cluster load balancer, see the following:

About Workload Load Balancers

To enable external access to your Enterprise PKS app on GCP, AWS, and on vSphere without NSX-T, you can either create a load balancer or expose a static port on your workload.

For information about configuring a load balancer for your app workload, see Deploying and Exposing Basic Workloads.

If you use AWS, you must configure routing in the AWS console before you can create a load balancer for your workload. You must create a public subnet in each availability zone (AZ) where you are deploying the workload and tag the public subnet with your cluster’s unique identifier.

See the AWS Prerequisites section of Deploying and Exposing Basic Workloads before you create a workload load balancer.

Deploy Your Workload Load Balancer with an Ingress Controller

A Kubernetes ingress controller sits behind a load balancer, routing HTTP and HTTPS requests from outside the cluster to services within the cluster. Kubernetes ingress resources can be configured to load balance traffic, provide externally reachable URLs to services, and manage other aspects of network traffic.

If you add an ingress controller to your Enterprise PKS deployment, traffic routing is controlled by the ingress resource rules you define. Pivotal recommends configuring Enterprise PKS deployments with both a workload load balancer and an ingress controller.

The following diagram shows how the ingress routing can be used within your Enterprise PKS deployment. PKS diagram that shows ingress routing for both Istio and NSX-T

The load balancer on Enterprise PKS on vSphere with NSX-T is automatically provisioned with Kubernetes ingress resources without the need to deploy and configure an additional ingress controller.

For information about deploying a load balancer configured with ingress routing on GCP, AWS, Azure, and vSphere without NSX-T, see Configuring Ingress Routing. For information about ingress routing on vSphere with NSX-T, see Configuring Ingress Resources and Load Balancer Services.

Load Balancers in Enterprise PKS Deployments on vSphere with NSX‑T

Enterprise PKS deployments on vSphere with NSX-T do not require a load balancer configured to access the PKS API. They require only a DNAT rule configured so that the PKS API host is accessible. For more information, see Share the Enterprise PKS Endpoint in Installing Enterprise PKS on vSphere with NSX-T Integration.

NSX-T handles load balancer creation, configuration, and deletion automatically as part of the Kubernetes cluster create, update, and delete process. When a new Kubernetes cluster is created, NSX-T creates and configures a dedicated load balancer tied to it. The load balancer is a shared resource designed to provide efficient traffic distribution to master nodes as well as services deployed on worker nodes. Each application service is mapped to a virtual server instance, carved out from the same load balancer. For more information, see Logical Load Balancer in the NSX-T documentation.

Virtual server instances are created on the load balancer to provide access to the following:

  • Kubernetes API and UI services on a Kubernetes cluster. This enables requests to be load balanced across multiple master nodes.
  • Ingress controller. This enables the virtual server instance to dispatch HTTP and HTTPS requests to services associated with Ingress rules.
  • type:loadbalancer services. This enables the server to handle TCP connections or UDP flows toward exposed services.

Load balancers are deployed in high-availability mode so that they are resilient to potential failures and able to recover quickly from critical conditions.

Note: The NodePort Service type is not supported for Enterprise PKS deployments on vSphere with NSX-T. Only type:LoadBalancerServices and Services associated with Ingress rules are supported on vSphere with NSX-T.

Resizing Load Balancers

When a new Kubernetes cluster is provisioned using the PKS API, NSX-T creates a dedicated load balancer for that new cluster. By default, the size of the load balancer is set to Small.

With network profiles, you can change the size of the load balancer deployed by NSX-T at the time of cluster creation. For information about network profiles, see Using Network Profiles (NSX-T Only).

For more information about the types of load balancers NSX-T provisions and their capacities, see Scaling Load Balancer Resources in the NSX-T documentation.


Please send any feedback you have to pks-feedback@pivotal.io.