Configuring Load Balancing for PAS

Page last updated:

This topic describes how to configure load balancing for Pivotal Application Service (PAS) by entering the names of your load balancers in the Resource Config pane of the PAS tile. This procedure varies by your IaaS and how you prepared (“paved”) it for installing PAS. See the section below that corresponds to your use case.

AWS

AWS with Terraform

To set up load balancing for PAS on AWS using Terraform:

  1. Create a file named vm_extensions_config.yml with the following content, depending on which release you are using:

    • Pivotal Application Service (PAS):

      ---
      product-name: cf
      resource-config:
        diego_brain:
          elb_names:
          - alb:SSH_TARGET_GROUP_1
          - alb:SSH_TARGET_GROUP_2
          additional_vm_extensions:
          - ssh-lb-security-groups
        router:
          elb_names:
          - alb:WEB_TARGET_GROUPS_1
          - alb:WEB_TARGET_GROUPS_2
          additional_vm_extensions:
          - web-lb-security-groups
        tcp_router:
          elb_names:
          - alb:TCP_TARGET_GROUP_1
          - alb:TCP_TARGET_GROUP_2
          additional_vm_extensions:
          - tcp-lb-security-groups
      
    • Small Footprint PAS:

        ---
        product-name: cf
        resource-config:
          control:
            elb_names:
            - alb:SSH_TARGET_GROUP_1
            - alb:SSH_TARGET_GROUP_2
            additional_vm_extensions:
            - ssh-lb-security-groups
          router:
            elb_names:
            - alb:WEB_TARGET_GROUPS_1
            - alb:WEB_TARGET_GROUPS_2
            additional_vm_extensions:
            - web-lb-security-groups
          tcp_router:
            elb_names:
            - alb:TCP_TARGET_GROUP_1
            - alb:TCP_TARGET_GROUP_2
            additional_vm_extensions:
            - tcp-lb-security-groups
      
  2. Replace values in the file as follows:

    • SSH_TARGET_GROUP_X: Enter your SSH target groups. You can find these values by running:

      terraform output ssh_target_groups
      
    • WEB_TARGET_GROUPS_X: Enter your web target groups. You can find these values by running:

      terraform output web_target_groups
      
    • TCP_TARGET_GROUP_X: Enter your TCP target groups. You can find these values by running:

      terraform output tcp_target_groups
      
  3. Apply the VM extension configuration using the om CLI. For more information about om, see the Om repository on GitHub.

    om -k \
      -t "OPS-MANAGER-FQDN" \
      -u "USERNAME" \
      -p "PASSWORD" \
      configure-product \
      -c vm_extensions_config.yml
    

    Where:

    • OPS-MANAGER-FQDN is the URL at which you access your Ops Manager instance. This corresponds to ops_manager_dns in the Terraform output.
    • USERNAME is the user name you entered when configuring internal authentication.
    • PASSWORD is the password you entered when configuring internal authentication.

      Note: If you did not configure internal authentication, you must modify this command to use a client ID and secret instead of user name and password. For more information, see Authentication in the Om repository on GitHub.

AWS Paved Manually

To configure the Gorouter or HAProxy to use AWS Elastic Load Balancers:

  1. Record the names of your ELBs. If you followed the procedures in Installing Pivotal Platform on AWS Manually, you created:

    • -ssh-elb: An SSH load balancer. This is a Classic Load Balancer.
    • -tcp-elb: A TCP load balancer. This is a Classic Load Balancer.
    • -web-elb: A web load balancer. This is an Application Load Balancer.
    • -web-elb-target-group: A target group for the web load balancer.
  2. In the PAS tile, select Resource Config.

  3. Enter the name of your SSH load balancer.

    1. Show the LOAD BALANCERS field underneath the job that handles SSH requests. This depends on the PAS release you are using:
      • PAS: Click the icon next to the Diego Brain job name to expand the row.
      • Small Footprint PAS: Click the icon next to the Control job name to expand the row.
    2. In the LOAD BALANCERS field, enter the name of your SSH load balancer: -ssh-elb.
  4. Click to expand the Router row, and fill in the LOAD BALANCERS field with a value determined by the type of load balancer you are using:

    • Application Load Balancer: Enter the name of the target group of your web load balancer, prefixed with alb:: alb:-web-elb-target-group. The prefix indicates to Ops Manager that you entered the name of a target group, and is required for AWS Application Load Balancers or Network Load Balancers.
    • Classic Load Balancer: Enter the name of the load balancer: -web-elb.

      Note: If you are using HAProxy in your deployment, then put the name of the load balancers in the LOAD BALANCERS field of the HAProxy row instead of the Router row. For a high-availability configuration, scale up the HAProxy job to more than one instance.

  5. If you enabled TCP routing, expand the TCP Router row and enter the name of your TCP load balancer: -tcp-elb.

Azure

Azure with Terraform

To configure the Gorouter to use Azure load balancers:

  1. Select Resource Config.

  2. Ensure a Standard VM type is selected for the Router VM. The PAS deployment fails if you select a Basic VM type.

  3. Click the icon next to the Router job name to expand the row, showing a LOAD BALANCERS field and INTERNET CONNECTED checkbox underneath.

  4. Enter the value of web_lb_name from your Terraform output in the Resource Config pane under LOAD BALANCERS for the Router job.

  5. Click to expand the row for the job that handles SSH requests. This depends on the PAS release you are using:

    • PAS: Click the icon next to Diego Brain.
    • Small Footprint PAS: Click the icon next to Control.
  6. For the SSH load balancer, enter the value of diego_ssh_lb_name from your Terraform output.

  7. Ensure that the INTERNET CONNECTED checkboxes are deselected for all jobs.

  8. Scale the number of instances as appropriate for your deployment.

    Note: For a high-availability deployment of Pivotal Platform on Azure, Pivotal recommends scaling the number of each PAS job to a minimum of three instances. Using three or more instances for each job creates a sufficient number of availability sets and fault domains for your deployment. For more information, see Azure Reference Architecture.

Azure Paved Manually

To configure the Gorouter to use Azure load balancers:

  1. Select Resource Config.

  2. Ensure a Standard VM type is selected for the Router VM. The PAS deployment fails if you select a Basic VM type.

  3. Retrieve the name(s) of your external ALB by navigating to the Azure portal, clicking All resources, and locating your Load balancer resource.

    Note: The Azure portal sometimes displays the names of resources with incorrect capitalization. Always use the Azure CLI to retrieve the correctly capitalized name of a resource. To see the list of resources, run az network lb list.

  4. In the Resource Config pane of the PAS tile, click the icon next to the Router job name to expand the row, showing a LOAD BALANCERS field and INTERNET CONNECTED checkbox underneath.

  5. Enter the name of your external ALB in the field under LOAD BALANCERS.

  6. Retrieve the name of your Diego SSH Load Balancer by navigating to the Azure portal, clicking All resources, and locating your Load balancer resource.

  7. In the Resource Config pane of the PAS tile, expand the row for the job that handles SSH requests. This depends on the PAS release you are using:

    • PAS: Click the icon next to Diego Brain.
    • Small Footprint PAS: Click the icon next to Control.
  8. Enter the name of the Diego SSH Load Balancer in the field under LOAD BALANCERS.

  9. Ensure that the INTERNET CONNECTED checkboxes are deselected for all jobs.

  10. Scale the number of instances as appropriate for your deployment.

    Note: For a high availability deployment of Pivotal Platform on Azure, Pivotal recommends scaling the number of each PAS job to a minimum of three instances. Using three or more instances for each job creates a sufficient number of availability sets and fault domains for your deployment. For more information, see Azure Reference Architecture.

GCP

GCP with Terraform

To configure the Gorouter to use GCP load balancers:

  1. Select Resource Config.

  2. Click the icon next to the Router job name to expand the row, showing a LOAD BALANCERS field and INTERNET CONNECTED checkbox underneath.

  3. In the the LOAD BALANCERS field, enter a comma-separated list consisting of the values of ws_router_pool and http_lb_backend_name from your Terraform output. For example, tcp:-cf-ws,http:-httpslb. These are the names of the TCP WebSockets and HTTP(S) load balancers for your deployment.

    Note: Do not add a space between key and value pairs in the LOAD BALANCER field, or it fails.

    Note: If you are using HAProxy in your deployment, then enter the above load balancer values in the LOAD BALANCERS field of the HAProxy row instead of the Router row. For a high-availability configuration, scale up the HAProxy job to more than one instance.

  4. If you enabled TCP routing in the Networking pane of the PAS tile, add the value of tcp_router_pool from your Terraform output, prepended with tcp:, to the LOAD BALANCERS column of the TCP Router row. For example, tcp:-cf-tcp.

  5. Expand the row for the job that handles SSH requests. This depends on the PAS release you are using:

    • PAS: Click the icon next to Diego Brain.
    • Small Footprint PAS: Click the icon next to Control.
  6. Under LOAD BALANCERS for the SSH job, enter the value of ssh_router_pool from your Terraform output, prepended with tcp:. For example, tcp:PCF-ssh-proxy.

  7. Verify that the Internet Connected checkbox for every job is enabled. The Terraform templates do not provision a Network Address Translation (NAT) box for Internet connectivity to your VMs, so they are provided with ephemeral public IP addresses to allow the jobs to reach the Internet.

    Note: If you want to provision a Network Address Translation (NAT) box to provide Internet connectivity to your VMs instead of providing them with public IP addresses, disable the Internet Connected checkboxes. For more information about using NAT in GCP, see VPC network overview in the GCP documentation.

  8. Click Save.

GCP Paved Manually

To configure the Gorouter to use GCP load balancers:

  1. Navigate to the GCP Console and select Load balancing.

    Config lb

    You should see the SSH load balancer, the HTTP(S) load balancer, the TCP WebSockets load balancer, and the TCP router that you created in Preparing to Deploy Ops Manager on GCP Manually.

  2. Record the name of your SSH load balancer and your TCP WebSockets load balancer, PCF-wss-logs and PCF-ssh-proxy.

  3. Click your HTTP(S) load balancer, PCF-global-. Pcf router

  4. Under Backend services, record the name of the back end service of the HTTP(S) load balancer, PCF-http-lb-backend.

  5. In the PAS tile, select Resource Config.

  6. Click the icon next to the Router job name to expand the row, showing a LOAD BALANCERS field and INTERNET CONNECTED checkbox underneath.

  7. In the the LOAD BALANCERS field, enter a comma-separated list consisting of the name of your TCP WebSockets load balancer and the name of your HTTP(S) load balancer back end with the protocol prepended. For example, tcp:PCF-wss-logs,http:PCF-http-lb-backend.

    Note: Do not add a space between key and value pairs in the LOAD BALANCER field, or it fails.

    Note: If you are using HAProxy in your deployment, enter the above load balancer values in the LOAD BALANCERS field of the HAProxy row instead of the Router row. For a high-availability configuration, scale up the HAProxy job to more than one instance.

  8. If you enabled TCP routing in the Networking pane of the PAS tile and set up the TCP load balancer in GCP, add the name of your TCP load balancer, prepended with tcp:, to the LOAD BALANCERS column of the TCP Router row. For example, tcp:-tcp-router.

  9. Expand the row for the job that handles SSH requests. This depends on the PAS release you are using:

    • PAS: Click the icon next to Diego Brain.
    • Small Footprint PAS: Click the icon next to Control.
  10. Under LOAD BALANCERS for the SSH job, enter the value of ssh_router_pool from your Terraform output, prepended with tcp:. For example, tcp:PCF-ssh-proxy.

  11. Verify that the Internet Connected checkbox for every job is disabled. When preparing your GCP environment, you provisioned a Network Address Translation (NAT) box to provide Internet connectivity to your VMs instead of providing them with public IP addresses to allow the jobs to reach the Internet.

    Note: If you want to provision a Network Address Translation (NAT) box to provide Internet connectivity to your VMs instead of providing them with public IP addresses, disable the Internet Connected checkboxes. For more information about using NAT in GCP, see VPC network overview in the GCP documentation.

  12. Click Save.

OpenStack

Unless you are using your own load balancer, you must provide HAProxy with public IP addresses to use as floating IP addresses. This allows the HAProxy route traffic into the OpenStack private subnet.

To provide HAProxy with public IP addresses:

  1. Select Resource Config.

  2. Click the icon next to the HAProxy job name to expand the row, showing the FLOATING IPS field underneath.

  3. Enter one or more IP addresses under FLOATING IPS.

  4. (Optional) If you enabled TCP routing, expand the TCP Router row and enter the IP addresses of your TCP routers under FLOATING IPS.

  5. Click Save.

vSphere with NSX-T or NSX-V

In the Resource Config pane of the PAS tile, you configure vSphere NSX-T or NSX-V load balancing and security group membership for PAS jobs.

To configure load balancing and security group membership for PAS jobs:

  1. Select Resource Config.

  2. For each PAS job that you want to load-balance, click the down-arrow to reveal its configuration fields. The external-facing job instance groups in PAS are:

    • Diego Brain in PAS, or Control in Small Footprint PAS, for SSH traffic
    • Router for web traffic
    • TCP Router for TCP traffic
  3. Enter a load balancer configuration for each job:

    • NSX-T:
      1. In the vSphere NSX Manager > Advanced Networks & Security > Groups pane, define an NSGroup to include VMs running each load-balanced job, such as pas-ssh-group, pas-tcp-group, and pas-web-group.
      2. In the NS Groups field in Resource Config, enter a comma-separated list of the NSGroups you defined.
      3. In the Logical Load Balancer: Enter a JSON-formatted structure defining a list of server_pools as pairs of name and port definitions.
    • NSX-V:
      1. Security Groups: Enter a comma-separated list of Security Groups defined in NSX-V to include each load-balanced job, such as pas-ssh-group, pas-tcp-group, and pas-web-group.
      2. Edge Load Balancers: Enter a JSON-formatted structure listing edge load balancers, each defined by edge_name, pool_name, security_group, port, and monitor_port definitions.
  4. Click Save.