Deploying TAS for VMs to AVS

Page last updated:

This topic describes how to install VMware Tanzu Application Service for VMs (TAS for VMs) on Azure VMware Solution (AVS) with NSX-T internal networking, using the VMware NSX-T Container Plug-in.

For more information about AVS, see the Azure VMware Solution documentation.

Overview

AVS provides you with private clouds that contain vSphere clusters built from dedicated bare-metal Azure infrastructure. AVS is a VMware-validated solution with ongoing validation and testing of enhancements and upgrades. Microsoft manages and maintains private cloud infrastructure and software.

All provisioned private clouds have vCenter Server, vSAN, vSphere, and NSX-T.

Deploying TAS for VMs to AVS is similar to traditional TAS for VMs deployments on vSphere, but there are a few minor differences due to the way that AVS deploys vSphere components.

To install NSX-T to run with TAS for VMs:

  1. Configure NSX-T to Integrate with TAS for VMs

  2. Enable NSX-T Mode in the BOSH Director

  3. Configure TAS for VMs for External Container Networking

  4. Install and Configure the NSX-T Container Plugin

Prerequisites

Before deploying TAS for VMs with NSX-T networking, you must have:

Configure NSX-T to Integrate with TAS for VMs

To set up NSX-T to integrate with TAS for VMs, complete these procedures:

Regenerate NSX-T Management TLS Certificate with a Valid SAN

The AVS-deployed NSX-T Manager includes a self-signed TLS certificate with an invalid Subject Alternative Name (SAN). This causes issues when connecting from Ops Manager and the BOSH Director, so you must create a new certificate using the NSX-T Manager’s fully qualified domain name (FQDN) or IP address as the new SAN.

For instructions, see Generate and Register the NSX-T Management TLS Certificate and Private Key in the Tanzu Kubernetes Grid Integrated Edition (TKGI) documentation.

Configure Logical Switches

To configure logical switches:

  1. In vSphere, create logical network switches to correspond to the networks that your foundation uses.
    1. Log in to the NSX-T Manager Dashboard.
    2. Go to Advanced Networking & Security.
    3. Go to the Switching pane.
    4. For each of these networks…
      • Infrastructure (BOSH and Ops Manager, defined in the Assign AZs and Networks pane of the BOSH Director tile)
      • Deployment (TAS for VMs, defined in the Assign AZs and Networks pane of the TAS for VMs tile)
      • Services and Dynamic Services (marketplace services and on-demand services, also defined in the TAS for VMs tile)
      • Isolation Segment (optional, defined in the Assign AZs and Networks pane of the Isolation Segment tile) …do the following:
        1. Click +ADD.
        2. Enter a name for the logical switch (such as TAS for VMs-Infrastructure, TAS for VMs-Deployment).
        3. Click ADD. NSX-T

Configure Routers

To configure routers:

  1. Create T1 routers for TAS for VMs, to connect from the T0 router. For each TAS for VMs network, Infrastructure, Deployment, and so on, create a T1 router as follows:
  2. In the NSX-T Manager UI, navigate to Advanced Networking & Security > Routing > Routers.

    1. Click +ADD > Tier-1 Router.
    2. Configure the router. Include the Edge Cluster and Edge Cluster Members; they are required to enable the Load Balancer. The Infrastructure network router configuration might look like the following diagram: NSX-T
  3. Create T1 router downlink ports for TAS for VMs. For each T1 router you created, add a New Router Port as follows, to allow traffic in and out:

    1. In the NSX-T Manager UI, select the T1 router.
    2. In Configuration > Router Ports, click +ADD to add a new router port.
    3. For Logical Switch, enter the name of the logical switch you defined for the network in Add New Logical Switch, above.
    4. For IP Address, use the first IP of the appropriate subnet. In this example, 192.168.1.0/24 is set aside for Infrastructure (Ops Manager and BOSH Director), and 192.168.2.0/24 for the Deployment, so 192.168.1.1 and 192.168.2.1 are used respectively. NSX-T
  4. Advertise the routes of the T1 routers to the T0 router, so the T0 router can correctly route incoming requests based on their destination IP address:

    1. Select your T1 Router and navigate to Routing > Route Advertisement.
    2. Under Edit Route Advertisement Configuration, enable route advertisement by setting Status to Enabled.
    3. Set Advertise All Connected Routes to Yes.
    4. Set Advertise All LB VIP Routes to Yes (necessary if Load Balancing service is configured). NSX-T
  5. Allocate an IP block for TAS for VMs orgs.

    1. From the NSX-T Manager, navigate to Advanced Networking & Security > Networking > IPAM and click +ADD.
    2. Enter a name (for example, TAS for VMs-container-ip-block). This IP block name is also used in the VMware NSX-T tile in the NCP section under IP Blocks of Container Networks.
    3. Enter a description, such as Subnets are allocated from this pool to each newly-created org.
    4. Enter a CIDR to allocate an address block large enough to accommodate all TAS for VMs apps. A /14 CIDR is large enough for ~1,000 Orgs with ~250 apps each. If you are planning such a large foundation, see VMware NSX-T TAS for VMs limits in the VMware documentation. NSX-T

Configure Load Balancer

To configure a load balancer:

  1. Create Active Health Monitors (health checks) for use by the virtual server later on:

    1. In the NSX-T Manager UI, navigate to Advanced Networking & Security > Networking > Load Balancing > Monitors > Active Health Monitors.
    2. Create the health monitor for web load balancing:
    3. Click +ADD.
    4. Enter Monitor Properties:
      • Name: pas-web-monitor
      • Health Check Protocol: LbHttpMonitor
      • Monitoring Port: 8080
    5. Click Next.
    6. Enter Health Check Parameters:
      • HTTP Method: GET
      • HTTP Request URL: /health
      • HTTP Response Code: 200
    7. Click Finish.
    8. Create the health monitor for TCP load balancing:
    9. Click +ADD.
    10. Enter Monitor Properties:
      • Name: pas-tcp-monitor
      • Health Check Protocol: LbHttpMonitor
      • Monitoring Port: 80
    11. Click Next.
    12. Enter Health Check Parameters:
      • HTTP Method: GET
      • HTTP Request URL: /health
      • HTTP Response Code: 200
    13. Create the health monitor for SSH load balancing:
    14. Click +ADD.
    15. Enter Monitor Properties:
      • Name: pas-ssh-monitor
      • Health Check Protocol: LbTcpMonitor
      • Monitoring Port: 2222
    16. Click Next, then click Finish.
  2. Create server pools (collections of VMs which handle traffic) for use by the virtual server:

    1. In the NSX-T Manager UI, navigate to Advanced Networking & Security > Networking > Load Balancing > Server Pools.
    2. Create the server pool for web load balancing:
    3. Click +ADD to add a new pool.
    4. Enter General Properties:
      • Name: pas-web-pool
    5. Click Next.
    6. Enter SNAT Translation:
      • Translation Mode: IP List
      • Enter a range of available IPs for SNAT translation. By default, ports from 4000 to 64000 are for all configured SNAT IP addresses. Allocate enough IPs to handle your traffic load. Without enough IPs, the SNAT port is exhausted.
    7. Click Next.
    8. Enter Pool Members:
      • Membership Type: Static
    9. Click Next.
    10. Enter Health Monitors:
      • Active Health Monitor: pas-web-monitor
    11. Click Finish.
    12. Create the server pool for TCP load balancing:
    13. Click +ADD to add new pool.
    14. Enter General Properties:
      • Name: pas-tcp-pool
    15. Click Next.
    16. Enter SNAT Translation:
      • Translation Mode: Transparent
    17. Click Next.
    18. Enter Pool Members:
      • Membership Type: Static
    19. Click Next.
    20. Enter Health Monitors:
      • Active Health Monitor: pas-tcp-monitor
    21. Click Finish.
    22. Create the server pool for SSH load balancing:
    23. Click +ADD to add new pool.
    24. Enter General Properties:
      • Name: pas-ssh-pool
    25. Click Next.
    26. Enter SNAT Translation:
      • Translation Mode: Transparent
    27. Click Next.
    28. Enter Pool Members:
      • Membership Type: Static
    29. Click Next.
    30. Enter Health Monitors:
      • Active Health Monitor: pas-ssh-monitor
    31. Click Finish.
  3. Create virtual servers:

    1. In the NSX-T Manager UI, navigate to Advanced Networking & Security > Networking > Load Balancing > Virtual Servers.
    2. Create the virtual server which forwards unencrypted web (HTTP) traffic to the foundation:

      Note: Foundations requiring end-to-end encryption should not enable the virtual server on port 80, or, if enabled, should configure it to redirect traffic to the encrypted port (443).

    3. Click +ADD.
    4. Enter General Properties:
      • Name: pas-web-vs
      • Application Type: Layer 4 (TCP)
      • Application Profile: nsx-default-lb-fast-tcp-profile
    5. Click Next.
    6. Enter Virtual Server Identifiers:
      • IP Address: use the address of the DNS record of *.system.YOUR-SYSTEM-DOMAIN.com
      • Port: 80,443
    7. Enter Server Pool and Rules:
      • Default Server Pool: pas-web-pool
    8. Click Next several times, then click Finish.
    9. Create the virtual server which forwards traffic to apps with custom ports to the foundation:
    10. Click +ADD to add a new virtual server.
    11. Enter General Properties:
      • Name: pas-tcp-vs
      • Application Type: Layer 4 (TCP)
      • Application Profile: nsx-default-lb-fast-tcp-profile
    12. Click Next.
    13. Enter Virtual Server Identifiers:
      • IP Address: use the address of the DNS record of tcp.apps.YOUR-SYSTEM-DOMAIN.com
      • Port: use the same ports as configured in the TAS for VMs Tile > Networking > TCP Routing Ports, e.g. 1024-1123,5900
    14. Click Next.
    15. Enter Server Pool and Rules:
      • Default Server Pool: pas-tcp-pool
    16. Click Next, then click Finish.
    17. Create the virtual server which forwards SSH traffic to the foundation:
    18. Click +ADD to add a new virtual server.
    19. Enter General Properties:
      • Name: pas-ssh-vs
      • Application Type: Layer 4 (TCP)
      • Application Profile: nsx-default-lb-fast-tcp-profile
    20. Click Next.
    21. Enter Virtual Server Identifiers:
      • IP Address: use the address of the DNS record of ssh.system.YOUR-SYSTEM-DOMAIN.com
      • Port: 2222
    22. Click Next.
    23. Enter Server Pool and Rules:
      • Default Server Pool: pas-ssh-pool
    24. Click Next, then click Finish.
  4. Create the load balancer:

    1. In the NSX-T Manager UI, navigate to Advanced Networking & Security > Networking > Load Balancing > Load Balancers.
    2. Click +ADD.
    3. Enter the fields:
      • Name: pas-lb
      • Load Balancer Size: Choose Small unless you have a larger Foundation
    4. Click OK.
    5. Select pas-lb.
    6. Click Actions > Attach to a Virtual Server, and then select pas-web-vs. Repeat this procedure for the Virtual Servers pas-tcp-vs and pas-ssh-vs.
    7. Click Action > Attach to a Logical Router, and then select T1-Router-TAS for VMs-Deployment.

Enable NSX-T Mode in the BOSH Director

To enable NSX-T mode in the BOSH Director:

  1. From the Ops Manager Installation Dashboard, open the BOSH Director tile.

  2. In the vCenter Configs pane, click the pencil icon for the vCenter Config you want to edit.

  3. Select NSX Networking below.

  4. Configure BOSH Director authentication to the NSX Manager by following the NSX Networking instructions in the Step 2: Configure vCenter section of Configuring BOSH Director on vSphere.

Configure TAS for VMs for External Container Networking

To configure TAS for VMs for external container networking:

  1. If you have not already done so, download the TAS for VMs tile from VMware Tanzu Network and import it to the Installation Dashboard. For instructions, see Adding and Importing Products. Ops Manager Installation Dashboard with NSX-T tile

  2. Configure TAS for VMs, following the directions in Configuring TAS for VMs. When you configure Networking, select External under Container networking interface plugin.
    NSX-T

  3. Update the server pool membership for the NSX-T load balancers:

    1. Open the BOSH Director for vSphere tile > Resource Config pane.
    2. Click the arrow next to each job to reveal the NSX-T CONFIGURATION column. See Step 10: Resource Config Pane in Configuring BOSH Director on vSphere.
    3. Under Logical Load Balancer, enter a JSON-formatted structure to defining a list of server_pools a VM extension for each of the three server pools: pas-web-pool, pas-tcp-pool, and pas-ssh-pool.

Install and Configure the NSX-T Container Plug-In

To install and configure the NSX-T Container Plug-In tile:

  1. If you have not already done so, download the VMware NSX-T Container Plug-in tile from VMware Tanzu Network and import it to the Installation Dashboard. For instructions, see Adding and Importing Products. Ops Manager Installation Dashboard with NSX-T tile

  2. Click the VMware NSX-T tile to open its Settings tab, and configure the NSX Manager pane as follows:

    • NSX Manager Address: The NSX-T Manager host address or IP address.
    • Use Client Certificates or Username/Password: Configure this setting as follows:
      1. If you are using VMware Workspace ONE Access, formerly called VMware Identity Manager (vIDM), then select Client Certificate Authentication.
      2. Otherwise, select Basic Authentication with Username and Password and enter NSX Manager Admin Username and Admin Password credentials in the fields underneath.
    • NSX Manager CA Cert: Obtain this certificate from NSX-T Manager as follows:
      1. ssh into NSX-T Manager using the admin account that you created when you deployed NSX-T Manager.
      2. From the NSX-T Manager command line, run get certificate api to retrieve the certificate. NSX-T tile config: NSX-T Manager
  3. Open and configure the NCP (NSX-T Container Plugin) pane as follows:

    • TAS for VMs Foundation Name: If unsure, use TAS for VMs. If multiple foundations co-exist on the same NSX-T Manager, choose a unique string, such as TAS for VMs-beta. NCP creates artifacts, such as T1 routers and prefixes their names with this string for easy identification.
    • Overlay Transport Zone: A uniquely identifying string for the Transport Zone that you chose when you created logical switches for each network. This can be the name of the transport zone if no other zones in NSX-T share the same name, or else the UUID for the transport zone.
    • Tier-0 Router: A uniquely identifying string for the T0 router. This can be the tag string that you gave the router in NSX-T Manager if no other T0 routers in NSX-T share the same name, or else the UUID for the router.
    • Subnet Prefix of Container Networks: Subnet mask to set the address range size for apps in a single org. Defaults to 24. This number must be higher than the mask for all TAS for VMs orgs in the NSX-T Manager New IP Block pane, to define each org’s fraction of the total TAS for VMs address space.
    • Enable SNAT for Container Network: Enable this checkbox. NSX-T tile config: NCP
  4. In the NSX Node Agent pane, enable the Enable Debug Level of Logging for NSX Node Agent checkbox. NSX-T tile config: NSX-T Node Agent

  5. Click Save and return to the Installation Dashboard.

  6. After you have configured both the TAS for VMs tile and the VMware NSX-T tile, click Review Pending Changes, then Apply Changes to deploy TAS for VMs with NSX-T networking.