AWS Reference Architecture

Page last updated:

This topic describes a reference architecture for Ops Manager, including VMware Tanzu Application Service for VMs (TAS for VMs) and VMware Tanzu Kubernetes Grid Integrated Edition (TKGI), on Amazon Web Services (AWS). It builds on the common base architectures described in Platform Architecture and Planning Overview.

For general requirements for running Ops Manager and specific requirements for running Ops Manager on AWS, see Ops Manager on AWS Requirements.

TAS for VMs

This section describes a reference architecture for a TAS for VMs installation on AWS.

Diagram

The diagram below illustrates the reference architecture for TAS for VMs on AWS.

This diagram illustrates TAS for VMs on AWS. For specific details about the components and communication represented by the diagram, see the base and AWS reference architectures.

View a larger version of this diagram.

Networking

These sections provide guidance about networking resources.

DNS

You can use AWS Route 53 for DNS resolution to host your TAS for VMs domains.

Load Balancing

AWS offer Application Load Balancers (ALBs) and Network Load Balancers (NLBs). VMware recommends using NLBs because of the simplicity and security of leaving all encrypted traffic in place until it reaches the Gorouters. If you require Amazon Certificate Management or more complex Layer 7 routing rules, you can front or replace the NLB with an ALB.

Networks, Subnets, and IP Spacing Planning

When planning networks, subnets, and IP spacing, some things to consider are:

  • In AWS, each availability zone (AZ) requires its own subnet of which the first five IPs are reserved.

  • Your organization might not have the IP space necessary to deploy TAS for VMs in a consistent manner. To alleviate this, VMware recommends using the 100.64 address space to deploy a NAT instance and router so that the Ops Manager deployment can use few IP addresses.

  • If you want the front end of TAS for VMs to be accessible from your corporate network or the services running on TAS for VMs to be able to access corporate resources, you must either provide routable IPs to your VPC or use NAT.

  • If you want a VPC that is only public-facing, no special consideration is necessary for the use of IPs.

  • To understand the action required depending on the type of traffic you want to allow for your Ops Manager deployment, see the table below:

    If you want… then…
    Internet ingress Create the load balancers with Elastic IPs.
    Internet egress Create the NAT Gateway toward the Internet.
    Corporate ingress Create load balancers on your corporate subnet.
    Corporate egress (for accessing corporate resources) Create the NAT instance on your corporate subnet or if business requirements dictate, make the entire VPC corporate-routable.

  • If you plan to install VMware Tanzu Service Broker for AWS, you might want to create another set of networks in addition to those outlined in the base reference architecture. This set of networks would be used for RDS and other AWS-managed instances.

Alternative Network Layouts

These options are available for choosing your network topology:

Type of Traffic Options
Internet ingress
  • Allow Internet ingress by associating load balancers with elastic IPs.
  • Do not allow Internet ingress.
Internet egress
  • Allow TAS for VMs to access the Internet through a NAT gateway.
  • Do not allow TAS for VMs to access the Internet. Use only a corporate network.
Peering
  • Isolated. This is normally Internet-only facing.
  • Peered with corporate or other VPCs through Transit Gateway.
  • Peered directly with other VPCs and corporate connections.
Corporate peering
  • SNAT for corporate resources. This requires three corporate IPs.
  • Routable IP addresses for VPC using /20 or a larger CIDR block necessary.

RDS

Use a single RDS instance for BOSH and TAS for VMs. This instance requires several databases. For more information, see Step 3: Director Config Page in Configuring BOSH Director on AWS and External System Database Configuration in Configuring TAS for VMs.

Blobstore Storage Accounts

Ops Manager requires a bucket for the BOSH blobstore.

TAS for VMs requires these buckets:

  • Buildpacks
  • Droplets
  • Packages
  • Resources

These buckets require an associated role for read/write access.

Identity Management

For identity management, use Instance Profiles whenever possible. For example, the AWS Config Page of the BOSH Director tile provides a Use AWS Instance Profile option. For more information, see Step 2: AWS Config Page in Configuring BOSH Director on AWS.

VMware Tanzu Kubernetes Grid Integrated Edition

These sections describe a reference architecture for a VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) installation on AWS.

Diagram

The diagram below illustrates the reference architecture for TKGI on AWS.

This diagram illustrates TKGI on AWS. For specific details about the components and communication represented by the diagram, see the base and AWS reference architectures.

View a larger version of this diagram.

Networking

These sections provide guidance about networking resources.

DNS

You can use AWS Route 53 for DNS resolution to host your TKGI domains.

Load Balancing

You can configure a front end load balancer to enable public access to the TKGI API VM.

For each TKGI cluster, VMware recommends creating a load balancer for the corresponding master node or nodes. For more information, see Creating and Configuring an AWS Load Balancer for Enterprise TKGI Clusters.

For deployed workloads, Kubernetes can automatically create load balancers when applying the service definition to the app instances. For more information, see Deploying and Exposing Basic Workloads.

Networks, Subnets, and IP Spacing Planning

When planning networks, subnets, and IP spacing, some things to consider are:

  • In AWS, each AZ requires its own subnet of which the first five IPs are reserved.

  • If you want the front end of TKGI to be accessible from your corporate network or the services running on TKGI to be able to access corporate resources, you must either provide routable IPs to your VPC or use NAT.

  • Create the TKGI subnets that host the Kubernetes cluster VMs with enough IP address capacity to accommodate the number of VMs and clusters you expect to deploy. The reference architecture diagram above shows a /24 CIDR for the TKGI subnets, which allows for about 250 VMs per AZ. If you expect a larger number of clusters for your organization, consider defining larger subnets, such as /20, to make room for future growth.

Alternative Network Layouts

These options are available for choosing your network topology:

Type of Traffic Options
Internet Ingress
  • Allow Internet ingress by associating load balancers with elastic IPs.
  • Do not allow Internet ingress.
Internet Egress
  • Allow nodes and pods to access the Internet through a NAT gateway.
  • Do not allow nodes or pods to access the Internet. Use only a corporate network.
Peering
  • Isolated. This is normally internet-only facing.
  • Peered with corporate or other VPCs through Transit Gateway.
  • Peered directly with other VPCs and corporate connections.
Corporate Peering
  • SNAT for corporate resources. This requires three corporate IPs.
  • Routable IP Addresses for VPC using /20 or a larger CIDR block necessary.

RDS

Use a single RDS instance for BOSH. This instance requires a database. For more information, see Step 3: Director Config Page in Configuring BOSH Director on AWS.

Identity Management

To create and manage Kubernetes master and worker nodes, use Instance Profiles. For more information, see Kubernetes Cloud Provider in Installing Enterprise TKGI on AWS.

For information about installing the required AWS constructs for TKGI, see Installing Ops Manager on AWS.