AWS Reference Architecture

Page last updated:

This guide presents a reference architecture for Pivotal Cloud Foundry (PCF), including Pivotal Application Service (PAS) and Enterprise Pivotal Container Service (PKS), on Amazon Web Services (AWS). It builds on the common base architectures described in Platform Architecture and Planning Overview.

See PCF on AWS Requirements for general requirements for running PCF and specific requirements for running PCF on AWS.

PAS

This section describes a reference architecture for a PAS installation on AWS.

Diagram

The following diagram illustrates the reference architecture for PAS on AWS.

This diagram illustrates PAS on AWS. For specific details about the components and communication represented by the diagram, see the base and AWS reference architectures.

View a larger version of this diagram.

Networking

The following sections provide guidance about networking resources.

DNS

You can use AWS Route 53 for DNS resolution to host your PAS domains.

Load Balancing

AWS offer Application Load Balancers (ALBs) and Network Load Balancers (NLBs). Pivotal recommends using NLBs because of the simplicity and security of leaving all encrypted traffic in place until it reaches the Gorouters. If you require Amazon Certificate Management or more complex layer 7 routing rules, you can front or replace the NLB with an ALB.

Networks, Subnets, and IP Spacing Planning

The following list provides considerations related to networks, subnets, and IP spacing planning.

  • In AWS, each AZ requires its own subnet of which the first 5 IPs are reserved.
  • Your organization might not have the IP space necessary to deploy PAS in a consistent manner. To alleviate this, Pivotal recommends using the 100.64 address space to deploy a NAT instance and router so that the PCF deployment can use few IP addresses.
  • If you want the front end of PAS to be accessible from your corporate network or the services running on PAS to be able to access corporate resources, you must either provide routable IPs to your VPC or use NAT.
  • If you want a VPC that is only public-facing, no special consideration is necessary for the use of IPs.
  • See the following table to understand the action required depending on the type of traffic you want to allow for your PCF deployment.
    If you want… then…
    Internet ingress Create the load balancers with Elastic IPs.
    Internet egress Create the NAT Gateway towards the internet.
    Corporate Ingress Create load balancers on your corporate subnet.
    Corporate Egress (For accessing corporate resources) Create the NAT instance on your corporate subnet or if business requirements dictate, make the entire VPC corporate-routable.
  • If you plan to install PCF Service Broker for AWS, you might want to create another set of networks in addition to those outlined in the base reference architecture. This set of networks would be used for RDS and other AWS managed instances.

Alternative Network Layouts

The following options are available for choosing your network topology:

Type of Traffic Options
Internet Ingress
  • Allow internet ingress by associating load balancers with elastic IPs.
  • Do not allow internet ingress.
Internet Egress
  • Allow PAS to access the internet through a NAT gateway.
  • Do not allow PAS to access the internet. Use only a corporate network.
Peering
  • Isolated. This is normally internet-only facing.
  • Peered with corporate or other VPCs through Transit Gateway.
  • Peered directly with other VPCs and corporate connections.
Corporate Peering
  • SNAT for corporate resources. This requires 3 corporate IPs.
  • Routable IP Addresses for VPC using /20 or a larger CIDR block necessary.

RDS

Use a single RDS instance for BOSH and PAS. This instance requires several databases. See the following sections of the installation documentation for more information:

Blobstore Storage Accounts

Ops Manager requires a bucket for the BOSH blobstore.

PAS requires the following buckets:

  • Buildpacks
  • Droplets
  • Packages
  • Resources

These buckets require an associated role for read/write access.

Identity Management

For identity management, use Instance Profiles whenever possible. For example, the AWS Config Page of the BOSH Director tile provides a Use AWS Instance Profile option. See Step 2: AWS Config Page.

PKS

This section describes a reference architecture for a PKS installation on AWS.

Diagram

The following diagram illustrates the reference architecture for PKS on AWS.

This diagram illustrates PKS on AWS. For specific details about the components and communication represented by the diagram, see the base and AWS reference architectures.

View a larger version of this diagram.

Networking

The following sections provide guidance about networking resources.

DNS

You can use AWS Route 53 for DNS resolution to host your PKS domains.

Load Balancing

You can configure a front end load balancer to enable public access to the PKS API VM.

For each PKS cluster, creating a load balancer is recommended for the corresponding master node or nodes. For more information, see Creating and Configuring an AWS Load Balancer for Enterprise PKS Clusters.

For deployed workloads, Kubernetes can automatically create load balancers when applying the service definition to the app instances. For more information, see Deploying and Exposing Basic Workloads.

Networks, Subnets, and IP Spacing Planning

The following list provides considerations related to networks, subnets, and IP spacing planning.

  • In AWS, each AZ requires its own subnet of which the first 5 IPs are reserved.
  • If you want the front end of PKS to be accessible from your corporate network or the services running on PKS to be able to access corporate resources, you must either provide routable IPs to your VPC or use NAT.
  • Create the PKS subnets that host the Kubernetes cluster VMs with enough IP address capacity to accommodate the number of VMs and clusters you expect to deploy. The reference architecture diagram above shows a /24 CIDR for the PKS subnets, which allows for about 250 VMs per AZ. If you expect a larger number of clusters for your organization, consider defining larger subnets, such as /20, to make room for future growth.

Alternative Network Layouts

The following options are available for choosing your network topology:

Type of Traffic Options
Internet Ingress
  • Allow internet ingress by associating load balancers with elastic IPs.
  • Do not allow internet ingress.
Internet Egress
  • Allow nodes and pods to access the internet through a NAT gateway.
  • Do not allow nodes or pods to access the internet. Use only a corporate network.
Peering
  • Isolated. This is normally internet-only facing.
  • Peered with corporate or other VPCs through Transit Gateway.
  • Peered directly with other VPCs and corporate connections.
Corporate Peering
  • SNAT for corporate resources. This requires 3 corporate IPs.
  • Routable IP Addresses for VPC using /20 or a larger CIDR block necessary.

RDS

Use a single RDS instance for BOSH. This instance requires a database. For more information, see Step 3: Director Config Page in Configuring BOSH Director on AWS.

Identity Management

To create and manage Kubernetes master and worker nodes, use Instance Profiles. For more information, see Kubernetes Cloud Provider in Installing Enterprise PKS on AWS.

For information about installing the required AWS constructs for PKS, see Installing Pivotal Cloud Foundry on AWS.