GCP Reference Architecture

Page last updated:

This topic presents two reference architectures for installing Pivotal Cloud Foundry (PCF) on Google Cloud Platform (GCP): on a shared virtual private cloud (VPC) and on a single-project VPC. This topic also outlines multiple networking variants for VPC deployment. The architectures are validated for production-grade PCF deployments using multiple AZs.

For general requirements for running PCF and specific requirements for running PCF on GCP, see PCF on GCP Requirements

PCF Reference Architectures

A PCF reference architecture describes a proven approach for deploying PCF on a specific IaaS, such as GCP.

A PCF reference architecture must meet the following requirements:

  • Be secure
  • Be publicly-accessible
  • Include common PCF-managed services such as MySQL, RabbitMQ, and Spring Cloud Services
  • Be able to host at least 100 app instances

Pivotal provides reference architectures to help you determine the best configuration for your PCF deployment.

Shared vs Single-Project VPCs

A shared VPC installation is harder to configure than a PCF deployment on a single-project VPC, because the required account privileges and resource allocations are more granular and complex. But the shared VPC architecture allows network assets to be centrally located, which simplifies auditing and security. Pivotal recommends the shared VPC model for:

  • Deployments with deep auditing and security requirements
  • When networks hosting the foundation need to connect back to an internal network via VPN or interconnect

A single-project VPC lets the platform architect give PCF full access to the VPC and its resources, which makes configuration easier. Pivotal recommends single-project VPC architecture for:

  • Standalone deployments that do not connect to an internal network
  • Test and experimental deployments, and for projects which do not belong to an organization

Shared VPC GCP Reference Architecture

The following diagram provides an overview of a reference architecture deployment of PCF on a shared VPC on GCP. This architecture requires an organization on the VPC that contains a host project and a service project.

For more information about shared VPCs on GCP, see Shared VPC Overview in the Google Cloud documentation.

For more information about how this architecture divides resources between projects, see Host / Service Architecture.

Gcp overview shared vpc

View a larger version of this diagram.

NAT Network Topology

To expose a minimal number of public IP addresses, set up your NAT as shown in this diagram.

Gcp nat flow

View a larger version of this diagram.

Cloud Interconnect

To speed communication between data centers, use Google Cloud Interconnect as shown in this diagram.

Gcp cloud interconnect

View a larger version of this diagram.

Host and Service Architecture

GCP allocates resources using a hierarchy that centers around projects. To create a VPC, architects define a host project that allocates network resources for the VPC, such as address space and firewall rules. Then they can define one or more service projects to run within the VPC, which share the network resources allocated by the host project and include their own non-network resources, such as VMs and storage buckets.

To install PCF in a shared VPC on GCP, you create a host project for the VPC and a service project dedicated to running PCF.

Host Project Resources

The host project centrally manages the following shared VPC network resources for PCF:

  • Infra subnet (Ops Manager and BOSH Director)
  • PAS subnet
  • Services subnet
  • Isolation Segments
  • Firewall rules
  • NAT instances and gateway
  • VPN/interconnect
  • Routes, e.g. egress internet through NAT or egress on-premises through VPN

Service Project Resources

The PCF service project manages the following resources:

  • Google Cloud Compute instances (VMs)

    • BOSH
    • Ops Manager
    • VMs deployed by BOSH, such as PCF and service components
  • Google Cloud Storage buckets, for blobstore

    • BOSH Director
    • Resources
    • Buildpacks
    • Droplets
    • Packages
  • Service account and a service account key for PCF to access the storage buckets

  • A service account for PCF

  • Load Balancers

  • Google Cloud SQL instances, if using external databases

Single-Project VPC Base GCP Reference Architecture

The following diagram provides an overview of a reference architecture deployment of PCF on a single-project VPC on GCP.

Gcp overview single vpc

View a larger version of this diagram.

Base Reference Architecture Components

The following table lists the components that are part of a reference architecture deployment with three availability zones.

Component Reference Architecture Notes
Domains & DNS CF Domain Zones and routes in use by the reference architecture include: domains for *.apps and *.system (required), a route for Ops Manager (required), a route for doppler (required), a route for Loggregator (required), a route for ssh access to app containers (optional) and a route for TCP routing to apps (optional). Reference architecture uses GCP Cloud DNS as the DNS provider.
Ops Manager Deployed on the infrastructure subnet and accessible by FQDN or through an optional jumpbox.
BOSH Director Deployed on the infrastructure subnet.
Gorouters Accessed through the HTTP and TCP WebSockets load balancers. Deployed on the Pivotal Application Service (PAS) subnet, one job per availability zone.
Diego Brains Required. However, the SSH container access functionality is optional and enabled through the SSH Proxy load balancer. Deployed on the PAS subnet, one job per availability zone.
TCP Routers Optional feature for TCP routing. Deployed on the PAS subnet, one job per availability zone.
CF Database Reference architecture uses GCP Cloud SQL rather than internal databases. Configure your database with a strong password and limit access only to components that require database access.
CF Blob Storage and Buckets For buildpacks, droplets, packages and resources. Reference architecture uses Google Cloud Storage rather than internal file storage.
Services Deployed on the PCF managed services subnet. Each service is deployed to each availability zone.

Alternative GCP Network Layouts for PCF

This section describes the possible network layouts for PCF deployments as covered by the reference architecture of PCF on GCP.

At a high level, there are currently two possible ways of granting public Internet access to PCF as described by the reference architecture:

However, if you require NAT, you may refer to the following section.

NAT-Based Solution

This diagram illustrates the case where you want to expose only a minimal number of public IP addresses.

Gcp net topology base

View a larger version of this diagram.

Public IP Addresses Solution

If you prefer not to use a NAT solution, you can configure PCF on GCP to assign public IP addresses for all components. This type of deployment may be more performant since most of the network traffic between PCF components are routed through the front end load balancer and the Gorouter.

Network Objects

The following table lists the network objects expected for each type of reference architecture deployment with three availability zones (assumes you are using NAT).

Network Object Notes Minimum Number: NAT-Based Minimum Number: Public IP Addresses
External IPs For a NAT solution, use global IP address for apps and system access, and Ops Manager or an optional jumpbox. 2 30+
NAT One NAT per availability zone. 3 0
Network One per deployment. GCP Network objects allow multiple subnets with multiple CIDRs, so a typical deployment of PCF will likely only ever require one GCP Network object. 1 1
Subnets Separate subnets for infrastructure (Ops Manager, BOSH Director, Jumpbox), PAS, and services. Using separate subnets allows you to configure different firewall rules due to your needs. 3 3
Routes Routes are typically created by GCP dynamically when subnets are created, but you may need to create additional routes to force outbound communication to dedicated SNAT nodes. These objects are required to deploy PCF without public IP addresses. 3+ 3
Firewall Rules GCP firewall rules are bound to a Network object and can be created to use IP ranges, subnets, or instance tags to match for source & destination fields in a rule. The preferred method use in the reference architecture deployment is instance tags. 6+ 6+
Load balancers Used to handle requests to Gorouters and infrastructure components. GCP uses two or more load balancers. The HTTP load balancer and TCP WebSockets load balancer are both required. The TCP Router load balancer used for TCP routing feature and the SSH load balancer that allows SSH access to Diego apps are both optional. The HTTP load balancer provides SSL termination. 2+ 2+
Jumpbox Optional. Provides a way of accessing different network components. For example, you can configure it with your own permissions and then set it up to access to Pivotal Network to download tiles. Using a jumpbox is particularly useful in IaaSes where Ops Manager does not have a public IP address. In these cases, you can SSH into Ops Manager or any other component through the jumpbox. (1) (1)

Network Communication in GCP Deployments

This section provides more background on the reasons behind certain network configuration decisions, specifically for the Gorouter.

Load Balancer to Gorouter Communications and TLS Termination

In a PCF on GCP deployment, the Gorouter receives two types of traffic:

  1. Unencrypted HTTP traffic on port 80 that is decrypted by the HTTP(S) load balancer.

  2. Encrypted secure web socket traffic on port 443 that is passed through the TCP WebSockets load balancer.

TLS is terminated for HTTPS on the HTTP load balancer and is terminated for WebSockets (WSS) traffic on the Gorouter.

PCF deployments on GCP use two load balancers to handle Gorouter traffic because HTTP load balancers currently do not support WebSockets.


GCP routers do not respond to ICMP. Pivotal recommends disabling ICMP checks in your BOSH Director network configuration. For more information, see the Step 5: Create Networks Page section of the Configuring BOSH Director on GCP Manually topic.