Networking for On-Demand Services
This section describes networking considerations for Pivotal Cloud Cache.
Service Network Requirement
When you deploy Pivotal Application Service (PAS), you must create a statically defined network to host the component VMs that make up the infrastructure. Components, such as Cloud Controller and UAA, run on this infrastructure network.
On-demand services might require you to host them on a separate network from the default network. You can also deploy on-demand services on a separate service networks to meet your own security requirements.
PAS supports dynamic networking. Operators can use dynamic networking with asynchronous service provisioning to define dynamically-provisioned service networks. For more information, see Default Network and Service Network below.
On-demand services are enabled by default on all networks. Operators can optionally create separate networks to host services in BOSH Director. Operators can select which network hosts on-demand service instances when they configure the tile for that service.
Default Network and Service Network
On-demand Pivotal Cloud Cache services use BOSH to dynamically deploy VMs and create single-tenant service instances in a dedicated network. On-demand services use the dynamically-provisioned service network to host single-tenant worker VMs. These worker VMs run as service instances within development spaces.
This on-demand architecture has the following advantages:
- Developers can provision IaaS resources for their services instances when the instances are created. This removes the need for operators to pre-provision a fixed amount of IaaS resources when they deploy the service broker.
- Service instances run on a dedicated VM and do not share VMs with unrelated processes. This removes the “noisy neighbor” problem, where an app monopolizes resources on a shared cluster.
- Single-tenant services can support regulatory compliances where sensitive data must be separated across different machines.
An on-demand service separates operations between the default network and the service network. Shared service components, such as executive controllers and databases, Cloud Controller, UAA, and other on-demand components, run on the default network. Worker pools deployed to specific spaces run on the service network.
The diagram below shows worker VMs in an on-demand service instance running on a separate services network, while other components run on the default network.
View a larger version of this image
### Required Networking Rules for On-Demand ServicesBefore deploying a service tile that uses the on-demand service broker (ODB), you must create networking rules to enable components to communicate with ODB. For instructions for creating networking rules, see the documentation for your IaaS.
The following table lists key components and their responsibilities in the on-demand architecture.
| Key Components | Component Responsibilities |
|---|---|
| BOSH Director | Creates and updates service instances as instructed by ODB. |
| BOSH Agent | Adds an agent on every VM that it deploys. The agent listens for instructions from the BOSH Director and executes those instructions. The agent receives job specifications from the BOSH Director and uses them to assign a role or job to the VM. |
| BOSH UAA | Issues OAuth2 tokens for clients to use when they act on behalf of BOSH users. |
| Pivotal Application Service | Contains the apps that consume services. |
| ODB | Instructs BOSH to create and update services. Connects to services to create bindings. |
| Deployed service instance | Runs the given service. For example, a deployed Pivotal Cloud Cache service instance runs the Pivotal Cloud Cache service. |
Regardless of the specific network layout, the operator must ensure network
rules are set up so that connections are open as described in the table below.
| This component… | Must communicate with… | Default TCP Port | Communication direction(s) | Notes |
|---|---|---|---|---|
| PCC cluster members | PCC cluster members | 49152-65535 | Two-way | Inclusive range. PCC servers and locators communicate with each other using UDP and TCP. |
| PCC Service Instance 1 | PCC Service Instance 2 | 5000-5499 | Two-way | Inclusive range. Gateway receivers and gateway senders communicate across WAN-separated service instances. Each PCC service instance uses GemFire defaults for the gateway receiver ports. |
| ODB |
|
|
One-way | The BOSH Director and BOSH UAA default ports are not configurable. The CredHub default port is configurable. |
| ODB | PAS | 8443 | One-way | The default port is not configurable. |
| Errand VMs |
|
|
One-way | The default port is not configurable. |
| BOSH Agent | BOSH Director | 4222 | Two-way | The BOSH Agent runs on every VM in the system, including the BOSH Director VM.
The BOSH Agent initiates the connection with the BOSH Director. The default port is not configurable. |
| Deployed apps on PAS | Deployed service instances |
|
Two-way | These port numbers are not configurable. |
| PAS |
|
8080 | One-way | PAS communicates with service instances because the Gorouter proxies gfsh requests to GemFire clusters. |
| PCC | PCC | 1053 | Two-way | Allows DNS resolution for clusters communicating across a WAN-connected system. |
PCC Instances Across WAN
PCC service instances running within distinct PCF foundations may communicate with each other across a WAN. In a topology such as this, the members within one service instance use their own private address space, as defined in RFC1918.
A VPN may be used to connect the private network spaces that lay across the WAN. The steps required to enable the connectivity by VPN are dependent on the IaaS provider(s).
The private address space for each service instance’s network must be configured with non-overlapping CIDR blocks. Configure the network prior to creating service instances. Locate directions for creating a network on the appropriate IAAS provider within the section titled Architecture and Installation Overview.
Open port 1053 to allow DNS resolution of other WAN-connected clusters.