Enabling TCP Routing

Route Ports
Pre-Deployment Steps
Post-Deployment Steps
- Configure CF with Your TCP Domain
- Configure a Quota for TCP Routes
Create a TCP Route
Router Groups
- Modify your TCP ports

Page last updated:

This topic describes enabling TCP Routing for your Cloud Foundry (CF) deployment. This feature enables developers to run applications that serve requests on non-HTTP TCP protocols. You can use TCP Routing to comply with regulatory requirements that require your organization to terminate the TLS as close to your apps as possible so that packets are not decrypted before reaching the application level.

Route Ports

The diagram below shows the layers of network address translation that occur in Cloud Foundry in support of TCP Routing. The descriptions step through an example work flow that covers route ports, backend ports, and app ports.

Route Ports

A developer creates a TCP route for their application based on a TCP domain and a route port, and maps this route to one or more applications. See the Creating Routes topic for more information.
Clients make requests to the route. DNS resolves the domain name to the load balancer.
The load balancer listens on the port and forwards requests for the domain to the TCP routers. The load balancer must listen on a range of ports to support multiple TCP route creation. Additionally, Cloud Foundry must be configured with this range, so that the platform knows what ports can be reserved when developers create TCP routes.
The TCP router can be dynamically configured to listen on the port when the route is mapped to an application. The domain the request was originally sent to is no longer relevant to the routing of the request to the application. The TCP router keeps a dynamically updated record of the backends for each route port. The backends represent instances of an application mapped to the route. The TCP Router chooses a backend using a round-robin load balancing algorithm for each new TCP connection from a client. As the TCP Router is protocol agnostic, it does not recognize individual requests, only TCP connections. All client requests transit the same connection to the selected backend until the client or backend closes the connection. Each subsequent connection triggers the selection of a backend.
Because containers each have their own private network, the TCP router does not have direct access to application containers. When a container is created for an application instance, a port on the Cell VM is randomly chosen and iptables are configured to forward requests for this port to the internal interface on the container. The TCP router then receives a mapping of the route port to the Cell IP and port.
The Diego Cell only routes requests to port 8080, the App Port, on the container internal interface. The App Port is the port on which applications must listen.

Pre-Deployment Steps

Before enabling TCP Routing, you must complete the following steps to set up networking requirements.

Choose a domain name from which your developers will create TCP routes for their applications. For example, create a domain which is similar to your app domain but prefixed by the TCP subdomain: tcp.APPS-DOMAIN.com
Configure DNS to resolve this domain name to the IP address of a highly-available load balancer that will forward traffic for the domain to the TCP routers. For more information, view the Domains topic. If you are operating an environment that does not require high-availability, configure DNS to resolve the TCP domain name you have chosen directly to a single instance of the TCP Router.
(Optional) Choose IP addresses for the TCP routers and configure your load balancer to forward requests for the domain you chose in the step above to these addresses. Skip this step if you have configured DNS to resolve the TCP domain name to an instance of the TCP Router. Review the Enable TCP Routing steps in the Deploying PAS topic for your IaaS to configure your IP addresses for your PCF deployment: AWS, OpenStack, or vSphere.
(Optional) Decide on the number of TCP routes you want to support. For each TCP route, you must reserve a port. Configure your load balancer to forward the range of ports to the TCP routers. Skip this step if you have configured DNS to resolve the TCP domain name to an instance of the TCP Router.
Review the “Enable TCP Routing” steps in the Deploying Pivotal Application Service topic for your IaaS to configure your ports for your PCF deployment: AWS, OpenStack, or vSphere.

Post-Deployment Steps

In the following steps you use the Cloud Foundry Command Line Interface (cf CLI) to add the TCP shared domain and configure organization quotas to grant developers the ability to create TCP routes. This requires an admin user account.

Configure CF with Your TCP Domain

After deployment, you must configure Cloud Foundry with the domain that you configured in the Pre-Deployment step above so that developers can create TCP routes from it.

Run cf router-groups. You should see default-tcp as a response.
Run cf create-shared-domain to create a shared domain and associate it with the router group.
```
$ cf create-shared-domain tcp.APPS-DOMAIN.com --router-group default-tcp
```
Run cf domains. Verify that next to your TCP domain, TCP appears under type.

Configure a Quota for TCP Routes

Since TCP route ports are a limited resource in some environments, quotas are configured to allow creation of zero TCP routes by default. After you deploy Cloud Foundry, you can increase the maximum number of TCP routes for all organizations or for particular organizations and spaces. Because you reserve a route port for each TCP route, the quota for this resource is managed with the cf CLI command option --reserved-route-ports. See the Creating and Modifying Quota Plans topic for more information.

If you have a default quota that applies to all organizations, you can update it to configure the number of route ports that can be reserved by each organization.

$ cf update-quota QUOTA —reserved-route-ports X

To create a new organization quota that can be allocated to particular organizations, provide the following required quota attributes in addition to the number of reserved route ports:

$ cf create-quota QUOTA —reserved-route-ports X

You can also create a quota that governs the number of route ports that can be created in a particular space.

$ cf create-space-quota QUOTA —reserved-route-ports X

Create a TCP Route

For instructions about creating a TCP Route, see the Create a TCP Route with a Port topic.

Router Groups

In Post-Deployment Steps, we describe that in order to create a domain from which to create TCP routes, it must be associated with the TCP Router Group. A router group is a cluster of identically configured routers. Router Groups were introduced as mechanism to support reservation of the same port for multiple TCP routes, thus increasing the capacity for TCP routes. However, only one router group is currently supported. In the Pre-Deployment Steps. we describe how an admin user can configure the port range available for TCP routes in preparation for deployment.

Modify your TCP ports

After deployment, you can modify the range of ports available for TCP routes using cf curl commands, as demonstrated with the commands below. These commands require an admin user with the routing.router_groups.read and routing.router_groups.write scopes.

In a terminal window, run cf curl /routing/v1/router_groups to view the reservable_ports:

 
$ cf curl /routing/v1/router_groups
[
  {
    "guid": "9d1c1da9-ed63-45e8-45ee-256f8579455c",
    "name": "default-tcp",
    "type": "tcp",
    "reservable_ports": "60000-60098"
  }
]

Modify the reservable_ports:

$ cf curl /routing/v1/router_groups/f7392031-a488-4890-8835-c4a038a3bded -X PUT -d '{"reservable_ports":"1024-1199"}'

Create a pull request or raise an issue on the source for this page in GitHub