Understanding Container-to-Container Networking
This topic provides an overview of how Container-to-Container Networking works.
The Container-to-Container Networking feature enables app instances to communicate with each other directly. Container-to-Container Networking is always enabled in PAS. For more information about how to configure Container-to-Container Networking, see the Administering Container-to-Container Networking topic.
Container-to-Container Networking integrates with Garden-runC in a Diego deployment. The Container-to-Container Networking BOSH release includes several core components, as well as swappable components.
To understand the components and how they work, see the diagram and tables below. The diagram highlights Pivotal Application Service components in blue and green. The diagram also highlights swappable components in red.
The Container-to-Container Networking BOSH release includes the following core components:
|Cloud Foundry Command Line Interface (CF CLI) plugin||A plugin that you download to control network access policies between apps.|
|Policy Server||A central management node that does the following:
|Garden External Networker||
A Garden-runC add-on deployed to every Diego cell that does the following:
The Container-to-Container Networking BOSH release includes the following swappable components:
|Silk CNI plugin
||A plugin that provides IP address management and network connectivity to app instances as follows:
|VXLAN Policy Agent
||Enforces network policy for traffic between apps as follows:
The diagram below illustrates how app instances communicate in a deployment with Container-to-Container Networking enabled. In this example, the operator creates two policies to regulate the flow of traffic between App A, App B, and App C.
- Allow traffic from App A to App B
- Allow traffic from App A to App C
If traffic and its direction is not explicitly allowed, it is denied. For example, App B cannot send traffic to App C.
Container-to-Container Networking uses an overlay network to manage communication between app instances.
Overlay networks are not externally routable, and traffic sent between containers does not exit the overlay. You can use the same overlay network range for different Cloud Foundry deployments in your environment.
The overlay network range defaults to
You can modify the default to any RFC 1918 range that meets the following requirements:
- The range is not used by services that app containers access.
- The range is not used by the underlying Cloud Foundry infrastructure.
All Diego cells in your Cloud Foundry deployment share this overlay network. By default, each cell is allocated a /24 range that supports 254 containers per cell, one container for each of the usable IP addresses,
.254. To modify the number of Diego cells your overlay network supports, see Overlay Network in Configuring Container-to-Container Networking.
Cloud Foundry container networking is currently supported only on Linux.
WARNING: The overlay network IP address range must not conflict with any other IP addresses in the network. If a conflict exists, Diego cells cannot reach any endpoint that has a conflicting IP address.
Note: Traffic to app containers from the Gorouter or from app containers to external services uses cell IP addresses and NAT, not the overlay network.
Enabling Container-to-Container Networking for your deployment allows you to create policies for communication between app instances. The Container-to-Container Networking feature also provides a unique IP address to each app container and provides direct IP reachability between app instances.
The policies you create specify a source app, destination app, protocol, and port so that app instances can communicate directly without going through the Gorouter, a load balancer, or a firewall. Container-to-Container Networking supports UDP and TCP, and you can configure policies for multiple ports. These policies apply immediately without having to restart the app.
Additionally, policies use and and track the GUIDs of the apps. The policies continue to work when apps redeploy, or if they crash and Diego places them in a new container. Pushing a brand new app requires a new policy, but not updates to an existing app because an an app always retains its GUID.
The Pivotal Application Service platform supports DNS-based service discovery that lets apps find each others’ internal addresses. For example, a front end app instance can use the service discovery mechanism to establish communications with a back end app instance. See the Developer Guide for how to set up and use app service discovery.
Container-to-Container app service discovery does not provide client-side load balancing or circuit-breaking, and it does not apply to
cf marketplace services or require application binding. It just lets apps publish service endpoints to each other, unbrokered and unmediated.
The BOSH release that contains the Container-to-Container Networking feature is composed of a pluggable network stack. Advanced users or third-party vendors can integrate a different network stack. For more information about third-party plugins, see the Container-to-Container Networking BOSH release documentation.
Both application security groups (ASGs) and Container-to-Container Networking policies affect traffic from app instances. The following table highlights differences between ASGs and Container-to-Container Networking policies.
|ASGs||Container-to-Container Networking Policies|
|Policy granularity||From a space to an IP address range||From a source app to a destination app|
|Scope||For a space, org, or deployment||For app to app only|
|Traffic direction||Outbound control||Policies apply for incoming packets from other app instances|
|Source app||Is not known||Is identified because of direct addressability|
|Policies take affect||After app restart||Immediately|