LATEST VERSION: 1.10 - CHANGELOG
Pivotal Cloud Foundry v1.10

Restricting App Access to Internal PCF Components

This topic describes how to secure the component VMs of your Pivotal Cloud Foundry (PCF) deployment from being accessed by apps.

Introduction

See the following list to understand the concepts for this topic:

  • How PCF determines where apps can send traffic:
    • PCF uses Application Security Groups (ASGs), which are network policy rules specifying protocols, ports, and IP ranges that apply to outbound network connections initiated from apps. See Understanding ASGs.
  • Why you must create new rules for outbound app traffic:
    • PCF installs with a default ASG that allows apps running on your deployment to send traffic to almost any IP address. This means apps are not blocked from initiating connections to most network destinations unless an administrator takes action to update the ASGs with a more restrictive policy.
  • How you can set up new rules:
    • To help secure your component VMs against apps while ensuring your apps can access the services they need, follow the procedure below, which includes these steps:
      Step Description
      1 Determine Your Network Layout:
      The procedure for securing your deployment with ASGs varies depending on your network layout, which you can determine using Ops Manager.
      2 Ensure Access for PCF System Apps:
      Bind the default ASG to the system org so that PCF system apps can continue accessing the system components they need after you remove the deployment-wide default ASG in Step 4.
      3 Create New ASGs:
      Block apps from sending traffic to system components, but allow them to send traffic to the services they need.
      4 Remove the Default ASG:
      After you create and bind new ASGs, you no longer need the deployment-wide default ASG bindings that allow apps to send traffic to any IP.
      5 Restart your Apps:
      To apply the ASG changes, you must restart all of the apps in your deployment.
  • When to set up new rules:
    • Pivotal recommends that you complete this procedure directly after installing PCF, prior to developers pushing apps to the platform. If you complete the procedure after apps have been pushed to the platform, you must restart all the apps in your deployment.

Prerequisites

The procedure below requires that you have the latest release of ASG Creator from the Cloud Foundry incubator repository on Github. See About the ASG Creator Tool.

Procedure

Follow these steps to apply ASGs that prevent apps running on your deployment from accessing internal PCF components.

Step 1: Determine Your Network Layout

The procedure for securing your deployment with ASGs varies depending on your network layout, which you can determine by following these steps:

  1. Log in to Ops Manager.

  2. For each tile, click Assign AZs and Networks and record the selected Network that the tile is installed on.

  3. Based on the information you gathered, determine which of the following network layouts you have:

    Layout Name Layout Description
    One Network
    • One network for Ops Manager and the Ops Manager Director, Elastic Runtime, and services.

    Note: You cannot secure your deployment with ASGs if you have this network layout. Because PCF dynamically allocates IPs, they cannot be easily excluded in the case of a single network.


    Two Networks
    • One network for Ops Manager and the Ops Manager Director.
    • One network for Elastic Runtime and Services.
    Three Networks
    • One network for Ops Manager and the Ops Manager Director.
    • One network for Elastic Runtime.
    • One network for all services.
    Three or More Networks
    • One network for Ops Manager and the Ops Manager Director.
    • One network for Elastic Runtime.
    • One network for each service.

  4. If your network layout includes two or more networks, continue Step 2: Ensure Access for PCF System Apps.

Step 2: Ensure Access for PCF System Apps

Follow these steps to apply the default ASG to the system org. This provides network access to PCF system apps without restrictions, which enables them to continue functioning properly after you perform Step 4: Remove the Deployment-wide Default ASG Binding.

  1. Bind the default ASG to the staging set in the system org:

    $ cf bind-staging-security-group default_security_group system

  2. Bind the default ASG to the running set in the system org:

    $ cf bind-running-security-group default_security_group system

Step 3: Create New ASGs

Follow these steps to create ASGs that block apps from accessing PCF components and create any additional ASGs that allow apps to access the services they require.

Part A: Record CIDRs

Gather the CIDRs for each network in your deployment:

  1. From the Ops Manager Director tile, click Create Networks within the Settings tab.

  2. In the Networks section, expand each network in your deployment by clicking its name.

  3. Record the CIDR for each network.

Part B: Create and Bind ASGs that Block Network Access

Create ASGs that block apps from sending traffic to the networks that host Ops Manager, Elastic Runtime, and (optional) any services installed.

  1. Create a config.yml containing the appropriate content for your network layout and replace the indicated values with the CIDRs you gathered:

    • Two Network Layout:

      excluded_networks:
      - YOUR-OPS-MANAGER-CIDR
      - YOUR-ELASTIC-RUNTIME-AND-SERVICES-CIDR
      
    • Three Network Layout:

      Note: If you only want to secure the Ops Manager and Elastic Runtime components, you can optionally exclude the services CIDR.

        excluded_networks:
        - YOUR-OPS-MANAGER-CIDR
        - YOUR-ELASTIC-RUNTIME-CIDR
        - YOUR-SERVICES-CIDR
      
    • Three or More Network Layout:

      Note: If you only want to secure the Ops Manager and Elastic Runtime components, you can optionally exclude the services CIDRs.

      excluded_networks:
      - YOUR-OPS-MANAGER-CIDR
      - YOUR-ELASTIC-RUNTIME-CIDR
      - YOUR-SERVICE-CIDR-1
      - YOUR-SERVICE-CIDR-2
      etc...
      
  2. Run the following command to create a json that contains ASG rules, using your config.yml as input:

    $ asg-creator create --config config.yml --output OUTPUT-FILE-NAME.json
    Replace OUTPUT-FILE-NAME with a name of your choice.

  3. Create an ASG by running the following command:

    1. Replace SECURITY-GROUP-NAME with a name of your choice.
    2. Replace OUTPUT-FILE-NAME with the name of the generated file from the previous step.
      $ create-security-group SECURITY-GROUP-NAME OUTPUT-FILE-NAME.json
  4. Bind the ASG to the default staging set:

    $ cf bind-staging-security-group SECURITY-GROUP-NAME

  5. Bind the ASG to the default running set:

    $ cf bind-running-security-group SECURITY-GROUP-NAME

Part C: Create and Bind ASGs for Service Access

Note: This part is only necessary if you blocked apps from a network that hosts services in the previous part. If you did not block apps from a network that hosts services, proceed to Step 4: Remove the Default ASG.

WARNING: In the two network layout, Elastic Runtime and services share the same network. This means that each time you create an ASG that allows apps to access a new port/protocol within the network, you further expose the Elastic Runtime component VMs. This is a limitation of a two network layout. For guidance on network topology, see Reference Architectures.

Now that you have created ASGs to secure the Ops Man, Elastic Runtime, and service components, work with developers to create additional ASGs that give apps access to the services they need.

For example, in any space where apps need to access the MySQL for PCF service, follow the steps in Creating Application Security Groups for MySQL.

For more information on creating and binding ASGs, see the following:

Step 4: Remove the Default ASG

Now that you have bound new ASGs to determine outbound traffic rules, you no longer need the default ASG bindings that allow apps to send traffic to any IP.

  1. Unbind the default ASG from the staging set:

    $ cf unbind-staging-security-group default_security_group

  2. Unbind the default ASG from the running set:

    $ cf unbind-running-security-group default_security_group

Step 5: Restart your Apps

To apply the ASG changes, you must restart all of the apps in your deployment. To mitigate app downtime during the restart, Pivotal recommends a blue-green deployment strategy.

Notes: You do not need to restart the apps in the system org.

  1. Work with developers to restart a few of their apps individually and test that they still work correctly with the new ASGs in place. If an app does not work as expected, you likely must create another ASG that allows the app to send traffic to a service it requires.

    Note: To quickly roll back to the original overly-permissive state, you can re-bind the default_security_group ASG to the default-staging and default-running sets. You must then restart your apps to re-apply the original ASGs.

  2. Restart the rest of the apps running on your deployment. Optionally, you can use the app-restarter cf CLI plugin to restart all apps in a particular space, org, or deployment.

Create a pull request or raise an issue on the source for this page in GitHub