Pivotal Cloud Foundry v1.7

Configuring Elastic Runtime for vSphere, vCloud, and vCloud Air

Page last updated:

Note: Pivotal Cloud Foundry (PCF) for vCloud Air and vCloud Director is deprecated and availability is restricted to existing customers. Contact Support for more information.

This topic describes how to configure the Pivotal Elastic Runtime components that you need to run Pivotal Cloud Foundry (PCF) for VMware vSphere, vCloud, or vCloud Air.

Note: If you plan to install the IPsec add-on, you must do so before installing any other tiles. Pivotal recommends installing IPsec immediately after Ops Manager, and before installing the Elastic Runtime tile.

Step 1: Add Elastic Runtime to Ops Manager

  1. Navigate to the Pivotal Network and click the Pivotal Cloud Foundry banner to access the PCF product page. Use the drop-down menu to select an Elastic Runtime release.

  2. From the Available Products view, click Import a Product.

    Import prod

  3. Select the Elastic Runtime .pivotal file that you downloaded from Pivotal Network and click Open. After the import completes, Elastic Runtime appears in the Available Products view.

  4. In the Available Products view, hover over Elastic Runtime and click Add.

    Add products

  5. Click the Elastic Runtime tile in the Installation Dashboard.

    Er tile

Step 2: Assign Availability Zones and Networks

  1. Select Assign AZs and Networks. These are the Availability Zones that you create when configuring Ops Manager Director.

  2. (vSphere Only) Select an Availability Zone under Place singleton jobs. Ops Manager runs any job with a single instance in this Availability Zone.

  3. (vSphere Only) Select one or more Availability Zones under Balance other jobs. Ops Manager balances instances of jobs with more than one instance across the Availability Zones that you specify.

  4. From the Network drop-down box, choose the network on which you want to run Elastic Runtime. Er az

  5. Click Save.

    Note: When you save this form, a verification error displays because the PCF security group blocks ICMP. You can ignore this error.

    Er network error

Step 3: Configure Domains

  1. Select Domains.

    Er17 config domains

  2. Enter the system and application domains.

    • The System Domain defines your target when you push apps to Elastic Runtime.
    • The Apps Domain defines where Elastic Runtime should serve your apps.

    Note: Pivotal recommends that you use the same domain name but different subdomain names for your system and app domains. Doing so allows you to use a single wildcard certificate for the domain while preventing apps from creating routes that overlap with system routes. For example, name your system domain and your apps domain

    Note: You configured wildcard DNS records for these domains in an earlier step.

  3. Click Save.

Step 4: Configure Networking

  1. Select Networking.

  2. (Optional) The values you enter in the Router IPs and HAProxy IPs fields depends on whether you are using your own load balancer or the HAProxy load balancer. Find your load balancer type in the table below to determine how to complete these fields.

    Note: If you choose to assign specific IP addresses in either the Router IPs or HAProxy IPs field, ensure that these IPs are in your subnet.

    Your own load balancer Enter the IP address or addresses for PCF that you registered with your load balancer. Refer to the Using Your Own Load Balancer topic for help using your own load balancer with PCF. Leave this field blank.
    HAProxy load balancer Leave this field blank. Enter at least one HAProxy IP address. Point your DNS to this address.

    For more information, refer to the Configuring PCF SSL Termination topic. For help understanding the Elastic Runtime architecture, refer to the Architecture topic.

  3. For Router IPs, enter one or more static IP addresses for your routers. These must be in the subnet that you configured in the Ops Manager Create Networks section. If you are using your own load balancer, configure it to point to these IPs. If you are using the Elastic Load Balancer (ELB), add the name of your ELB in the router column of the Resource Config section.

  4. For HAProxy IPs, enter one or more IP addresses for HAProxy. You must point your DNS to this IP unless you are using your own load balancer, and HAProxy’s IP must be in your subnet.

  5. Under Configure the point-of-entry to this environment, choose one of the following:

    • External Load Balancer with Encryption: Select this option if your deployment uses an external load balancer that can forward encrypted traffic to the Elastic Runtime Router, or for a development environment that does not require load balancing. Complete the fields for the Router SSL Termination Certificate and Private Key and Router SSL Ciphers.
    • External Load Balancer without Encryption: Select this option if your deployment uses an external load balancer that cannot forward encrypted traffic to the Elastic Runtime Router, or for a development environment that does not require load balancing.
    • HAProxy: Select this option to use HAProxy as your first point of entry. Complete the fields for SSL Certificate and Private Key, and HAProxy SSL Ciphers. Select Disable HTTP traffic to HAProxy if you want the HAProxy to only allow HTTPS traffic.

      Note: For details about different SSL termination point options, which correspond to different points-of-entry for Elastic Runtime, see the Providing a Certificate for your SSL Termination Point topic.

  6. If you are not using SSL encryption or if you are using self-signed certificates, select Disable SSL certificate verification for this environment. Selecting this checkbox also disables SSL verification for route services.

  7. Select the Disable insecure cookies on the Router checkbox to turn on the secure flag for cookies generated by the router. Networking config

  8. In the Choose whether or not to enable route services section, choose either Enable route services or Disable route services. Route services are a class of marketplace services that perform filtering or content transformation on application requests and responses. See the Route Services topic for details.

  9. (Optional) The Loggregator Port defaults to 443 if left blank. Enter a new value to override the default.

  10. (Optional) Use the Applications Subnet field if you need to avoid address collision with a third-party service on the same subnet as your apps. Enter a CIDR subnet mask specifying the range of available IP addresses assigned to your app containers. The IP range must be different from the network used by the system VMs.

  11. (Optional) You can change the value in the Applications Network Maximum Transmission Unit (MTU) field. Pivotal recommends setting the MTU value for your application network to 1454. Some configurations, such as networks that use GRE tunnels, may require a smaller MTU value.

  12. (Optional) Increase the number of seconds in the Router Timeout to Backends field to accommodate larger uploads over connections with high latency.

  13. (Optional) Increase the value of Load Balancer Unhealthy Threshold to specify the amount of time, in seconds, that the router continues to accept connections before shutting down. During this period, healthchecks may report the router as unhealthy, which causes load balancers to failover to other routers. Set this value to an amount greater than or equal to the maximum time it takes your load balancer to consider a router instance unhealthy, given contiguous failed healthchecks.

    Networking config bottom

  14. Click Save.

Step 5: Configure Application Containers

  1. Select Application Containers.

    Er config app containers

  2. The Enable Custom Buildpacks checkbox governs the ability to pass a custom buildpack URL to the -b option of the cf push command. By default, this ability is enabled, letting developers use custom buildpacks when deploying apps. Disable this option by disabling the checkbox. For more information about custom buildpacks, refer to the buildpacks section of the PCF documentation.

  3. The Allow SSH access to app containers checkbox controls SSH access to application instances. Enable the checkbox to permit SSH access across your deployment, and disable it to prevent all SSH Access. See Application SSH Overview for information on SSH access permissions at the space and app scope.

  4. You can configure Elastic Runtime to run app instances in Docker containers by supplying their IP address range(s) in the Private Docker Insecure Registry Whitelist textbox. See the Using Docker Registries topic for more information.

  5. Select your preference for Docker Images Disk-Cleanup on Cell VMs. If you choose Clean up disk-space once threshold is reached, enter a Threshold of Disk-Used in megabytes.

  6. Click Save.

Step 6: Configure Application Developer Controls

  1. Select Application Developer Controls.

    Er17 config appdevctrl

  2. Enter your intended maximum file upload size.

  3. Enter your default RAM memory allocation per app.

  4. Enter your default total RAM memory (RAM) quota per Org. You can change this in the CLI.

  5. Enter your maximum and default disk quotas per app.

  6. Enter your default service instances quota per Org. You can change this in the CLI.

  7. Click Save.

Step 7: Review Application Security Group

Setting appropriate Application Security Groups is critical for a secure deployment. Type X in the box to acknowledge that once the Elastic Runtime deployment completes, you will review and set the appropriate application security groups. See Restricting App Access to Internal PCF Components for instructions.


Step 8: Configure Authentication and Enterprise SSO

  1. Select Authentication and Enterprise SSO.

    Er17 config authsso uaa

  2. To authenticate user sign-ons, your deployment can use one of three types of user database: the UAA server’s internal user store, an external SAML identity provider, and an external LDAP server.

    1. To use the internal UAA, select the Internal option and follow the instructions in Configuring UAA Password Policy to configure your password policy.
    2. To connect to an external identity provider via SAML, scroll down to select the SAML Identity Provider option and follow the instructions in Configuring PCF for SAML.
    3. To connect to an external LDAP server, scroll down to select the LDAP Server option and follow the instructions in Configuring LDAP.
  3. (Optional) In the Apps Manager Access Token Lifetime, Apps Manager Refresh Token Lifetime, Cloud Foundry CLI Access Token Lifetime, Cloud Foundry CLI Refresh Token Lifetime fields, you can change the lifetimes of tokens granted for Apps Manager and cf CLI login access and refresh. Most deployments use the defaults.

  4. (Optional) The Proxy IPs Regular Expression field contains a pipe delimited set of regular expressions that UAA considers to be reverse proxy IP addresses. UAA respects the x-forwarded-for and x-forwarded-proto headers coming from IP addresses that match to these regular expressions. To configure UAA to respond properly to Router or HAProxy requests coming from public IP address(es), append a regular expression or regular expressions to match the public IP address(es).

    Authsso uaa bottom

Step 9: Configure System Databases

Note: If you are performing an upgrade, do not modify your existing internal database configuration or you may lose data. You must migrate your existing data first before changing the configuration. See Upgrading Pivotal Cloud Foundry for additional upgrade information.

  1. Select Databases. Sys db
  2. If you want to use internal databases for your deployment, select Internal Databases - MySQL and Postgres or Internal Databases - MySQL. If you want to use external databases such as Amazon Web Services (AWS) RDS, select External Databases and complete the following steps:
    • For Hostname, enter the hostname of your database.
    • For TCP Port, enter the port of your database.
    • For each database username and database password field, enter a unique username and password.

      Note: Pivotal recommends that you use internal databases unless you require the functionality of AWS RDS.

  3. Click Save.

Step 10 (Optional) Configure Internal MySQL

Note: You only need to configure this section if you have selected Internal Databases - MySQL in the Databases section.

  1. Select Internal MySQL.

  2. In the MySQL Proxy IPs field, enter one or more comma-delimited IP addresses that are not in the reserved CIDR range of your network. If a MySQL node fails, these proxies re-route connections to a healthy node. See the Proxy section of the MySQL for PCF topic for more information.

    Mysql config

  3. For MySQL Service Hostname, enter an IP address or hostname for your load balancer. If a MySQL proxy fails, the load balancer re-routes connections to a healthy proxy. If you leave this field blank, components are configured with the IP address of the first proxy instance entered above.

  4. In the Replication canary time period field, leave the default of 30 seconds or modify the value based on the needs of your deployment. Lower numbers cause the canary to run more frequently, which means that the canary reacts more quickly to replication failure but adds load to the database.

  5. In the Replication canary read delay field, leave the default of 20 seconds or modify the value based on the needs of your deployment. This field configures how long the canary waits, in seconds, before verifying that data is replicating across each MySQL node. Clusters under heavy load can experience a small replication lag as write-sets are committed across the nodes.

  6. (Required): In the E-mail address field, enter the email address where the MySQL service sends alerts when the cluster experiences a replication issue or when a node is not allowed to auto-rejoin the cluster.

    Mysql replication canary

  7. Under Automated Backups Configuration, choose one of three options for MySQL backups:

    • Disable automatic backups of MySQL
    • Enable automated backups from MySQL to an S3 bucket or other S3-compatible file store saves your backups to an existing Amazon Web Services (AWS) or Ceph S3-compatible blobstore. Mysql backups s3 This option requires the following fields:
      • For S3 Bucket Name, enter the name of your S3 bucket. Do not include an s3:// prefix, a trailing /, or underscores. If the bucket does not already exist, it will be created automatically.
      • For Bucket Path, specify a folder within the bucket to hold your MySQL backups. Do not include a trailing /.
      • For AWS Access Key ID and AWS Secret Access Key, enter your AWS or Ceph credentials.
      • For Cron Schedule, enter a valid cron expression to schedule your automated backups. Cron uses your computer’s local time zone.
    • Enable automated backups from MySQL to a remote host via SCP saves your backups to a remote host using secure copy protocol (SCP). Mysql backups scp This option requires the following fields:
      • For Hostname, enter the name of your SCP host.
      • For Port, enter your SCP port. This should be the TCP port that your SCP host uses for SSH. The default port is 22.
      • For Username, enter your SSH username for the SCP host.
      • For Private key, paste in your SSH private key.
      • For Destination directory, enter the directory on the SCP host where you want to save backup files.
      • For Cron Schedule, enter a valid cron expression to schedule your automated backups. Cron uses your computer’s local time zone.
      • Enable Backup All Nodes to make unique backups from each instance of the MySQL server rather than just the first MySQL server instance.

        Note: If you choose to enable automated MySQL backups, set the number of instances for the Backup Prepare Node under the Resource Config section of the Elastic Runtime tile to 0.

  8. If you want to log audit events for internal MySQL, select Enable server activity logging under Server Activity Logging.

    1. For the Event types field, you can enter the events you want the MySQL service to log. By default, this field includes connect and query, which tracks who connects to the system and what queries are processed. For more information, see the Logging Events section of the MariaDB documentation.

      Audit Event Logging Config

  9. Click Save.

Step 11: Configure File Storage

  1. Select File Storage.

  2. To use the PCF internal filestore, select the Internal option and click Save.

  3. To use an external S3-compatible filestore for your Elastic Runtime file storage, select the External S3-Compatible Filestore option and complete the following procedure: External filestore config

    1. Enter the URL Endpoint for your filestore.
    2. Enter your Access Key and Secret Key.
    3. For S3 Signature Version and Region, use V4 Signature values.
    4. Enter a Buildpacks Bucket Name.
    5. Enter a Droplets Bucket Name.
    6. Enter a Packages Bucket Name.
    7. Enter a Resources Bucket Name.
    8. Click Save.

Note: For more information about AWS S3 Signatures, see the Authenticating Requests documentation.

Step 12: (Optional) Configure System Logging

If you are forwarding logging messages to an external Reliable Event Logging Protocol (RELP) server, complete the following steps:

  1. Select System Logging. Sys logging
  2. If you want to include security events in your log stream, select the Enable Cloud Controller security event logging checkbox. This logs all API requests, including the endpoint, user, source IP, and request result, in the Common Event Format (CEF).
  3. Enter the IP address of your syslog server in External Syslog Aggregator Hostname and its port in External Syslog Aggregator Port. The default port for a syslog server is 514.

    Note: The host must be reachable from the Elastic Runtime network, accept TCP connections, and use the RELP protocol. Ensure your syslog server listens on external interfaces.

  4. Select an External Syslog Network Protocol to use when forwarding logs.
  5. For the Syslog Drain Buffer Size, enter the number of messages the Doppler server can hold from Metron agents before the server starts to drop them. See the Loggregator Guide for Cloud Foundry Operators topic for more details.
  6. Click Save.

Step 13: (Optional) Customize Apps Manager

The Custom Branding and Apps Manager sections customize the appearance and functionality of Apps Manager. Refer to the Custom Branding Apps Manager topic for more information.

  1. Select Custom Branding. Custombranding Use this section to configure the text, colors, and images of the interface that developers see when they log in, create an account, reset their password, or use Apps Manager.

  2. Select Apps Manager. Config apps man Use this section to control access and role options for Apps Manager, as well as specify the headers and sidebar links that it displays.

    • Select Enable Internal User Store to use an internal user store in the PCF local UAA server. With the internal store enabled, PCF admins do not need to configure an external user store such as an LDAP / AD server.
    • Select Enable Non Admin Role Management to allow Org managers and Space managers to assign roles to users in the Orgs and Spaces that they manage.

    Both the Enable Internal User Store and the Enable Non Admin Role Management checkboxes must be selected if you want to invite new members to join an Org in Apps Manager. For more information about inviting users, see Inviting New Users. Ensure that you select the Push Apps Manager post-deploy errand on the Errands tab so that these edits are applied to your Apps Manager installation. Follow the steps in the “Configure Errands” below.

  3. Click Save to save your settings for each section.

Step 14: (Optional) Configure Email Notifications

Elastic Runtime uses SMTP to send invitations and confirmations to Apps Manager users. You must complete the Email Notifications page if you want to enable end-user self-registration.

  1. Select Email Notifications.


  2. Enter your reply-to and SMTP email information

  3. Verify your authentication requirements with your email administrator and use the SMTP Authentication Mechanism drop-down menu to select None, Plain, or CRAMMD5. If you have no SMTP authentication requirements, select None.

  4. Click Save.

Note: If you do not configure the SMTP settings using this form, the administrator must create orgs and users using the cf CLI tool. See Creating and Managing Users with the cf CLI for more information.

Step 15: (Optional) Add CCDB Restore Key

Perform this step if all of the following are true:

  • You deployed Elastic Runtime previously
  • You then stopped Elastic Runtime or it crashed
  • You are re-deploying Elastic Runtime with a backup of your Cloud Controller database
  1. Click Restore CCDB Encryption Key.

  2. Enter your Cloud Controller DB Encryption Key.

Er17 config ccdb closeup

See Backing Up Pivotal Cloud Foundry for more information.

Step 16: Configure Smoke Tests

The Smoke Tests errand runs basic functionality tests against your Elastic Runtime deployment after an installation or update. In this section, choose where to run smoke tests. In the Errands section, you can choose whether or not to run the Smoke Tests errand.

  1. Select Smoke Tests.

  2. If you have a shared apps domain, select On-demand org and space, which creates an ad-hoc org and space for running smoke tests and deletes them afterwards. Otherwise, select Specfied org and space and complete the fields to specify where you want to run smoke tests.

    Smoke test er config

  3. Click Save.

Step 17: (Optional) Enable Experimental Features

Use caution when enabling experimental features if you have other Pivotal Cloud Foundry service tiles installed in your Pivotal Cloud Foundry deployment. Not all of the services are guaranteed to work as expected with these features enabled.

Diego Cell Memory and Disk Overcommit

If your apps do not use the full allocation of disk space and memory set in the Resource Config tab, you might want use this feature. These fields control the amount to overcommit disk and memory resources to each Diego Cell VM.

For example, you might want to use the overcommit if your apps use a small amount of disk and memory capacity compared with the Resource Config settings for Diego Cell.

Note: Due to the risk of app failure and the deployment-specific nature of disk and memory use, Pivotal has no recommendation about how much, if any, memory or disk space to overcommit.

To enable this feature, follow these steps:

  1. Select Experimental Features.

  2. Enter the total desired amount of Diego cell memory value in the Cell Memory Capacity (MB) field. Refer to the Diego Cell row in the Resource Config tab for the current Cell memory capacity settings that this field overrides.

  3. Enter the total desired amount of Diego cell disk capacity value in the Cell Disk Capacity (MB) field. Refer to the Diego Cell row in the Resource Config tab for the current Cell disk capacity settings that this field overrides.

    Disk memory overcommit

    Note: Entries made to each of these two fields set the total amount of resources allocated, not the overage.

  4. Click Save.

CF CLI Connection Timeout

The CF CLI Connection Timeout field allows you to override the default 5 second timeout of the Cloud Foundry Command Line Interface (cf CLI) used within your PCF deployment. This timeout affects the cf command used to push Elastic Runtime errand apps such as Notifications, Autoscaler, Apps Manager and so on.

Set the value of this field to a higher value, in seconds, if you are experiencing domain name resolution timeouts when pushing errands in Elastic Runtime.

To modify your CF CLI connection timeout, perform the following steps:

  1. Select Experimental Features.

  2. Add a value, in seconds, to the CF CLI Connection Timeout field. Cf cli connection timeout

  3. Click Save.

Step 18: Configure Errands

Errands are scripts that Ops Manager runs to automate tasks. By default, Ops Manager runs the post-install errands listed below when you deploy Elastic Runtime. However, you can prevent a specific post-install errand from running by deselecting its checkbox on the Errands page.

Note: Several errands deploy apps that provide services for your deployment, such as Autoscaling and Notifications. Once one of these apps is running, deselecting the checkbox for the corresponding errand on a subsequent deployment does not stop the app.


  • Run Smoke Tests verifies that your deployment can do the following:

    • Push, scale, and delete apps
    • Create and delete orgs and spaces
  • Push Apps Manager deploys the Apps Manager, a dashboard for managing apps, services, orgs, users, and spaces. Until you deploy Apps Manager, you must perform these functions through the cf CLI. After Apps Manager has been deployed, we recommend deselecting the checkbox for this errand on subsequent Elastic Runtime deployments. For more information about the Apps Manager, see Getting Started with the Apps Manager.

  • Notifications deploys an API for sending email notifications to your PCF platform users.

    Note: The Notifications app requires that you configure SMTP with a username and password, even if SMTP Authentication Mechanism is set to none.

  • Notifications-UI deploys a dashboard for users to manage notification subscriptions.

  • Deploy CF Autoscaling App enables your deployment to automatically scale the number of instances of an app in response to changes in its usage load. To enable Autoscaling for an app, you must also bind the Autoscaling service to it. For more information, see the Bind a Service Instance section of the Managing Service Instances with the CLI topic.

    Note: The Autoscaling app requires the Notifications app to send scaling action alerts by email.

  • Register Autoscaling Service Broker makes the Autoscaling service available to your applications. Without this errand, you cannot bind the Autoscaling app to your apps.

Step 19: (Optional) Configure Resources

Note: Ops Manager 1.7 defines specific instance types instead of custom sizes. Each instance adopts an instance type that is the closest match to its previous custom size for CPU, memory, and disk space. You can modify the automatically selected instance size by selecting a different type under Resource Config.

Scale the number of instances in order to reduce resources and configure your deployment.

Resource config v

By default, Elastic Runtime uses an internal filestore and internal databases. If you configure Elastic Runtime to use external resources, you can disable the corresponding system-provided resources in Ops Manager to reduce costs and administrative overhead.

For more information regarding scaling instances, see the Zero Downtime Deployment and Scaling in CF and the Scaling Instances in Elastic Runtime topics.

Complete the following procedures to disable specific VMs in Ops Manager:

  1. Click Resource Config.

  2. If you configure Elastic Runtime to use an external S3-compatible filestore, edit the following fields:

    • NFS Server: Enter 0 in Instances.
  3. If you configure Elastic Runtime to use an external Relational Database Service (RDS), edit the following fields:

    • MySQL Proxy: Enter 0 in Instances.
    • MySQL Server: Enter 0 in Instances.
    • Cloud Controller Database: Enter 0 in Instances.
    • UAA Database: Enter 0 in Instances.
    • Apps Manager Database: Enter 0 in Instances.
  4. If you are using an External Load Balancer instead of HAProxy, enter 0 in the Instances field for HAProxy.

  5. Click Save.

Step 20: Configure Stemcell

  1. Select Stemcell. This page displays the stemcell version that shipped with Ops Manager. Stemcell

    You can also use this page to import a new stemcell version. You only need to import a new Stemcell if your Ops Manager does not already have the Stemcell version required by Elastic Runtime.

Step 21: Complete the Elastic Runtime Installation

  1. Click the Installation Dashboard link to return to the Installation Dashboard.

  2. Click Apply Changes. If the following ICMP error message appears, click Ignore errors and start the install.

    Install error

    The install process generally requires a minimum of 90 minutes to complete. The image shows the Changes Applied window that displays when the installation process successfully completes.

    Ops manager complete

Create a pull request or raise an issue on the source for this page in GitHub