Configuring PAS

Page last updated:

This topic describes how to configure Pivotal Application Service (PAS) as part of deploying Pivotal Cloud Foundry (PCF).

Prerequisites

Before beginning this procedure, you must:

  • Ensure that you have successfully completed the steps to prepare your environment for PCF and install and configure the BOSH Director. For more information, see the Ops Manager configuration topic for your IaaS.

  • If you plan to install the PCF IPsec add-on, you must do so before installing any other tiles. Pivotal recommends installing IPsec immediately after Pivotal Ops Manager and before installing the PAS Runtime tile. For more information, see Installing the IPsec Add-On for PCF.

Add PAS to Ops Manager

Before you can configure PAS, you must add the PAS tile to your Ops Manager Installation Dashboard.

To add the PAS tile to your Ops Manager Installation Dashboard:

  1. If you have not already downloaded PAS, log in to Pivotal Network and click Pivotal Application Service.

  2. From the Releases dropdown, select the release to install and choose one of the following:

    1. Click Pivotal Application Service to download the PAS .pivotal file.
    2. Click Small Footprint PAS to download the Small Footprint PAS .pivotal file. For more information, see Getting Started with Small Footprint PAS.
  3. Navigate to the Pivotal Cloud Foundry Ops Manager Installation Dashboard.

  4. Click Import a Product to add your tile to Ops Manager. For more information, see Adding and Deleting Products.

  5. Click the Pivotal Application Service tile in the Installation Dashboard.

Assign AZs and Networks

Note: For Azure environments, this configuration pane is Assign Networks and does not include AZ configuration.

In the Assign AZ and Networks pane, you assign jobs to your Availability Zones (AZs) and networks.

To configure the Assign AZ and Networks pane:

  1. Select Assign AZ and Networks.

  2. Select the first AZ under Place singleton jobs. Ops Manager runs any job with a single instance in this AZ.

  3. Select all AZs under Balance other jobs. Ops Manager balances instances of jobs with more than one instance across the AZs that you specify.

    Note: For production deployments, Pivotal recommends at least three AZs for a highly available installation.

  4. From the Network dropdown, choose the runtime network that you created when configuring the BOSH Director tile.

  5. Click Save.

Configure Domains

In the Domains pane, you configure a wildcard DNS record for both the apps domain and system domain.

To configure the Domains pane:

  1. Select Domains.

  2. Enter the name of your system domain in the System Domain field. For example, system.example.com. The system domain defines your target when you push apps to PAS, such as your load balancer or HAProxy. PAS assigns system components such as UAA and Apps Manager to subdomains under this domain.

  3. Enter the name of your apps domain in the Apps Domain field. For example, apps.example.com. The apps domain is the default domain that apps use for their hostnames. PAS hosts each app at subdomains under this domain. You can use the Cloud Foundry Command Line Interface (cf CLI) to add or delete subdomains assigned to individual apps.

    For additional guidance based on your installation method, see the following table:

    Installation Method Guidance
    Manual Enter the domains you created when preparing your environment for PCF.
    Terraform Enter the values for sys_domain and apps_domain from the Terraform output.

    Note: Pivotal recommends that you use the same domain name but different subdomain names for your system and app domains. Doing so allows you to use a single wildcard certificate for the domain while preventing apps from creating routes that overlap with system routes.

  4. Click Save.

Configure Networking

In the Networking pane, you configure security and routing services for your IaaS.

To configure the Networking pane:

  1. For Router IPs and HAProxy IPs, see the following guidance:

    • For AWS, Azure, and GCP, leave these fields blank. You do not need to complete these fields when deploying PCF on these infrastructures.
    • For OpenStack and vSphere the values you enter in the Router IPs and HAProxy IPs fields depend on whether you are using HAProxy in your deployment. Use the table below to determine how to complete these fields.

      Note: If you choose to assign specific IP addresses in either the Router IPs or HAProxy IPs field, ensure that these IP addresses are in the subnet that you configured for PAS in Ops Manager.

      Using HAProxy? Router IPs Field HAProxy IPs Field
      No
      1. Choose IP addresses from the subnet you configured in Ops Manager.
      2. Enter these IP addresses in the Router IPs field. You should specify more than one IP address for high availability. The IP addresses must be within your subnet CIDR block.
      3. Configure your load balancer to forward requests for the domains that you have configured for your deployment to these IP addresses.
      Leave this field blank.
      Yes Leave this field blank.
      1. Choose IP addresses from the subnet you configured in Ops Manager.
      2. Enter these IP addresses in the HAProxy IPs field. You should specify more than one IP address for high availability.
      3. Configure your load balancer to forward requests for the domains you have configured for your deployment to these IP addresses.
  2. For SSH Proxy IPs and TCP Router IPs, see the following guidance:

    • For AWS, Azure, and GCP, leave these fields blank. You do not need to complete these fields when deploying PCF on these infrastructures.
    • For OpenStack and vSphere:
      • (Optional) In SSH Proxy IPs, add the IP address for your Diego Brain, which accepts requests to SSH into app containers on port 2222.
      • (Optional) In TCP Router IPs, add the IP address(es) you want to assign to the TCP Routers. You enable this feature at the bottom of this screen.

        Note: If you have mutual TLS app identity verification enabled, app containers accept incoming communication only from the Gorouter. This disables TCP routing.

  3. Under Certificates and Private Key for HAProxy and Router, you must provide at least one certificate and private key name and certificate key pair for HAProxy and the Gorouter. HAProxy and the Gorouter are enabled to receive TLS communication by default. You can configure multiple certificates for HAProxy and the Gorouter.

    Note: When providing custom certificates, enter them in the following order: wildcard, Intermediate, CA. For more information, see Creating a .pem File for SSL Certificate Installations in the DigiCert documentation.

    1. Click Add to add a name for the certificate chain and its private key pair. This certificate is the default used by HAProxy and the Gorouter. You can either provide a certificate signed by a Certificate Authority (CA) or click on the Generate RSA Certificate link to generate a certificate generated by the Ops Manager CA. For the values to use, see Providing a Certificate for Your TLS Termination Point.

      Note: If you configured Ops Manager Front End without a certificate, you can use this new certificate to complete Ops Manager configuration. To configure your Ops Manager Front End certificate, see the Configure Front End section in the Preparing to Deploy Ops Manager on GCP Manually topic.

    2. If you want to configure multiple certificates for HAProxy and the Gorouter, click Add and fill in the appropriate fields for each additional certificate key pair. For details about generating certificates in Ops Manager for your wildcard system domains, see Providing a Certificate for Your TLS Termination Point.

      Note: Ensure that you add any certificates that you generate in this pane to your infrastructure load balancer.

  4. (Optional) When validating client requests using mutual TLS, the Gorouter trusts multiple certificate authorities (CAs) by default. To configure HAProxy to trust the same CAs as the Gorouter, enter your CA certificates under Certificate Authorities trusted by Router and HAProxy. All CA certificates should be appended together into a single collection of PEM-encoded entries.

  5. In the Minimum version of TLS supported by HAProxy and Router field, select the minimum version of TLS to use in HAProxy and Gorouter communications. HAProxy and the Gorouter use TLS v1.2 by default. If you need to accommodate clients that use an older version of TLS, select a lower minimum version. For a list of TLS ciphers supported by the Gorouter, see the TLS Cipher Suite Support section of the Securing Traffic into Cloud Foundry topic.

  6. (Optional) Under Balancing algorithm used by Router, select your load balancing algorithm for Router. Pivotal recommends Round robin for most use cases. Selecting Least connection may result in a more even load between application instances. For more information, see HTTP Routing.

  7. Configure Logging of Client IPs in CF Router. The Log client IPs option is set by default. To comply with the General Data Protection Regulation (GDPR), select one of the following options to disable logging of client IP addresses:

    • If your load balancer exposes its own source IP address, select Disable logging of X-Forwarded-For header only.
    • If your load balancer exposes the source IP of the originating client, select Disable logging of both source IP and X-Forwarded-For header.

  8. Under Configure support for the X-Forwarded-Client-Cert header, configure PCF handles x-forwarded-client-cert (XFCC) HTTP headers based on where TLS is terminated for the first time in your deployment.

    The following table indicates which option to choose based on your deployment configuration.

    Deployment Configuration TLS Option Additional Notes
    • The Load Balancer is terminating TLS, and
    • Load balancer is configured to put the client certificate from a mutual authentication TLS handshake into the X-Forwarded-Client-Cert HTTP header
    TLS terminated for the first time at infrastructure load balancer (default). Both HAProxy and the Gorouter forward the XFCC header when included in the request.
    • The Load Balancer is configured to pass through the TLS handshake via TCP to the instances of HAProxy, and
    • HAProxy instance count is > 0
    TLS terminated for the first time at HAProxy. HAProxy sets the XFCC header with the client certificate received in the TLS handshake. The Gorouter forwards the header.

    Breaking Change: In the Router behavior for Client Certificates field, you cannot select the Router does not request client certificates option.

    • The Load Balancer is configured to pass through the TLS handshake via TCP to instances of the Gorouter

    TLS terminated for the first time at the Gorouter. The Gorouter strips the XFCC header if it is included in the request and forwards the client certificate received in the TLS handshake in a new XFCC header.

    If you have deployed instances of HAProxy, app traffic bypasses those instances in this configuration. If you have also configured your load balancer to route requests for ssh directly to the Diego Brain, consider reducing HAProxy instances to 0.

    Breaking Change: In the Router behavior for Client Certificates field, you cannot select the Router does not request client certificates option.


    For a description of the behavior of each configuration option, see the Forward Client Certificate to Applications section of the HTTP Routing topic.

  9. To configure HAProxy to handle client certificates, select one of the following options in the HAProxy behavior for Client Certificate Validation field:

    • HAProxy does not request client certificates: This option requires mutual authentication, which makes it incompatible with TLS termination point option HAProxy. HAProxy does not request client certificates, so the client does not provide them and no validation occurs. This is the default configuration.
    • HAProxy requests but does not require client certificates: The HAProxy requests client certificates in TLS handshakes and validates them when presented, but does not require them. This option is required if you want to enable mutual TLS app identity verification and TLS is terminated for the first time at HAProxy.

      Warning: Upon upgrade, PAS fails to receive requests if your load balancer is configured to present a client certificate in the TLS handshake with HAProxy but HAProxy has not been configured with the certificate authority used to sign it. To mitigate this issue, select HAProxy does not request client certificates or configure the HAProxy with the appropriate CA.

  10. To configure Gorouter behavior for handling client certificates, select one of the following options in the Router behavior for Client Certificate Validation field:

    • Router does not request client certificates: Client certificates are not requested, so the client does not provide them and validation of client certificates does not occur. This option is incompatible with the TLS termination point options HAProxy and Router because these options require mutual authentication.
    • Router requests but does not require client certificates: The Gorouter requests client certificates in TLS handshakes and validates them when presented, but does not require them. This is the default configuration.
    • Router requires client certificates: The Gorouter validates that the client certificate is signed by a Certificate Authority that the Gorouter trusts. If the Gorouter cannot validate the client certificate, the TLS handshake fails.

      Warning: Requests to the platform fail upon upgrade if your load balancer is configured with client certificates and the Gorouter does not have the CA. To mitigate this issue, select Router does not request client certificates.

  11. In the TLS Cipher Suites for Router field, review the TLS cipher suites for TLS handshakes between the Gorouter and front end clients such as load balancers or HAProxy. The default value for this field is ECDHE-RSA-AES128-GCM-SHA256:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384.

    To modify the default configuration, use an ordered, colon-separated list of Golang-supported TLS cipher suites in the OpenSSL format.

    Operators should verify that the ciphers are supported by any clients or front end components that will initiate TLS handshakes with the Gorouter. For a list of TLS ciphers supported by the Gorouter, see Securing Traffic into Cloud Foundry.

    Verify that every client participating in TLS handshakes with the Gorouter has at least one cipher suite in common with the Gorouter.

    Note: Specify cipher suites that are supported by the versions configured in the Minimum version of TLS supported by HAProxy and Router field.

    Note: AWS Classic Load Balancers do not support PCF’s default cipher suites. For more information about configuring your AWS load balancers and Gorouter, see the TLS Cipher Suite Support by AWS Load Balancers section of the Securing Traffic into Cloud Foundry topic.

  12. In the TLS Cipher Suites for HAProxy field, review the TLS cipher suites for TLS handshakes between HAProxy and its clients such as load balancers and the Gorouter. The default value for this field is DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384

    To modify the default configuration, use an ordered, colon-separated list of TLS cipher suites in the OpenSSL format.

    Operators should verify that the ciphers are supported by any clients or front end components that will initiate TLS handshakes with HAProxy.

    Verify that every client participating in TLS handshakes with HAProxy has at least one cipher suite in common with HAProxy.

    Note: Specify cipher suites that are supported by the versions configured in the Minimum version of TLS supported by HAProxy and Router field.

  13. Under HAProxy forwards requests to Router over TLS, select Enable or Disable based on your deployment layout.

    • To enable communication between HAProxy and the Gorouter:
      1. Leave Enable selected.
      2. In the Certificate Authority for HAProxy Backend field, specify the CA that signed the certificate you configured in the Certificate and Private Keys for HAProxy and Router field.

        Note: If you used the Generate RSA Certificate link to generate a certificate, then the CA to specify is the Ops Manager CA, which you can locate at the /api/v0/certificate_authorities endpoint in the Ops Manager API.

      3. Make sure that the Gorouter and HAProxy have TLS cipher suites in common in the TLS Cipher Suites for Router and TLS Cipher Suites for HAProxy fields.
        For more information, see the Terminating TLS at the Load Balancer and Gorouter section of the Securing Traffic into Cloud Foundry topic, Providing a Certificate for Your TLS Termination Point, and Using the Ops Manager API.
    • To use non-encrypted communication between HAProxy and the Gorouter:
      1. Select Disable.
      2. If you are not using HAProxy, set the number of HAProxy job instances to 0 on the Resource Config page. For more information, see Scale Down and Disable Resources.
        For more information, see the Terminating TLS at the Gorouter Only and Terminating TLS at the Load Balancer Only sections of the Securing Traffic into Cloud Foundry topic.

  14. If you want to force browsers to use HTTPS when making requests to HAProxy, select Enable in the HAProxy support for HSTS field and complete the following optional configuration steps:

    1. (Optional) Enter a Max Age in Seconds for the HSTS request. HAProxy forces HTTPS requests from browsers for the duration of this setting. The maximum age is one year, or 31536000 seconds.
    2. (Optional) Enable the Include Subdomains checkbox to force browsers to use HTTPS requests for all component subdomains.
    3. (Optional) Enable the Enable Preload checkbox to force instances of Google Chrome, Firefox, and Safari that access your HAProxy to refer to their built-in lists of known hosts that require HTTPS, of which HAProxy is one. This ensures that the first contact a browser has with your HAProxy is an HTTPS request, even if the browser has not yet received an HSTS header from HAProxy.

  15. If you are not using SSL encryption or if you are using self-signed certificates, select the Disable SSL certificate verification for this environment checkbox. Enabling this checkbox also disables SSL verification for route services and disables mutual TLS app identity verification.

    Note: For production deployments, Pivotal does not recommend disabling SSL certificate verification.

  16. (Optional) If you want HAProxy or the Gorouter to reject any HTTP (non-encrypted) traffic, select the Disable HTTP on HAProxy and Gorouter checkbox. When selected, HAProxy and the Gorouter do not listen on port 80.

  17. (Optional) Select the Disable insecure cookies on the Router checkbox to set the secure flag for cookies generated by the router.

  18. (Optional) To disable the addition of Zipkin tracing headers on the Gorouter, deselect the Enable Zipkin tracing headers on the router checkbox. Zipkin tracing headers are enabled by default. For more information about using Zipkin trace logging headers, see the HTTP Headers for Zipkin Tracing section of the HTTP Routing topic.

  19. (Optional) The Enable Router to write access logs locally checkbox is selected by default. Pivotal recommends disabling this checkbox for high-traffic deployments, since logs may not be rotated fast enough and can fill up the disk.

  20. By default, PAS routers handle traffic for apps deployed to an isolation segment created by the PCF Isolation Segment tile. To configure PAS routers to reject requests for apps within isolation segments, enable the Routers reject requests for Isolation Segments checkbox. Do not enable this option without deploying routers for each isolation segment. For more information, see Installing PCF Isolation Segment and the Sharding Routers for Isolation Segments section of the Routing for Isolation Segments topic.

  21. (Optional) By default, Gorouter support for the PROXY protocol is disabled. To enable the PROXY protocol, select the Enable support for PROXY protocol in CF Router checkbox. When enabled, client-side load balancers that terminate TLS but do not support HTTP can pass along information from the originating client. Enabling this option may impact Gorouter performance. For more information about enabling the PROXY protocol in the Gorouter, see the About HTTP Header Forwarding sections in the Securing Traffic in Cloud Foundry topic.

  22. In the Choose whether to enable route services section, choose either Enable route services or Disable route services. Route services are a class of marketplace services that perform filtering or content transformation on app requests and responses. For more information, see Route Services and the List Marketplace Services section of the Managing Service Instances with the cf CLI topic. See the Route Services topic for details.

    1. If you enabled route services, you can also configure the Bypass security checks for route service lookup field. Pivotal recommends that you do not enable this field because it has potential security concerns. However, you may need to enable it if your load balancer requires mutual TLS from clients. For more information, see Configuring Route Service Lookup.

  23. (Optional) If you want to limit the number of app connections to the back end, enter a value in the Max Connections Per Backend field. You can use this field to prevent a poorly behaving app from all the connections and impacting other apps. No value or a value of 0 sets no limit. To choose a value for this field, review the peak concurrent connections received by instances of the most popular apps in your deployment. You can determine the number of concurrent connections for an app from the httpStartStop event metrics emitted for each app request.

    If your deployment uses PCF Metrics, you can also obtain this peak concurrent connection information from the Network Metrics section of the Monitoring and Troubleshooting Apps with PCF Metrics topic. The default value is 500.

  24. Under Enable Keepalive Connections for Router, select Enable or Disable. Keep-alive connections are enabled by default. For more information, see the Keep-Alive Connections section of the HTTP Routing topic.

  25. (Optional) To accommodate larger uploads over connections with high latency, increase the number of seconds in the Router Timeout to Backends field.

  26. (Optional) Use the Frontend Idle Timeout for Gorouter and HAProxy field to help prevent connections from your load balancer to Gorouter or HAProxy from being closed prematurely. The value you enter sets the duration, in seconds, that Gorouter or HAProxy maintains an idle open connection from a load balancer that supports keep-alive connections.

    Set the value higher than the back end idle timeout for your load balancer to avoid the race condition where the load balancer sends a request before it discovers that the Gorouter or HAProxy has closed the connection.

    See the following table for specific guidance and exceptions to this rule:

    IaaS Guidance
    AWS AWS ELB has a default timeout of 60 seconds, so Pivotal recommends a value greater than 60.
    Azure By default, Azure load balancer times out at 240 seconds without sending a TCP RST to clients, so Pivotal recommends a value lower than 240 to force the load balancer to send the TCP RST.
    GCP GCP has a default timeout of 600 seconds. For GCP HTTP load balancers, Pivotal recommends a value greater than 600. For GCP TCP load balancers, Pivotal recommends a value less than 600 to force the load balancer to send a TCP RST.
    Other Set the timeout value to be greater than that of the load balancer’s back end idle timeout.

    Note: Do not set a front end idle timeout lower than six seconds.

  27. (Optional) Increase the value of Load Balancer Unhealthy Threshold to specify the amount of time, in seconds, that the Gorouter continues to accept connections before shutting down. During this period, health checks may report the Gorouter as unhealthy, which causes load balancers to failover to other routers. Set this value to an amount greater than or equal to the maximum time it takes your load balancer to consider a Gorouter instance unhealthy, given repeated failed health checks.

  28. (Optional) Modify the value of Load Balancer Healthy Threshold. This field specifies the amount of time, in seconds, to wait until declaring the Gorouter instance started. This allows an external load balancer time to register the Gorouter instance as healthy.

  29. (Optional) If app developers in your organization want certain HTTP headers to appear in their app logs with information from the Gorouter, specify them in the HTTP Headers to Log field with a comma-separated list. For example, to support app developers that deploy Spring apps to PCF, you can enter Spring-specific HTTP headers.

  30. If you expect requests larger than the default maximum of 16.384 KB, enter a new value in bytes for HAProxy Request Max Buffer Size. You may need to do this, for example, to support apps that embed a large cookie or query string values in headers. Requests larger than the maximum value result in a gateway error.

  31. If your PCF deployment uses HAProxy and you want it to receive traffic only from specific sources, use the following fields:

    • HAProxy Protected Domains: Enter a comma-separated list of domains to protect from unknown source requests.
    • HAProxy Trusted CIDRs: Optionally, enter a space-separated list of CIDRs to limit which IP addresses from the Protected Domains can send traffic to PCF.

  32. The Loggregator Port defaults to 443 if left blank. For AWS environments that are not using an Application Load Balancer, enter 4443.

  33. For Container Network Interface Plugin, select one of the following:

    • Silk: This option is the default container network interface (CNI) for PAS.
    • External: Select this if you are deploying the VMware NSX-T Container Plug-in for PCF.
      • If you select External, follow the instructions in Deploying PAS with NSX-T Networking in addition to the PAS configuration instructions in this topic.

        Warning: The NSX-T integration only works for fresh installs of PCF. If your PAS is already deployed and running with Silk as its CNI, you cannot change the CNI plugin to NSX-T.

  34. If you selected Silk in the previous step, review the following fields:

    1. (Optional) The default value for the Applications Network Maximum Transmission Unit (MTU) field is 1454. Some configurations, such as networks that use GRE tunnels, may require a smaller MTU value. To change this value, enter your desired MTU value in bytes.

    2. (Optional) Enter an IP range for the overlay network in the Overlay Subnet box. If you do not set a custom range, Ops Manager uses 10.255.0.0/16.

      Warning: The overlay network IP range must not conflict with any other IP addresses in your network.

    3. Enter a UDP port number in the VXLAN Tunnel Endpoint Port box. If you do not set a custom port, Ops Manager uses 4789.

      • For Denied logging interval, set the per-second rate limit for packets blocked by either a container-specific networking policy or by App Security Group (ASG) rules applied across the space, org, or deployment. This field defaults to 1. For more information, see the Policies section of the Container-to-Container Networking topic and Restricting App Access to Internal PCF Components.
      • For UDP logging interval, set the per-second rate limit for UDP packets sent and received. This field defaults to 100.
      • To enable logging for app traffic, enable the Log traffic for all accepted/denied application packets checkbox. This increases log volume. For more information, see the App Traffic Logging section of the Configuring Logging in PAS topic.

    4. The Enable Silk Policy Enforcement checkbox is selected by default. To disable Silk network policy enforcement between apps, disable the checkbox. Disabling network policy enforcement allows all apps to send network traffic to all other apps in the foundation despite no policy specifically allowing it.
  35. For DNS Search Domains, enter DNS search domains for your containers as a comma-separated list. DNS on your containers appends these names to its host names, to resolve them into full domain names.

  36. For Database Connection Timeout, set the connection timeout for clients of the policy server and Silk databases, in seconds. The default value is 120. You may need to increase this value if your deployment experiences timeout issues related to container-to-container networking.

  37. (Optional) TCP Routing is disabled by default. You should enable this feature if your DNS sends TCP traffic through a load balancer rather than directly to a TCP router. To enable TCP routing:

    1. Select Enable TCP Routing.

      Note: If you have mutual TLS app identity verification enabled, app containers accept incoming communication only from the Gorouter. This disables TCP routing.

    2. For TCP Routing Ports, enter a single port or a range of ports for the load balancer to forward to. These are the same ports that you configured in the Pre-Deployment Steps of the Enabling TCP Routing topic.
      • To support multiple TCP routes, Pivotal recommends allocating multiple ports.
      • To allocate a list of ports rather than a range:
        1. Enter a single port in the TCP Routing Ports field.
        2. After deploying PAS, follow the directions in the Configuring a List of TCP Routing Ports section of the Pivotal Application Service v2.3 Release Notes to add TCP routing ports using the cf CLI to add TCP routing ports using the cf CLI.
    3. For TCP request timeout, you can optionally modify the default value of 300 seconds. This field determines when the TCP Router closes idle connections from clients to apps that use TCP routes. You may want to increase this value to enable developers to push apps that require long-running idle connections with clients.
    4. Ensure that you follow the additional instructions based on your IaaS:
      IaaS Instructions
      GCP Specify the name of a GCP TCP load balancer in the LOAD BALANCER column of the TCP Router job in the Resource Config pane. You configure this later on in PAS. For more information, see Configure Resources.
      AWS Specify the name of a TCP ELB in the LOAD BALANCER column of the TCP Router job in the Resource Config pane. You configure this later on in PAS. For more information, see Configure Resources.
      Azure Specify the name of a Azure load balancer in the LOAD BALANCER column of the TCP Router job in the Resource Config pane. You configure this later on in PAS. For more information, see Configure Resources.
      OpenStack and vSphere
      1. Return to the top of the Networking pane.
      2. In the TCP router IPs field, ensure that you have entered IP addresses that are within your subnet CIDR block. These are the same IP addresses you configured your load balancer with in the Pre-Deployment Steps section of the Enabling TCP Routing topic, unless you configured DNS to resolve the TCP domain name directly to an IP you have chosen for the TCP router.

  38. (Optional) To disable TCP routing, click Select this option if you prefer to enable TCP Routing at a later time. For more information, see Configuring TCP Routing in PAS.

  39. (Optional) To enable the creation of dynamic egress policies that allow apps to reach external services, select Enable Beta Dynamic Egress Enforcement. For more information, see Administering Dynamic Egress Policies (Beta).

  40. (Optional) For additional security, enter headers that you want the Gorouter to remove from app responses in Remove Specified HTTP Response Headers.

  41. Click Save.

Configure Networking - Service Mesh (Beta)

PAS includes an optional, beta routing plane that uses a service mesh. A service mesh provides traffic management, security, and observability for microservices. It enables new features such as weighted routing. For more information, see Service Mesh (Beta).

To add service mesh to your deployment:

  1. Under Service Mesh (Beta), select Enable.

  2. For IP Addresses for Ingress Router, do the following depending on your IaaS:

    • vSphere: Enter static IPs for the Istio Routers. You must configure your load balancer with these IPs as well.
    • Other: Leave this field blank.
  3. For External Domain, enter the domain name that points to the load balancer in front of your Istio router.

  4. For Ingress Router TLS Keypairs, complete the following fields. You can add more than one keypair if desired using the Add button.

    • Name: Enter a name for the keypair.
    • Certificate and Private Key for Istio Router: Enter the private key and certificate for TLS handshakes with clients. These must be in PEM block format.
  5. Click Save.

  6. Create a load balancer and add it to the Resource Config pane as described in Deploying Service Mesh (Beta).

Configure Application Containers

In the Application Containers pane, you enable microservice frameworks, private Docker registries, and other services that support your apps at the container level.

To configure the Application Containers pane:

  1. Select Application Containers.

  2. The Enable Custom Buildpacks checkbox governs the ability to pass a custom buildpack URL to the -b option of the cf push command. By default, this ability is enabled, letting developers use custom buildpacks when deploying apps. Disable this option by disabling the checkbox. For more information about custom buildpacks, see Buildpacks.

  3. The Allow SSH access to app containers checkbox controls SSH access to app instances. Enable the checkbox to permit SSH access across your deployment, and disable it to prevent all SSH access. For more information about SSH access permissions at the space and app scope, see App SSH Overview.

  4. To enable SSH access for new apps by default in spaces that allow SSH, select the Enable SSH when an app is created checkbox. If you disable the checkbox, developers can still enable SSH after pushing their apps by running cf enable-ssh APP-NAME.

    Note: If you are using a load balancer instead of HAProxy, ensure that it has port 2222 open to enable SSH traffic.

    You can give SSH access to an app only if an admin assigns you a Space Developer role in the space where the app runs. For more information, see the Manage App Space Roles section in the Managing User Roles with Apps Manager topic.

  5. To enable SSH access for new apps by default in spaces that allow SSH, select the Enable SSH when an app is created checkbox. If you disable this checkbox, developers can still enable SSH after pushing their apps by running cf enable-ssh APP-NAME.

  6. Choose the method the Gorouter uses to verify app identity under Router application identity verification. To enable the Gorouter to verify app identity using TLS, select Router uses TLS to verify application identity. To enable the Gorouter and your apps to verify each other’s identity using TLS, select Router and apps use mutual TLS to verify each other’s identity.

    Note: Mutual TLS (mTLS) app identity verification does not work for Windows containers.

    Verifying app identity using TLS or mTLS enables encryption between router and app containers and guards against misrouting during control plane failures. For more information about Gorouter route consistency modes, see the Preventing Misrouting section in the HTTP Routing topic.

    Note: To support mTLS app identity verification, you need v2.3 or later of both PAS and PCF Isolation Segment (IST). The Gorouter and Diego Cell components in PCF v2.2 and earlier do not support mTLS handshakes.

    Note: This feature does not work if the Disable SSL certificate verification for this environment checkbox is enabled in the Networking pane.

  7. To enable time-to-live (TTL) expiration for routes, select the Prune Routes on TTL Expiry for TLS Backends checkbox. You should only enable TTL expiration for TLS back ends if you are experiencing occasional misrouting of apps due to stale routes. For more information, see the Preventing Misrouting section in the HTTP Routing topic and Intermittent Misrouting of Apps in Large PCF Foundations in the Pivotal Application Service v2.6 Release Notes.

  8. You can configure PAS to run app instances in Docker containers by providing a comma-separated list of their IP address ranges in the Private Docker Insecure Registry Whitelist textbox. For more information, see Using Docker Registries.

  9. Select your preference for Docker Images Disk-Cleanup Scheduling on Cell VMs. If you choose Clean up disk-space once usage fills disk, enter a value in MB for Reserved amount of disk space for other jobs. For more information about the configuration options and how to configure a reserved amount, see Configuring Cell Disk Cleanup Scheduling.

  10. The Enable Containerd Delegation governs whether or not Garden delegates container create and destroy operations to the containerd tool. By default, this option is enabled and Garden uses containerd. Disable this option by disabling the checkbox. For more information about the containerd tool, see containerd.

  11. Enter a number in the Max Inflight Container Starts field. This number configures the maximum number of started instances across the Diego cells in your deployment. Entering 0 sets no limit. For more information, see the Set a Maximum Number of Started Containers section of the Configuring PAS for Upgrades topic.

  12. Under Enabling NFSv3 volume services, select Enable or Disable. NFS volume services allow app developers to bind existing NFS volumes to their apps for shared file access. For more information, see Enabling Volume Services.

    Note: In a fresh install, NFSv3 volume services is enabled by default. In an upgrade, NFSv3 volume services is set to the same setting as it was in the previous deployment.

  13. (Optional) To configure LDAP for NFSv3 volume services:

    • For LDAP Service Account User, enter the username of the service account in LDAP that will manage volume services.
    • For LDAP Service Account Password, enter the password for the service account.
    • For LDAP Server Host, enter the hostname or IP address of the LDAP server.
    • For LDAP Server Port, enter the LDAP server port number. If you do not specify a port number, Ops Manager uses 389.
    • For LDAP User Search Base, enter the location in the LDAP directory tree from which any LDAP User search begins. The typical LDAP Search Base matches your domain name.
      For example, a domain named cloud.example.com typically uses the following LDAP User Search Base: ou=Users,dc=example,dc=com.
    • For LDAP Server CA Cert, you can optionally enter a certificate if your LDAP server supports TLS and you want to enable TLS connections from the NFS driver to your LDAP server. Paste in the root certificate from your CA certificate or your self-signed certificate.

      Note: UAA can only parse one certificate entered into this field. If you enter multiple certificates, UAA only uses the first one you entered and ignores the rest. You only need to include one root certificate or self-signed certificate.

  14. Select the Format of timestamps in Diego logs, either RFC3339 timestamps or Seconds since the Unix epoch.

  15. (Optional) Select the Enable SMB volume services checkbox to enable SMB volume services. Enabling SMB volume services allows developers to bind existing SMB shares to their apps. For more information, see Enabling Volume Services.

    Note: If you enable SMB volume services, you must set the SMB Broker Errand to On in the Errands pane.

  16. (Optional) Modify the Default health check timeout. The value configured for this field is the amount of time allowed to elapse between starting up an app and the first healthy response from the app. If the health check does not receive a healthy response within the configured timeout, then the app is declared unhealthy. The default timeout is 60 seconds and the maximum configurable timeout is 600 seconds.

    Warning: If you decrease the default health check timeout value below its current value, existing apps with startup times greater than the new value may fail to start up.

  17. Click Save.

Configure Application Developer Controls

In the Application Developer Controls pane, you configure restrictions and default settings for your apps.

To configure the Application Developer Controls pane:

  1. Select Application Developer Controls.

  2. Enter the Maximum File Upload Size (MB) in MB. This is the maximum size of an app upload.

  3. Enter the Default App Memory (MB) in MB. This is the default amount of memory allocated to a newly-pushed app if no value is specified with cf push.

  4. Enter the Default App Memory Quota per Org in MB. This is the default memory limit for all apps in an org. The specified limit only applies to the first installation of PAS. After the initial installation, operators can use the cf CLI to change the default value.

  5. Enter the Maximum Disk Quota per App (MB) in MB. This is the maximum amount of disk allowed per app.

    Note: If you allow developers to push large apps, PAS may have trouble placing them on Diego Cells. Additionally, in the event of a system upgrade or an outage that causes a rolling deploy, larger apps may not successfully re-deploy if there is insufficient disk capacity. Monitor your deployment to ensure your Diego Cells have sufficient disk to run your apps.

  6. Enter the Default Disk Quota per App (MB) in MB. This is the amount of disk allocated by default to a newly-pushed app if no value is specified with cf push.

  7. Enter the Default Service Instances Quota per Org. The specified limit only applies to the first installation of PAS. After the initial installation, operators can use the cf CLI to change the default value.

  8. Enter the Staging Timeout (Seconds) in seconds. When you stage an app droplet with the Cloud Controller, the server times out after the number of seconds you specify in this field.

  9. For Internal Domain, enter a domain that apps use for internal DNS service discovery. If you specify this domain using cf push -d, other PAS apps can reach the pushed app at APP-NAME.INTERNAL-DOMAIN. This value defaults to apps.internal.

  10. Enable the Allow Space Developers to manage network policies checkbox to permit developers to manage their own network policies for their apps.

  11. Click Save.

Configure Application Security Groups

Setting appropriate ASGs is critical for a secure deployment. To acknowledge that once the PAS deployment completes, it is your responsibility to set the appropriate ASGs:

  1. Select Application Security Groups.

  2. In the Type “X” to acknowledge this requirement field, enter X.

  3. Click Save.

For more information about ASGs, see App Security Groups. For more information about setting ASGs, see Restricting App Access to Internal PAS Components.

Configure Authentication and Enterprise SSO

In the Authentication and Enterprise SSO pane, you configure your user store access.

To configure the Authentication and Enterprise SSO pane:

  1. Select Authentication and Enterprise SSO.

  2. To authenticate user sign-ons, your deployment can use one of three types of user database: the UAA server’s internal user store, an external SAML identity provider, or an external LDAP server.

    • To use the internal UAA, select the Internal option and follow the procedure in Configuring UAA Password Policy to configure your password policy.
    • To connect to an external identity provider through SAML, select the SAML Identity Provider option and follow the instructions in the Configure PAS to Use a SAML Identity Provider section of the Configuring Authentication and Enterprise SSO for PAS topic.
    • To connect to an external LDAP server, select the LDAP Server option and follow the instructions in the Configure LDAP as an Identity Provider for PAS section of the Configuring Authentication and Enterprise SSO for PAS topic.
  3. Click Save.

Configure UAA

In the UAA pane, you configure the User Account and Authentication server.

To configure the UAA pane:

  1. Select UAA.

  2. Under Choose the location of your UAA database, select one of the following:

    • PAS database (configured on the Databases pane): Use the same database server that other PAS components use. This system database is configured in the Databases pane, and it can be either internal or external.
    • Other external database: Use a separate, dedicated database server for UAA.

      Warning: Protect whichever database you use in your deployment with a password.

      Note: For GCP installations, Pivotal recommends using an external database on Google Cloud SQL.

  3. (Optional) If you selected Other external database, complete the fields as follows. Each field includes additional guidance for specific IaaSes and installation methods.

    • For Hostname, enter the hostname of the database server.
      • AWS Terraform: Enter the value of rds_address in your Terraform output.
      • GCP Terraform: Enter the value of sql_db_ip from your Terraform output.
    • For TCP Port, enter the port of the database server.
      • AWS Terraform: Enter the value of rds_port in the Terraform output.
      • GCP and GCP Terraform: Enter 3306.
    • For Username, specify a unique username that can access this specific database on the database server.
      • AWS Terraform: Enter the value of rds_username from your Terraform output.
      • GCP Terraform: Enter the value of pas_sql_username from your Terraform output.
    • For Password, specify a password for the provided username.
      • AWS Terraform: Enter the value of rds_password from your Terraform output.
      • GCP Terraform: Enter the value of pas_sql_password from your Terraform output.
    • For CA Certificate, enter a certificate to use for encrypting traffic to and from the database.

      Note: The CA Certificate field only works if your external database hostname matches a name specified in the certificate. This is not true with GCP CloudSQL.

  4. (Optional) Under JWT Issuer URI, enter the URI that UAA uses as the issuer when generating tokens.

  5. Under SAML Service Provider Credentials, enter a certificate and private key to be used by UAA as a SAML Service Provider for signing outgoing SAML authentication requests. You can provide an existing certificate and private key from your trusted Certificate Authority or generate a certificate. The following domain must be associated with the certificate: *.login.YOUR-SYSTEM-DOMAIN.

    Note: The Pivotal Single Sign-On Service and Pivotal Spring Cloud Services tiles require the *.login.YOUR-SYSTEM-DOMAIN.

  6. If the private key specified under SAML Service Provider Credentials is password-protected, enter the password under SAML Service Provider Key Password.

  7. (Optional) To override the default value, enter a custom SAML Entity ID in the SAML Entity ID Override field. By default, the SAML Entity ID is http://login.YOUR-SYSTEM-DOMAIN where YOUR-SYSTEM-DOMAIN is set in the Domains > System Domain field.

  8. For Signature Algorithm, choose an algorithm from the dropdown to use for signed requests and assertions. The default is SHA256.

  9. (Optional) In the Apps Manager Access Token Lifetime, Apps Manager Refresh Token Lifetime, Cloud Foundry CLI Access Token Lifetime, and Cloud Foundry CLI Refresh Token Lifetime fields, change the lifetimes of tokens granted for Apps Manager and Cloud Foundry Command Line Interface (cf CLI) login access and refresh. Most deployments use the defaults.

  10. (Optional) In the Global Login Session Max Timeout and Global Login Session Idle Timeout fields, change the maximum number of seconds before a global login times out. These fields apply to the following:

    • Default zone sessions: Sessions in Apps Manager, PCF Metrics, and other web UIs that use the UAA default zones.
    • Identity zone sessions: Sessions in apps that use a UAA identity zone, such as a Single Sign-On service plan.
  11. (Optional) Customize the text prompts used for username and password from the cf CLI and Apps Manager login popup by entering values for Customize Username Label (on login page) and Customize Password Label (on login page).

  12. (Optional) The Proxy IPs Regular Expression field contains a pipe-delimited set of regular expressions that UAA considers to be reverse proxy IP addresses. UAA respects the x-forwarded-for and x-forwarded-proto headers coming from IP addresses that match these regular expressions. To configure UAA to respond properly to Gorouter or HAProxy requests coming from a public IP address, append a regular expression or regular expressions to match the public IP address.

  13. Click Save.

Configure CredHub

In the CredHub pane, you configure the CredHub server.

Note: Enabling CredHub is not required. However, you cannot leave the fields under Encryption Keys blank. If you do not intend to use CredHub, enter any text in the Name and Key fields as placeholder values.

To configure the CredHub pane:

  1. Select CredHub.

  2. Select the location of your CredHub database. PAS includes this CredHub database for services to store their service instance credentials.

    • PAS database (configured on the Databases pane): If you select this option, CredHub uses the same database as other PAS components, whether the database is internal or external.

      Note: If you deploy PAS on GCP, you must select PAS database (configured on the Databases pane). For more information, see CredHub Database Cannot be External on GCP in the Pivotal Application Service v2.4 Release Notes.

    • Other external database: If you select this option, configure the following fields:
      • Hostname: Enter the IP address of your database server.
      • TCP Port: Enter the port of your database server, such as 3306.
      • Username: Enter a unique username that can access your CredHub database on the database server.
      • Password: Enter the password for the provided username.
      • Database CA Certificate: This certificate is used when encrypting traffic to and from the database.
        Note: If you deploy PAS on AWS using Terraform, you can locate the following values in your Terraform output:
        • Hostname: rds_address
        • TCP Port: rds_port
        • Username: rds_username
        • Password: rds_password
  3. Under Encryption Keys, specify one or more keys to use for encrypting and decrypting the values stored in the CredHub database.

    • If you plan to use internal encryption, configure the following fields:
      • Name: Enter any key name.
      • Provider: Select Internal.
      • Key: Enter a randomly generated value that is at least 20 characters long. This key is used for encrypting all data.
      • Primary: This checkbox marks the key you specified above as the primary encryption key. You must mark one key as primary. Do not mark more than one key as primary.
    • If you plan to use a hardware security module (HSM) as your encryption provider, configure the following fields:
      • Name: Enter a key name that already exists on your HSM or a new key name. For each new key name, CredHub generates a key in HSM Provider Partition that you configure below.
      • Provider: Select HSM.
      • Key: Enter a placeholder value under Key. CredHub does not use this key for encryption. However, you cannot leave the Key field blank.
      • Primary: This checkbox is used for marking the key you specified above as the primary encryption key. You must mark one key as primary. Do not mark more than one key as primary.

        Note: For information about using additional keys for key rotation, see Rotating Runtime CredHub Encryption Keys.

  4. (Optional) To configure CredHub to use an HSM, configure the following fields:

    • HSM Provider Partition: Enter the name of the HSM provider partition.
    • HSM Provider Partition Password: Enter a password to use to access the HSM provider partition.
    • HSM provider client certificate: Enter the client certificate for the HSM. For more information, see the Create and Register HSM Clients section in the Preparing CredHub HSMs for Configuration topic.
    • In the HSM Provider Servers section, click Add to add an HSM server. You can add multiple HSM servers. For each HSM server, complete the following fields:
      • Host Address: Enter the hostname or IP address of the HSM server.
      • Port: Enter the port of the HSM server. If you do not know your port address, enter 1792.
      • Partition Serial Number: Enter the serial number of the HSM partition.
      • HSM Certificate: Enter the certificate for the HSM server. The HSM presents this certificate to CredHub to establish a two-way TLS connection.
  5. If your deployment uses any PCF services that support storing service instance credentials in CredHub and you want to enable this feature, enable the Secure Service Instance Credentials checkbox.

  6. Click Save.

  7. Go to the Resource Config pane.

  8. Under the Job column of the CredHub row, ensure that the number of instances is set to 2. This is the minimum instance count required for high availability.

  9. Click Save.

For more information about using CredHub for securing service instance credentials, see Securing Service Instance Credentials with Runtime CredHub.

Configure System Databases

In the Databases pane, you can configure PAS to use an internal MySQL database provided with PCF, or you can configure an external database provider for the databases required by PAS.

Note: If you are performing an upgrade, do not modify your existing internal database configuration or you may lose data. You must migrate your existing data first before changing the configuration. For more information, see Upgrading Pivotal Cloud Foundry.

Internal Database Configuration

Note: For Runtime CredHub to work on GCP installations, you must use internal MySQL. For more information, see CredHub Database Cannot Be External on GCP in the Pivotal Application Service v2.4 Release Notes.

Note: For GCP installations, Pivotal recommends selecting External and using Google Cloud SQL. Only use internal MySQL for non-production or test installations on GCP.

To configure internal databases for your deployment:

  1. Under Choose the location of your system databases, select Internal Databases - MySQL - Percona XtraDB Cluster.

  2. Click Save.

To configure high availability for your internal MySQL databases, see Configure Internal MySQL.

External System Database Configuration

Note: If you use external MySQL, you cannot use Runtime CredHub. For more information, see CredHub Database Cannot Be External on GCP in the Pivotal Application Service v2.4 Release Notes.

To configure external databases for your deployment:

  1. Ensure that you have a database instance with the following databases created. The steps vary depending on your database type. For an example procedure, see Creating Databases for PAS.

    • account
    • app_usage_service
    • autoscale
    • ccdb
    • credhub
    • diego
    • locket
    • networkpolicyserver
    • nfsvolume
    • notifications
    • routing
    • silk
    • uaa
  2. In PAS, select Databases.

  3. Under Choose the location of your system databases, select External Databases.

    Note: If you configure external databases, you cannot configure an internal database in the UAA pane.

  4. For Hostname, enter the hostname of the database server. If you are installing PCF using Terraform, this value corresponds to the following variable:

    • AWS Terraform: rds_address
    • GCP Terraform: sql_db_ip
  5. For TCP Port, enter the port of the database server.

    • If you are using GCP CloudSQL, enter 3306.
    • If you are installing PCF on AWS using Terraform, enter the value for rds_port.
  6. Each component that requires a relational database has two corresponding fields: one for the database username and one for the database password. For each set of fields, specify a unique username that can access this specific database on the database server and a password for the provided username. If you are installing PCF using Terraform, these values correspond to the following variables:

    • AWS Terraform: rds_username and rds_password
    • GCP Terraform: pas_sql_username and pas_sql_password

      Note: Ensure that the networkpolicyserver database user has the ALL PRIVILEGES permission.

  7. (Optional) To enable TLS for your external databases, paste your CA certificate in the Database CA Certificate field.

    Note: TLS is not currently supported for databases that do not include a matching hostname in their server certificate, e.g. GCP. For more information, see Connection Options for External Applications in the GCP documentation.

  8. Click Save.

(Optional) Configure Internal MySQL

In the Internal MySQL pane, you configure the internal MySQL clusters for PAS. Only configure this section if you selected one of the Internal Databases - MySQL options in the Databases pane.

To configure the Internal MySQL pane:

  1. Select Internal MySQL.

  2. In the Replication canary time period field, specify how frequently the canary checks for replication failure. Leave the default of 30 seconds or modify the value based on the needs of your deployment. Lower numbers cause the canary to run more frequently, which means that the canary reacts more quickly to replication failure but adds load to the database.

  3. In the Replication canary read delay field, specify how long the canary waits, in seconds, before verifying that data is replicating across each MySQL node. Leave the default of 20 seconds or modify the value based on the needs of your deployment. Clusters under heavy load can experience a small replication lag as write-sets are committed across the nodes.

  4. In the E-mail address field, enter the email address where the MySQL service sends alerts when the cluster experiences a replication issue or when a node is not allowed to auto-rejoin the cluster.

  5. The Allow command history checkbox is enabled by default. To prohibit the creation of command line history files on the MySQL nodes, disable this checkbox.

  6. To allow admin and read-only admin users to connect from any remote host, enable the Allow Remote Admin Access checkbox. When the checkbox is disabled, admins must bosh ssh into each MySQL VM to connect as the MySQL super user.

    Note: Network configuration and ASG restrictions may still limit a client’s ability to establish a connection with the databases.

  7. For Cluster Probe Timeout, enter the maximum amount of time, in seconds, that a new node searches for existing cluster nodes. If left blank, the default value is 10 seconds.

  8. For Max Connections, enter the maximum number of connections allowed to the database. If left blank, the default value is 1500.

  9. To log audit events for internal MySQL, select Enable server activity logging under Server Activity Logging.

    • In the Event types field, you can enter the events you want the MySQL service to log. By default, this field includes connect and query, which tracks who connects to the system and what queries are processed.
      For more information about which events the Percona MySQL server can log, see Audit Log Plugin in the Percona documentation.

      Note: Internal MySQL audit logs are not forwarded to the syslog server because they can contain personally identifying information (PII) and secrets.
      You can use the download-logs script to retrieve the logs, which each MySQL cluster node VM stores in /var/vcap/store/mysql_audit_logs/. For more information, see Script to download MySQL logs for PAS or Tile HA Clusters in the Pivotal Knowledge Base.

  10. In the Load Balancer Healthy Threshold field, enter the amount of time, in seconds, to wait until reporting that the MySQL Proxy instance has started. This allows an external load balancer time to register the instance as healthy.

  11. In the Load Balancer Unhealthy Threshold field, enter the amount of time, in seconds, that the MySQL Proxy continues to accept connections before shutting down. During this period, the health check reports the MySQL Proxy instance as unhealthy to cause load balancers to fail over to other proxies. You must enter a value greater than or equal to the maximum time it takes your load balancer to consider a proxy instance unhealthy, given repeated failed health checks.

  12. To enable MySQL Proxy to listen on port 3336, select the Enable inactive mysql port checkbox. When you run MySQL in high availability mode, this feature allows you to connect to a MySQL node that is not currently serving traffic so that you can run auditing and reporting queries without affecting performance.

  13. To enable the MySQL interruptor feature, enable the Prevent node auto re-join checkbox. This feature stops all writes to the MySQL database if it notices an inconsistency in the dataset between the nodes. For more information, see the Interruptor section in the Troubleshooting and Diagnostics topic of the MySQL for PCF documentation.

  14. Click Save.

For more information on how to monitor the node health of your MySQL Proxy instances, see Using the MySQL Proxy.

Configure File Storage

In File Storage, you determine where you want to place the file storage for your Cloud Controller.

To configure the File Storage pane:

  1. Select File Storage.

  2. To set the maximum number of recent valid packages that your app can store, not including the package for the current droplet, enter a value in the Max Valid Packages per App field. Pivotal recommends leaving the default value of 5. However, you can lower the value if you have strict storage requirements and want to use less disk space.

  3. To set the maximum number of recent valid droplets that your app can store, not including the package for the current droplet, enter a value in the Max Staged Droplets per App field. Pivotal recommends leaving the default value of 5. However, you can lower the value if you have strict storage requirements and want to use less disk space.

  4. For Configure your Cloud Controller’s filesystem, see Configuring File Storage for PAS.

(Optional) Configure System Logging

In the System Logging pane, you can configure system logging in PAS to forward log messages from PAS component VMs to an external service. Pivotal recommends forwarding logs to an external service for use in troubleshooting. If you do not fill these fields, platform logs are not forwarded but remain available on the component VMs and for download through Ops Manager.

Note: The following instructions explain how to configure system logging for PAS component VMs. To forward logs from PCF tiles to an external service, you must also configure system logging in each tile. See the documentation for the given tiles for information about configuring system logging.

To configure the System Logging pane:

  1. Select System Logging.

  2. For Address, enter the hostname or IP address of the syslog server.

  3. For Port, enter the port of the syslog server. The default port for a syslog server is 514.

    Note: The host must be reachable from the PAS network and accept UDP or TCP connections. Ensure the syslog server listens on external interfaces.

  4. For Transport Protocol, select a transport protocol for log forwarding.

  5. For Encrypt syslog using TLS?, select Yes to use TLS encryption when forwarding logs to a remote server.

    1. For Permitted Peer, enter either the name or SHA1 fingerprint of the remote peer.
    2. For TLS CA Certificate, enter the TLS CA certificate for the remote server.
  6. For Number of syslog drain addresses to retrieve, enter the number of apps with syslog drain addresses to retrieve from Cloud Controller per request.

  7. Disable the Include container metrics in Syslog Drains checkbox to prevent the CF Drain CLI plugin from including app container metrics in syslog drains. This feature is enabled by default. For more information about the CF Drain CLI plugin, see the CF Drain CLI Plugin repository on GitHub.

  8. Select the Enable agent-based syslog egress for app logs checkbox to use agent-based syslog. Agent-based syslog provides an alternative to scalable-syslog architecture. For more information about agent-based syslog, see the Loggregator Architecture and Components section of the Loggregator Architecture topic.

    Note: Selecting this checkbox disables the Syslog Adapter and Syslog Scheduler VMs to prevent log duplication.

  9. Select the Enable Cloud Controller security event logging checkbox to include security events in the log stream. This feature logs all API requests, including the endpoint, user, source IP address, and request result, in the Common Event Format (CEF).

  10. Enable the Use TCP for file forwarding local transport checkbox to transmit logs over TCP. This prevents log truncation, but may cause performance issues.

  11. The Don’t Forward Debug Logs checkbox is enabled by default. To forward DEBUG syslog messages to an external service, disable the checkbox.

    Note: Some PAS components generate a high volume of DEBUG syslog messages. Enabling the Don’t Forward Debug Logs checkbox prevents PAS components from forwarding the DEBUG syslog messages to external services. However, PAS still writes the messages to the local disk.

  12. For Custom rsyslog Configuration, enter a custom syslog rule. For more information about adding custom syslog rules, see Customizing Platform Log Forwarding.

  13. Select the Enable System Metrics checkbox to emit system-level metrics about all VMs in the deployment. For a list of the VM metrics that the System Metric Agent emits, see VM Metrics in GitHub.

  14. Click Save.

To configure Ops Manager for system logging, see the Settings section in the Using the Ops Manager Interface topic.

(Optional) Configure Custom Branding and Apps Manager

In the Custom Branding pane, you can customize the appearance of the PCF login portal and Apps Manager. In the Apps Manager pane, you can configure the functionality of Apps Manager, as well as how it displays the pages for your apps in the Pivotal Services Marketplace.

To configure the Custom Branding and Apps Manager panes:

  1. Select Custom Branding.

  2. Configure the fields in the Custom Branding pane. For more information about the Custom Branding configuration settings, see Custom-Branding Apps Manager.

  3. Click Save.

  4. Select Apps Manager.

  5. Select the Enable Invitations checkbox to enable invitations in Apps Manager. Space Managers can invite new users for a given space, Org Managers can invite new users for a given org, and Admins can invite new users across all orgs and spaces. For more information, see the Invite New Users section of the Managing User Roles with Apps Manager topic.

  6. Enable the Display Marketplace Service Plan Prices checkbox to display the prices for your services plans in the Marketplace.

  7. Enter the Supported currencies as JSON to appear in the Marketplace. Use the format { "CURRENCY-CODE": "SYMBOL" }. This defaults to { "usd": "$", "eur": "€" }.

  8. Use Product Name, Marketplace Name, and Customize Sidebar Links to configure page names and sidebar links in the Apps Manager and Marketplace pages. You may configure up to 10 links in the Apps Manager sidebar.

  9. The Apps Manager Memory Usage (MB) field sets the memory limit with which to deploy the Apps Manager app. Use this field to increase the memory limit if the app fails to start with an out of memory error. Enter the value in MB. To use the default value of 128 MB, leave this field blank.

  10. The Invitations Memory Usage (MB) field sets the memory limit with which to deploy the Invitations app. Use this field to increase the memory limit if the app fails to start with an out of memory error. Enter the value in MB. To use the default value of 256 MB, leave this field blank.

  11. The Apps Manager Polling Interval field provides a temporary fix if Apps Manager usage degrades Cloud Controller response times. In this case, you can use this field to reduce the load on the Cloud Controller and ensure Apps Manager remains available while you troubleshoot the Cloud Controller. Pivotal recommends that you do not keep this field modified as a long-term fix because it can degrade Apps Manager performance. Optionally, you can:

    • Increase the polling interval above the default of 30 seconds. Enter the value in seconds.

      Note: If you enter a value between 0 and 30, the field is automatically set to 30.

    • Disable polling by entering 0. This stops Apps Manager from refreshing data automatically, but users can update displayed data by reloading Apps Manager manually.
  12. The App Details Polling Interval field provides an additional way to reduce the load on the Cloud Controller when the Apps Manager Polling Interval field is not sufficient. This field controls the rate at which Apps Manager polls for data when a user views the Overview page of an app. Pivotal recommends that you do not keep this field modified as a long-term fix because it can degrade Apps Manager performance. Optionally, you can:

    • Increase the polling interval above the default of 10 seconds. Enter the value in seconds.

      Note: If you enter a value between 0 and 30, the field is automatically set to 10.

    • Disable polling by entering 0. This stops Apps Manager from refreshing data automatically, but users can update displayed data by reloading Apps Manager manually.
  13. To enable multi-foundation support for Apps Manager, enter a JSON object containing all the foundations you want Apps Manager to manage in Multi-Foundation Configuration (BETA). Enabling multi-foundation support allows you to manage orgs, spaces, apps, and service instances from multiple PCF foundations from a single Apps Manager interface. For more information, see Configuring Multi-Foundation Support in Apps Manager.

  14. Click Save.

(Optional) Configure Email Notifications

In the Email Notifications pane, you can enable end-user self-registration. PAS uses SMTP to send invitations and confirmations to Apps Manager users. If you do not need this service, leave this pane blank and disable the Notifications and Notifications UI errands in the Errands pane.

To configure the Email Notifications pane:

  1. Select Email Notifications.

  2. For From Email, enter the email address from which email notifications are sent.

  3. For SMTP Server Address, enter the SMTP address of the server that sends emails.

  4. For SMTP Server Port, enter the port of the SMTP server that sends email notifications.

    Note: For GCP, you must use port 2525. Ports 25 and 587 are not allowed on GCP Compute Engine.

  5. For SMTP Server Credentials, enter the username and password for the SMTP server that sends email notifications.

  6. Enable the SMTP Enable Automatic STARTTLS checkbox if you want your SMTP server to automatically create a secure TLS connection when sending emails.

  7. Verify your authentication requirements with your email administrator and use the SMTP Authentication Mechanism dropdown to select None, Plain, or CRAMMD5. If you have no SMTP authentication requirements, select None.

  8. If you selected CRAMMD5 as your authentication mechanism, enter a secret in the SMTP CRAMMD5 secret field.

  9. Click Save.

Note: If you do not configure the SMTP settings using this form, the administrator must create orgs and users using the cf CLI. For more information, see Creating and Managing Users with the cf CLI.

(Optional) Configure App Autoscaler

In the App Autoscaler pane, you configure the App Autoscaler service. To use App Autoscaler, you must create an instance of the service and bind it to an app. To create an instance of App Autoscaler and bind it to an app, see the Set Up App Autoscaler section in the Scaling an App Using App Autoscaler topic.

To configure the App Autoscaler pane:

  1. Select App Autoscaler.

  2. For Autoscaler Instance Count, enter the number of instances of the App Autoscaler service you want to deploy. The default value is 3. For high availability, set this number to 3 or higher. You should set the instance count to an odd number to avoid split-brain scenarios during leadership elections. Larger environments may require more instances than the default number.

  3. For Autoscaler API Instance Count, enter the number of instances of the App Autoscaler API you want to deploy. The default value is 1. Larger environments may require more instances than the default number.

  4. For Metric Collection Interval, enter how many seconds of data collection you want App Autoscaler to evaluate when making scaling decisions. The minimum interval is 60 seconds, and the maximum interval is 3600 seconds. The default value is 120. Increase this number if the metrics you use in your scaling rules are emitted less frequently than the existing metric collection interval.

  5. For Scaling Interval, enter in seconds how frequently App Autoscaler evaluates an app for scaling. The minimum interval is 15 seconds, and the maximum interval is 120 seconds. The default value is 35.

  6. To enable verbose logging for App Autoscaler, enable the Verbose Logging checkbox. Verbose logging is disabled by default. Enable this checkbox to see more detailed logs. Verbose logs show specific reasons why App Autoscaler scaled the app, including information about minimum and maximum instance limits, App Autoscaler’s status. For more information about App Autoscaler logs, see the App Autoscaler Events and Notifications section of the Scaling an App Using App Autoscaler topic.

  7. If you do not want the Autoscaler API to reuse HTTP connections, enable the Disable API Connection Pooling checkbox. This may be necessary if your front end idle timeout for the Gorouter is set to a low value, such as 1 second. For more information, see Configuring Frontend Idle Timeout for Gorouter and HAProxy.

  8. To enable email notifications of Autoscaler events, enable the Enable Notifications checkbox. For more information about managing email notifications, see the App Autoscaler Events and Notifications section of the Scaling an App Using App Autoscaler topic.

  9. Click Save.

Configure Cloud Controller

In the Cloud Controller pane, you configure the Cloud Controller.

To configure the Cloud Controller pane:

  1. Select Cloud Controller.

  2. Enter your Cloud Controller DB Encryption Key if all of the following are true:

  3. CF API Rate Limiting prevents API consumers from overwhelming the platform API servers. Limits are imposed on a per-user or per-client basis and reset on an hourly interval.

    To disable Cloud Foundry API rate limiting, select Disable under Enable CF API Rate Limiting. To enable Cloud Foundry API rate limiting:

    1. Under Enable CF API Rate Limiting, select Enable.
    2. For General Limit, enter the number of requests a user or client is allowed to make over an hour-long interval for all endpoints that do not have a custom limit. The default value is 2000.
    3. For Unauthenticated Limit, enter the number of requests an unauthenticated client is allowed to make over an hour-long interval. The default value is 100.
  4. (Optional) Enter in seconds your Database Connection Validation Timeout. By default, the setting is 3600 seconds or 60 minutes. You can enter -1 to cause Cloud Controller to make an additional query to the database whenever connections are checked out from the pool. Choosing -1 has performance implications.

  5. (Optional) Enter in seconds your Database Read Timeout. By default, the setting is 3600 seconds or 60 minutes.

  6. (Optional) Enter the Age in days after which completed tasks will be pruned from the Cloud Controller Database.

  7. Click Save.

Configure Smoke Tests

In the Smoke Tests pane, you choose where to run smoke tests. The Smoke Tests errand runs basic functionality tests against your PAS deployment after an installation or update. In the Errands pane, you can choose whether or not to run the Smoke Tests errand.

To configure the Smoke Tests pane:

  1. Select Smoke Tests.

  2. If you have a shared apps domain, select Temporary space within the system organization, which creates a temporary space within the system org for running smoke tests and deletes the space afterwards. Otherwise, select Specified org and space and complete the fields to specify where you want to run smoke tests:

    • For Organization, enter the org you want PAS to use when running smoke tests.
    • For Space, enter the space you want PAS to use when running smoke tests.
    • For Domain, enter the domain you want PAS to use when running smoke tests.
  3. Click Save.

(Optional) Configure Advanced Features

The Advanced Features pane includes new functionality that may have certain constraints. Although these features are fully supported, Pivotal recommends caution when using them in production environments.

Diego Cell Memory and Disk Overcommit

If your apps do not use the full allocation of disk space and memory set in the Resource Config tab, you might want use this feature. These fields control the amount to overcommit disk and memory resources to each Diego Cell VM.

For example, you might want to use the overcommit if your apps use a small amount of disk and memory capacity compared to the amounts set in the Resource Config settings for Diego Cell.

Note: Due to the risk of app failure and the deployment-specific nature of disk and memory use, Pivotal has no recommendation about how much, if any, memory or disk space to overcommit.

To enable overcommit:

  1. Select Advanced Features.

  2. Enter in MB the total desired amount of Diego cell memory value in the Cell Memory Capacity (MB) field. Refer to the Diego Cell row in the Resource Config tab for the current Diego Cell memory capacity settings that this field overrides.

  3. Enter in MB the total desired amount of Diego cell disk capacity value in the Cell Disk Capacity (MB) field. Refer to the Diego Cell row in the Resource Config tab for the current Diego Cell disk capacity settings that this field overrides.

  4. Click Save.

Note: Entries made to each of these two fields set the total amount of resources allocated, not the overage.

Whitelist for Non-RFC-1918 Private Networks

Some private networks require extra configuration so that internal file storage (WebDAV) can communicate with other PCF processes.

The Whitelist for non-RFC-1918 Private Networks field is provided for deployments that use a non-RFC 1918 private network. This is typically a private network other than 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16.

Most PCF deployments do not require any modifications to this field.

To add your private network to the whitelist:

  1. Select Advanced Features.

  2. Append a new allow rule to the existing contents of the Whitelist for non-RFC-1918 Private Networks field. Include the word allow, the network CIDR range to allow, and a semi-colon (;) at the end. For example: allow 172.99.0.0/24;

  3. Click Save.

CF CLI Connection Timeout

The CF CLI Connection Timeout field allows you to override the default five-second timeout of the Cloud Foundry Command Line Interface (cf CLI) used within your PCF deployment. This timeout affects the cf CLI command used to push PAS errand apps such as Notifications, Autoscaler, and Apps Manager.

Set the value of this field to a higher value, in seconds, if you are experiencing domain name resolution timeouts when pushing errands in PAS.

To modify the value of the CF CLI connection timeout:

  1. Select Advanced Features.

  2. Enter a value, in seconds, to the CF CLI Connection Timeout field.

  3. Click Save.

Enable TLS for Internal System Database

You can optionally enable TLS for clients of the internal system database. This feature is in beta, and the checkbox is disabled by default. For more information about the internal system database, see Managing Internal Databases.

To enable TLS for clients of the internal system database:

  1. Select Advanced Features.

  2. Select the BETA: Enable TLS for internal system database checkbox.

  3. Click Save.

Database Connection Limits

You can configure the maximum number of concurrent database connections that Diego and container networking components can have.

To configure the maximum number of concurrent database connections:

  1. Select Advanced Features.

  2. Enter a value in each field beginning with Maximum number of open connections… for a given component. The placeholder values for each field are the default values.

  3. Click Save.

When there are not enough connections available, such as during a time of heavy load, components may experience degraded performance and sometimes failure. To resolve or prevent this, you can increase and fine-tune database connection limits for the component.

Warning: Decreasing the value of this field for a component may affect the amount of time it takes for it to respond to requests.

Rolling App Deployments

You can optionally disable rolling app deployments. For more information, see Rolling App Deployments (Beta).

To disable rolling app deployments:

  1. Select Advanced Features.

  2. Select the Disable Zero Downtime App Deployments checkbox.

  3. Click Save.

(Optional) Configure Metric Registrar

Note: The Metric Registrar is bundled with PAS, and you configure it within the PAS tile. You do not install and configure Metric Registrar as a separate product tile.

In the Metric Registrar pane, you configure the Metric Registrar. Metric Registrar allows the conversion of structured logs into metrics. It also scrapes metrics endpoints and forwards the metrics to Loggregator. If enabled, Pivotal recommends also enabling the Metric Registrar Smoke Test errand.

Disable the Metric Registrar

The Metric Registrar is enabled by default.

To disable the Metric Registrar:

  1. Select Metric Registrar.

  2. Clear the Enable Metric Registrar checkbox.

Edit Default Scraping Interval

The scraping interval defines how often the Metric Registrar polls custom metric endpoints. The default is 35 seconds.

To edit the Metric Registrar scraping interval:

  1. Select Metric Registrar.

  2. Edit the Endpoint Scraping Interval field.

Add Blacklisted Tags

To prevent the Metric Registrar from consuming the value of a metric or event tag, you can add the tag to the Blacklisted Tags field. For example, if you tag your metrics with a customer_id, you may want to add customer_id to the list of blacklisted tags.

By default, the following tags are blacklisted to prevent interference with other products like PCF Metrics that use and rely on such tags.

  • deployment
  • job
  • index
  • id

To prevent the Metric Registrar from consuming the value of a metric or event tag:

  1. Select Metric Registrar.

  2. Add the desired tags to the Blacklisted Tags field in a comma-separated list.

Configure Errands

Errands are scripts that Ops Manager runs automatically when it installs or uninstalls a product, such as a new version of PAS. There are two types of errands: post-deploy errands run after the product is installed, and pre-delete errands run before the product in uninstalled.

By default, Ops Manager always runs all errands.

In the Errands pane, you can change these run rules. For each errand, you can select On to run it always or Off to never run it.

For more information about how Ops Manager manages errands, see Managing Errands in Ops Manager.

Note: Several errands, such as App Autoscaler and Notifications, deploy apps that provide services for your deployment. When one of these apps is running, selecting Off for the corresponding errand on a subsequent installation does not stop the app.

  • Smoke Test Errand verifies that your deployment can:

    • Push, scale, and delete apps
    • Create and delete orgs and spaces
  • Usage Service Errand deploys the Pivotal Usage Service app, which Apps Manager depends on.

  • Apps Manager Errand deploys Apps Manager, a dashboard for managing apps, services, orgs, users, and spaces. Until you deploy Apps Manager, you must perform these functions through the cf CLI. After Apps Manager has been deployed, Pivotal recommends setting this errand to Off for subsequent PAS deployments. For more information about Apps Manager, see Getting Started with Apps Manager.

  • Notifications Errand deploys an API for sending email notifications to your PCF platform users.

    Note: The Notifications app requires that you configure SMTP with a username and password, even if you set the value of SMTP Authentication Mechanism to none. To configure SMTP with a username and password, see Configure Email Notifications.

  • Notifications UI Errand deploys a dashboard for users to manage notification subscriptions.

  • App Autoscaler Errand pushes the App Autoscaler app, which enables you to configure your apps to automatically scale in response to changes in their usage load. For more information, see Scaling an App Using App Autoscaler.

  • App Autoscaler Smoke Test Errand runs smoke tests against App Autoscaler.

  • NFS Broker Errand pushes the NFS Broker app, which supports NFS Volume Services for PAS. For more information, see the Enable NFS Volume Services section of the Enabling Volume Services topic.

  • Metric Registrar Smoke Test Errand verifies the Metric Registrar can access custom metrics emitted by an app and convert them into Loggregator metrics.

    Note: The Metric Registrar Smoke Test errand runs only if the Metric Registrar is enabled. For more information about configuring the Metric Registrar, see Configuring the Metric Registrar.

  • SMB Broker Application Errand pushes the SMB Broker app, which supports SMB Volume Services for PAS. For more information, see the Enable SMB Volume Services section of the Enabling Volume Services topic.

Configure Resources

In the Resource Config pane, you must associate load balancers with the VMs in your deployment to enable traffic. For more information, see Configure Load Balancing for PAS.

Note: This step does not apply to vSphere deployments.

(Optional) Scale Down and Disable Resources

Note: The Resource Config pane has fewer VMs if you are installing Small Footprint PAS. For more information, see Getting Started with Small Footprint PAS.

Note: Small Footprint PAS does not default to a highly available configuration. It defaults to the minimum configuration. To make Small Footprint PAS highly available, scale the Compute, Router, and Database VMs to 3 instances and scale the Control VM to 2 instances.

PAS defaults to a highly available resource configuration. However, you may need to perform additional procedures to make your deployment highly available. For more information, see High Availability in Cloud Foundry and Scaling PAS.

If you do not want a highly available resource configuration, you must scale down your instances manually by navigating to the Resource Config section and using the dropdowns under Instances for each job.

By default, PAS also uses an internal filestore and internal databases. If you configure PAS to use external resources, you can disable the corresponding system-provided resources in Ops Manager to reduce costs and administrative overhead.

To disable specific VMs in Ops Manager:

  1. Select Resource Config.

  2. If you configured PAS to use an external S3-compatible filestore, enter 0 in Instances in the File Storage field.

  3. If you selected External when configuring the UAA, System, and CredHub databases, edit the following fields:

    • MySQL Proxy: Enter 0 in Instances.
    • MySQL Server: Enter 0 in Instances.
    • MySQL Monitor: Enter 0 in Instances.
  4. If you disabled TCP routing, enter 0 Instances in the TCP Router field.

  5. If you are not using HAProxy, enter 0 Instances in the HAProxy field.

  6. Click Save.

Download Stemcell

This step is only required if your Ops Manager does not already have the stemcell version PAS requires. For more information about importing stemcells, see Importing and Managing Stemcells.

To download the stemcell version PAS requires:

  1. Open the Stemcell product page in the Pivotal Network. You may have to log in.

  2. Download the appropriate stemcell version targeted for your IaaS.

  3. Navigate to Stemcell Library in the Installation Dashboard.

  4. Click Import Stemcell to import the downloaded stemcell .tgz file.

  5. When prompted, enable the Ops Manager product checkbox to stage your stemcell.

  6. Click Apply Stemcell to Products.

Complete the PAS Installation

To complete your installation of PAS:

  1. Click the Installation Dashboard link to return to the Installation Dashboard.

  2. Click Review Pending Changes, then Apply Changes. The install process generally requires a minimum of 90 minutes to complete. The following image shows the Changes Applied window that displays when the installation process successfully completes.

    At the top of the pop-up window is a teal checkmark and the words 'Changes Applied'. In the upper-right corner is a gray circle with a white X in the middle. Below 'Changes Applied' is the text 'Ops Manager Director was successfully installed.' Below this text is the italicized text 'We recommend that you export a backup of this installation from the actions menu.' Below this text are two buttons, a gray rectangular button labeled 'Close' and a blue rectangular button labeled 'Return to Installation Dashboard'.