LATEST VERSION: 1.12 - CHANGELOG

Pivotal Cloud Foundry v1.12

• v1.11
• v1.10
• v1.9
• v1.8
• v1.7
• v1.6
• v1.5
• v1.4
• v1.3
• v1.2

Download Ask for Help PCF Knowledge Base PDF

Pivotal Elastic Runtime v1.12 Release Notes

Pivotal Cloud Foundry is certified by the Cloud Foundry Foundation for 2017.

Read more about the certified provider program and the requirements of providers.

Releases

1.12.4

This release includes the new Small Footprint Elastic Runtime. This is a new product that reorganizes the components in the Elastic Runtime into a much smaller deployment. Operators will be able to use the Small Footprint Elastic Runtime to deploy a working Cloud Foundry installation in as few as 4 VMs. Please checkout the documentation for more details.

• [Security Fix] Bumps cflinuxfs2-release to v1.158.0 to resolve multiple security issues. Release Notes
• [Bug Fix] Bumps usage-service-release to v663.0.4 to resolve instability caused by low memory constraints.
• [Bug Fix] Bumps apps-manager-release to v662.0.14 to resolve an issue where Docker applications would crash Apps Manager.

1.12.3

• [Bug Fix] Bumps scalable-syslog-release to v11. Release Notes.
• [Bug Fix] Bumps usage-service-release to v663.0.3 to resolve an issue that prevented users from using a custom CA-signed certificate.
• [Security Improvement] Bumps garden-runc-release to v1.9.4. Release Notes.
• [Bug Fix] Bumps uaa-release to v45.3. Release Notes.

1.12.2

• [Security Fix] Bumps cflinuxfs2-release to v1.156.0 to resolve multiple security issues. Release Notes
• [Security Fix] Resolves an issue with an incorrect Host header being set on incoming requests through the Router CVE Notice.
• [Security Fix] Bumps cf-mysql-release to v36.6 to patch vulnerabilities in Bundler and RubyGems CVE-2016-7954 CVE-2017-0902
• [Security Fix] Resolves a remote code execution security vulnerability when the zip program is executed by the Cloud Controller.
• [Bug Fix] Bumps haproxy-boshrelease to v8.4.1 to resolve an issue with certificate/key concatenation Release Notes.
• [Bug Fix] Bumps apps-manager-release to v662.0.12 to patch the following:
  • AppsManager will now show all Application Security Group rules.
  • Fixes a bug that prevented arbitrary schema parameters from working when provisioning a new service from the flyout component.
  • When a space has zero members, a message is displayed indicating that the space has no members.
  • When creating a new org, the current user is added to that org as a user and an org manager.
• [Bug Fix] Resolves a bug caused by a missing “selector” option on the “Networking” tab. Operators who had previous chosen their Networking Point of Entry as a non-TLS external Load Balancer would experience this as a tile that looked fully configured, but could not be deployed due to an OpsMan configuration issue.
• Changes the default “Router Max Connections Per Backend” from unlimited to 500.
• [Feature] Operators can now configure a “Frontend Idle Timeout” for the Router and HAProxy. The default timeout is 900 seconds.
• [Feature] Bumps diego-release to v1.25.3 to include support for Azure MySQL Release Notes.
• [Feature] Patches cf-networking-release to include support for Azure MySQL
• Bumps buildpacks to the following versions:
  • dotnet-core-buildpack
  • nodejs-buildpack
  • php-buildpack
  • python-buildpack
  • ruby-buildpack
  • staticfile-buildpack
• [Stability Improvement] Changes the default for Galera MySQL state snapshot transfers (SST). Automatic SST is now enabled by default. Operators can disable this feature by visiting the “Internal MySQL” tab and checking the “Prevent node auto re-join” checkbox.

1.12.1

• [Security Fix] Bumps stemcell to v3445.11 to address USN-3420-2.
• [Security Fix] Bumps cflinuxfs-release to v1.155.0 to address USN-3415-1.

1.12.0

How to Upgrade

The procedure for upgrading to Pivotal Cloud Foundry (PCF) Elastic Runtime v1.12 is documented in the Upgrading Pivotal Cloud Foundry topic.

When upgrading to v1.12, be aware of the following upgrade considerations:

• You must upgrade first to a version of Elastic Runtime v1.11.x to successfully upgrade to v1.12.

• If your existing PCF v1.11.x installation includes both PCF Runtime for Windows and MySQL for PCF v1.x, you must upgrade to MySQL for PCF v1.10.3 or later before you upgrade to PCF Elastic Runtime v1.12. For instructions on how to upgrade MySQL for PCF, see the MySQL for PCF documentation.

• Some partner service tiles may be incompatible with PCF v1.12. Pivotal is working with partners to ensure their tiles are being updated to work with the latest versions of PCF.
  
  For information about which partner service releases are currently compatible with PCF v1.12, review the appropriate partners services release documentation at https://docs.pivotal.io, or contact the partner organization that produces the tile.

New Features in Elastic Runtime v1.12.0

This section describes new features of the release.

Multiple Buildpack App Support

Developers can now push apps that take advantage of multiple buildpacks simultaneously. You can upgrade apps to use multiple buildpacks with the cf v3-push command. This makes binaries, libraries, and language modules provided by all specified buildpacks available to the app. The final buildpack specified controls how the app starts.

Multiple buildpack support adds flexibility to the Cloud Foundry app development model. You can now use the official Cloud Foundry buildpacks together to support polyglot (multiple language) apps. Additionally, you can specify custom buildpacks before official Cloud Foundry buildpacks to supply dependencies that previously had to be provided with apps.

Granular cf push Commands and Procfile Support

This release includes experimental commands that provide developers with the ability to better orchestrate app deployment workflows. Additionally, developers can supply a Procfile with their app to run multiple long-lived processes using a single codebase. For more information, see Using Experimental cf CLI Commands.

Migration of Internal Credentials to CredHub

Internal credentials, the secret and simple_credentials that Elastic Runtime uses for inter-component communication, are generated and stored in CredHub instead of Ops Manager. This is part of an ongoing effort to migrate all credentials to CredHub, which will reduce the amount of places credentials are stored, aid in credential rotation, and increase security.

If you want to access the following credentials, you must use the CredHub CLI or the Ops Manager API instead of the Credentials tab of the Elastic Runtime tile. For instructions on how to to retrieve Elastic Runtime credentials, see Retrieving Credentials from Your Deployment.

• .mysql.autoscale_credentials
• .mysql.ccdb_credentials
• .mysql.diag_agent_credentials
• .mysql.diegodb_credentials
• .mysql.locketdb_credentials
• .mysql.monitordb_credentials
• .mysql.mysql_backup_server_credentials
• .mysql.mysql_bootstrap_credentials
• .mysql.networkpolicyserverdb_credentials
• .mysql.nfsvolume_credentials
• .mysql.notifications_credentials
• .mysql.pivotal_account_credentials
• .mysql.routingdb_credentials
• .mysql.silkdb_credentials
• .mysql.uaadb_credentials
• .nfsbrokerpush.nfs_broker_push_credentials
• .cloud_controller.bulk_api_credentials
• .cloud_controller.internal_api_user_credentials
• .cloud_controller.staging_upload_credentials
• .mysql.app_usage_credentials
• .mysql.cluster_health_user
• .mysql.galera_sidecar_user
• .mysql.mysql_admin_credentials
• .mysql_proxy.dashboard_credentials
• .nfs_server.blobstore_credentials
• .router.status_credentials

Introducing GrootFS

GrootFS is the new container image management plugin for Garden-runC. It helps with the filesystem isolation of Garden-runC containers, image caching, and disk quota enforcement. GrootFS replaces the previous built-in functionality, which used an obsolete layer filesystem (AUFS) that lacks support from the Linux Kernel community. Additionally, GrootFS uses OCI standards for container images.

For more information about GrootFS in PCF, see the following topics:

• Understanding Container Security
• Component: Garden

Application Instance Identity Credentials

The instance identity system in Diego provides each app container with a PEM-encoded X.509 certificate and PKCS #1 RSA private key. The values of the environment variables CF_INSTANCE_CERT and CF_INSTANCE_KEY contain the absolute paths to the certificate and private key files. The validity period is 3 years for the Instance Identity root and 2 years for the intermediate CA certificates.

See the App Instance Container Identity Credentials section for more information.

Simplified TLS Configuration

The point of entry options on the Elastic Runtime Networking pane have been restructured to be more understandable and flexible. Operators no longer need to configure the Router or HAProxy separately as both components are configured using the same options. This includes the following changes:

• Gorouter and HAProxy always listen for TLS requests. You provide an SSL certificate for both Gorouter and HAProxy using a single field.
• HAProxy forwards all requests to Gorouter over TLS by default. You can optionally disable this feature.
• You can configure the minimum version of TLS for Gorouter and HAProxy with a single field.
• You can provide a list of CAs to HAProxy for it to validate the Gorouter certificate.
• You can optionally disable the HTTP listener for both Gorouter and HAProxy with a single checkbox.
• You can specify TLS cipher suites for HAProxy and Gorouter independently.

See the Elastic Runtime installation instructions for your IaaS for more information.

Mutual TLS Headers on Inbound Application Traffic

Gorouter can now forward the X-Forwarded-Client-Cert header to app instances when provided. Alternatively, operators can configure Gorouter to forward the header only when the mutual TLS connection from the client can be validated. Additionally, operators may now configure Gorouter to overwrite the XFCC header with the client certificate received in mTLS handshakes.

This configuration is available in the Networking pane under Configure the CF Router support for the X-Forwarded-Client-Cert header. See the Elastic Runtime installation instructions for your IaaS for more information.

Secure Communication Between Cloud Controller and Diego

In previous versions of PCF, the Diego Brain VM ran the Cloud Controller Bridge component, which translated Cloud Controller requests into Diego API commands. The Cloud Controller Bridge conveyed communications between the Cloud Controller and Diego over plaintext HTTP. In PCF v1.12, the Cloud Controller and Diego communicate directly via secure TLS protocol. This change streamlines and secures internal communications, and removes the Cloud Controller Bridge.

Securing this communication path will require a second deployment after completing your upgrade to PCF v1.12. Follow the steps in our upgrade guide to secure your PCF installation.

Secure Communication Between Diego and Loggregator

Diego Cells now use the Metron API v2. This gRPC-based API supports mutual TLS authentication and secures the communication path between the Diego rep and Loggregator.

Scaling Loggregator

As part of this release, the Loggregator team has provided guidelines for scaling the Loggregator system. For more information, see Scaling Loggregator and Scaling Nozzles and Operator Guidebook.

HAProxy Release

This release removes the old HAProxy job, which was the last remaining component from cf-release. It now uses the newly incubated haproxy-boshrelease. This replacement allows Elastic Runtime to expose new HAProxy features, such as request filtering.

HAProxy Request Filtering

If your PCF deployment uses HAProxy and you want it to receive traffic only from specific sources, you can use the Protected Domains and Trusted CIDRs fields in the Networking Pane of the Elastic Runtime tile. A key use case for this feature is when a deployment must only allow requests to the system domain from a private network or VPN. See the Elastic Runtime installation instructions for your IaaS for more information.

Container-to-Container Networking Updates

Container-to-container networking is now always enabled. The commands are integrated with the cf CLI and now include the option to specify a port range when adding and removing policies. See Create Policies for Container-to-Container Networking.

Support for Logging All App Traffic

Operators can enable logging of all accepted and denied packets due to ASGs or container-to-container networking policies. This provides more visibility into app traffic, including denied traffic.

Operators configure this global logging in the Networking pane of the Elastic Runtime tile under the Log traffic for all accepted/denied application packets field. See the App Traffic Logging section for more information.

Orphaned Blob Cleanup

The Cloud Controller now scans the blobstore on a regular interval to identify and remove orphaned blobs. For more information, see the Blobstore section of the Cloud Controller topic.

Router Sharding Mode

This release includes support for router sharding between the Elastic Runtime and Isolation Segment tiles. Operators can choose to have the Elastic Runtime tile routers only acknowledge requests from apps deployed within the its Cells, or reject requests for any isolation segment.

You can configure this feature using the following fields:

• Elastic Runtime tile: Routers reject requests for Isolation Segments checkbox
• Isolation Segment tile: Router Sharding Mode selector

See the Elastic Runtime installation instructions for your IaaS for more information.

Gorouter Max Connection Configuration

Operators can limit the number of app instance connections to the backend using the Max Connections Per Backend field in the Networking pane of the Elastic Runtime tile. This field can help prevent malicious apps from consuming all available Gorouter resources. See the Elastic Runtime installation instructions for your IaaS for more information.

Authenticating with Google Container Registry (GCR) to Push Docker Images

For PCF v1.12 and later, Pivotal recommends authenticating with GCR using the procedure documented in the following section: Push a Docker Image from Google Container Registry (GCR). The alternative authentication mechanism provided by GCR passes a short lived (12 hours) access token to PCF. This enables PCF to pull images from GCR during the initial cf push, but subsequent restage, push, or rescheduling operations fail once the access token expires.

NFSV3 Volume Services with LDAP

Operators can now configure LDAP for NFSv3 volume services. Using LDAP secures the NFSV3 volume service by preventing a developer from binding to an NFS share using an arbitrary UID and potentially gaining access to sensitive data stored by another user or app. If you enable LDAP support, developers must provide credentials for any user they wish to bind as. See Enabling NFS Volume Services.

Metrics for MySQL

The internal MySQL job included in Elastic Runtime now emits metrics. See the Elastic Runtime MySQL KPIs.

CloudFormation Template Improvements

This release includes an improved CloudFormation template file available with the Elastic Runtime tile on Pivotal Network. The new template creates three availability zones, a load balancer for TCP routing, and the Ops Manager VM. For updated installation instructions, see Installing PCF on AWS Using CloudFormation.

Diego Cell Max-in-Flight Default

This release lowers the default max-in-flight percentage on Diego Cells to 4%. Previously, this value was set to 10%, which can cause the following issues in larger environments:

• Many simultaneous VM creates/deletes and BOSH blob updates placing significant stress on the underlying infrastructure, especially on vSphere which has a greater probability of being under-provisioned.
• Cells that are draining are no longer available for allocation, resulting in a 10% decrease in total memory and disk capacity during deployment. This can cause deployments to no longer have sufficient total capacity to run all work, or to have insufficient headroom to place larger workloads successfully.

Operators can still use the Ops Manager API to configure this setting to fit their needs. For more information about this property, see Managing Diego Cell Limits During Upgrade.

Removal of etcd

This release removes the etcd server VMs from the PCF deployment. Operators must ensure they are deploying service tiles that are known to be compatible with PCF Elastic Runtime 1.12.

Removal of Postgres

This release removes the legacy Postgres databases for the Cloud Controller and UAA. If your deployment was originally installed before PCF 1.6 and still uses Postgres, you must contact your dedicated Support Engineer or Platform Architect for assistance migrating your Cloud Controller and UAA databases to MySQL. They will have access to the PostgreSQL-to-MySQL Migrator tool and instructions on Pivotal Network.

If you do not migrate to MySQL before upgrading to Elastic Runtime 1.12, the upgrade will fail.

Apps Manager: In-context Service Creation

Developers can create services without leaving the app or space view for an accelerated workflow.

Apps Manager: Service Configuration Parameter Discovery

When creating a new service in Apps Manager, developers can discover additional parameter options as fields, or a JSON editor that enables them to define the parameters.

Known Issues

Manual CredHub Restart Required During an Elastic Runtime Redeploy

In Elastic Runtime v1.12.0, the BOSH Backup and Restore (BBR) script does not restart the CredHub process. When following the Restoring Pivotal Cloud Foundry from Backup with BBR procedure, the Elastic Runtime redeploy fails after clicking Apply Changes since CredHub requires a restart.

To work around this issue, manually restart the CredHub process on the BOSH Director by running monit restart credhub, then click Apply Changes.

For more information, see the corresponding Knowledge Base article.

Lack of Autoscaler Scaling

You cannot scale the Autoscaler job to greater than one instance.

About Advanced Features

The Advanced Features section of the Elastic Runtime tile includes new functionality that may have certain constraints.

Although these features are fully supported, Pivotal recommends caution when using them in production.