Pivotal Application Service v2.6 Release Notes

Pivotal Cloud Foundry is certified by the Cloud Foundry Foundation for 2019.

Read more about the certified provider program and the requirements of providers.


Releases

2.6.4

Release Date: 08/15/2019

  • [Security Fix] Upgrade Envoy to Fix Security Vulnerabilities
  • [Security Fix] Upgrade libseccomp in bpm to 2.4.1 to address CVE-2019-9893
  • [Bug Fix] Keep application navigation from overlapping with buildpack information of Spring applications in Apps Manager
  • [Bug Fix] Fix horizontal scrolling in Apps Manager for smaller browser windows
  • [Bug Fix] Pass through arbitrary parameters when binding a service to an app in Apps Manager
  • [Bug Fix] Show all contexts for Spring Boot actuator mappings in Apps Manager, not just mappings that have the ‘application’ context
  • [Bug Fix] Fix bug in Apps Manager where Spring Boot actuator trace tab data was not shown
  • [Bug Fix] Improve output of Garden diagnostic tool (i.e. dontpanic) and increase resiliency in edge cases through improvements in containerd
  • [Bug Fix] Users should only get an external mesh domain seeded when the istio service mesh is enabled.
  • Bump ubuntu-xenial stemcell to version 315.72
  • Bump bpm to version 1.1.1
  • Bump capi to version 1.80.5
  • Bump cf-autoscaling to version 221
  • Bump cf-smoke-tests to version 40.0.116
  • Bump garden-runc to version 1.19.5
  • Bump istio to version 1.0.2
  • Bump push-apps-manager-release to version 669.0.10
  • Bump push-usage-service-release to version 670.0.7
Component Version
ubuntu-xenial stemcell315.72
backup-and-restore-sdk1.16.0
binary-offline-buildpack1.0.33
bosh-dns-aliases0.0.3
bosh-system-metrics-forwarder0.0.18
bpm1.1.1
capi1.80.5
cf-autoscaling221
cf-backup-and-restore0.0.11
cf-cli1.16.0
cf-networking2.22.2
cf-smoke-tests40.0.116
cf-syslog-drain10.2
cflinuxfs30.118.0
consul-drain0.0.3
consul198
credhub2.4.0
diego2.30.1
dotnet-core-offline-buildpack2.2.12
garden-runc1.19.5
go-offline-buildpack1.8.42
haproxy9.5.2
istio1.0.2
java-offline-buildpack4.20
leadership-election1.4
log-cache2.1.6
loggregator-agent3.14
loggregator105.5
mapfs1.1.5
metric-registrar1.0.4
mysql-monitoring9.3.0
nats26
nfs-volume2.2.2
nginx-offline-buildpack1.0.15
nodejs-offline-buildpack1.6.52
notifications-ui36
notifications57
php-offline-buildpack4.3.78
push-apps-manager-release669.0.10
push-usage-service-release670.0.7
pxc0.18.0
python-offline-buildpack1.6.36
r-offline-buildpack1.0.11
routing0.188.2
ruby-offline-buildpack1.7.42
silk2.22.2
smb-volume1.3.0
staticfile-offline-buildpack1.4.43
statsd-injector1.10.0
syslog11.4.0
uaa71.2

2.6.3

Release Date: 08/01/2019

  • [Security Fix] When a Spring Boot app has a route with unencrypted HTTP as the protocol, Apps Manager attempts requests via encrypted HTTPS
  • [Feature] Allow operator to configure service mesh domain
  • [Feature] Add option to allow queries to inactive MySQL servers so auditing and reporting queries can be made without impacting performance on the active MySQL node.
  • [Bug Fix] Fix issue in which Enable/Disable Autoscaling button in Apps Manager temporarily shows the wrong autoscaling state
  • [Bug Fix] Space Developer Networking Self Service checkbox in PAS tile configuration now gives proper permissions to Apps Manager users
  • [Bug Fix] Fix issue where services shared across orgs/spaces never load apps it is bound to from the other orgs/spaces on the Apps Manager service overview tab
  • [Bug Fix] Make search bar in Apps Manager case insensitive
  • [Bug Fix] Fix various Apps Manager UI bugs
  • [Bug Fix] Fix race condition when starting the PAS MySQL database that caused potential failures during upgrade/deploy
  • [Bug Fix] Fixes a regression causing mount bind configuration to be rejected by the SMB volume service broker
  • [Bug Fix] Creating a space via the V3 API generates an audit event
  • [Bug Fix] Fix issue in SMB startup scripts that can cause restart failure or inadvertent application data permission change
  • [Bug Fix] Fix evaluation of nfsbrokerpush.db.ca_cert property in nfs-volume-release when using external DB without TLS.
  • Bump ubuntu-xenial stemcell to version 315.70
  • Bump backup-and-restore-sdk to version 1.16.0
  • Bump binary-offline-buildpack to version 1.0.33
  • Bump capi to version 1.80.4
  • Bump cflinuxfs3 to version 0.118.0
  • Bump go-offline-buildpack to version 1.8.42
  • Bump java-offline-buildpack to version 4.20
  • Bump log-cache to version 2.1.6
  • Bump nfs-volume to version 2.2.2
  • Bump nginx-offline-buildpack to version 1.0.15
  • Bump nodejs-offline-buildpack to version 1.6.52
  • Bump php-offline-buildpack to version 4.3.78
  • Bump push-apps-manager-release to version 669.0.8
  • Bump pxc to version 0.18.0
  • Bump python-offline-buildpack to version 1.6.36
  • Bump r-offline-buildpack to version 1.0.11
  • Bump ruby-offline-buildpack to version 1.7.42
  • Bump smb-volume to version 1.3.0
Component Version
ubuntu-xenial stemcell315.70
backup-and-restore-sdk1.16.0
binary-offline-buildpack1.0.33
bosh-dns-aliases0.0.3
bosh-system-metrics-forwarder0.0.18
bpm1.0.4
capi1.80.4
cf-autoscaling219
cf-backup-and-restore0.0.11
cf-cli1.16.0
cf-networking2.22.2
cf-smoke-tests40.0.113
cf-syslog-drain10.2
cflinuxfs30.118.0
consul-drain0.0.3
consul198
credhub2.4.0
diego2.30.1
dotnet-core-offline-buildpack2.2.12
garden-runc1.19.1
go-offline-buildpack1.8.42
haproxy9.5.2
istio1.0.1
java-offline-buildpack4.20
leadership-election1.4
log-cache2.1.6
loggregator-agent3.14
loggregator105.5
mapfs1.1.5
metric-registrar1.0.4
mysql-monitoring9.3.0
nats26
nfs-volume2.2.2
nginx-offline-buildpack1.0.15
nodejs-offline-buildpack1.6.52
notifications-ui36
notifications57
php-offline-buildpack4.3.78
push-apps-manager-release669.0.8
push-usage-service-release670.0.6
pxc0.18.0
python-offline-buildpack1.6.36
r-offline-buildpack1.0.11
routing0.188.2
ruby-offline-buildpack1.7.42
silk2.22.2
smb-volume1.3.0
staticfile-offline-buildpack1.4.43
statsd-injector1.10.0
syslog11.4.0
uaa71.2

2.6.2

Release Date: 07/15/2019

  • [Security Fix] Fix high severity CVE in UAA: CVE-2019-3787
  • [Security Fix] UAA should prevent SCIM query injection attacks
  • Bump cf-smoke-tests to version 40.0.113
  • Bump cflinuxfs3 to version 0.113.0
  • Bump uaa to version 71.2
Component Version
ubuntu-xenial stemcell315.45
backup-and-restore-sdk1.15.1
binary-offline-buildpack1.0.32
bosh-dns-aliases0.0.3
bosh-system-metrics-forwarder0.0.18
bpm1.0.4
capi1.80.3
cf-autoscaling219
cf-backup-and-restore0.0.11
cf-cli1.16.0
cf-networking2.22.2
cf-smoke-tests40.0.113
cf-syslog-drain10.2
cflinuxfs30.113.0
consul-drain0.0.3
consul198
credhub2.4.0
diego2.30.1
dotnet-core-offline-buildpack2.2.12
garden-runc1.19.1
go-offline-buildpack1.8.40
haproxy9.5.2
istio1.0.1
java-offline-buildpack4.18
leadership-election1.4
log-cache2.1.4
loggregator-agent3.14
loggregator105.5
mapfs1.1.5
metric-registrar1.0.4
mysql-monitoring9.3.0
nats26
nfs-volume2.1.0
nginx-offline-buildpack1.0.13
nodejs-offline-buildpack1.6.51
notifications-ui36
notifications57
php-offline-buildpack4.3.77
push-apps-manager-release669.0.7
push-usage-service-release670.0.6
pxc0.16.0
python-offline-buildpack1.6.34
r-offline-buildpack1.0.10
routing0.188.2
ruby-offline-buildpack1.7.40
silk2.22.2
smb-volume1.1.0
staticfile-offline-buildpack1.4.43
statsd-injector1.10.0
syslog11.4.0
uaa71.2

2.6.1

  • [Security Fix] Bump UAA to address CVE-2019-3788
  • [Security Fix] Update CF CLI for Autoscaler
  • [Feature] Allow users to configure max package size so that they can upload packages larger than 2GB
  • [Feature Improvement] Add ability to configure max search depth for LDAP in UAA
  • [Feature Improvement] Set maximum database connection lifetime to 1 hour for Diego Locket component to reduce resource contention on PAS database
  • [Bug Fix] Fix missing “actee_name” for certain CAPI user role related audit events
  • [Bug Fix] Make sure logged-in users are rate-limited according to authenticated rate limit
  • [Bug Fix] Fix failure of inviting new members via Apps Manager in some networking configurations
  • [Bug Fix] Cause Apps Manager errand to fail if environment variable assignment fails
  • [Bug Fix] Fix issue where creating a new organization fails in Apps Manager
  • [Bug Fix] Fix credentials for service instances in Apps Manager that failed to display
  • [Bug Fix] Add labels to key value forms in Apps Manager to enhance accessibility
  • [Bug Fix] Generate valid form ids in Apps Manager to enhance accessibility
  • [Bug Fix] Ellipsify long names of service instances in the services tables of Apps Manager
  • [Bug Fix] Fix issue in which flyouts in Apps Manager did not open in Internet Explorer
  • [Bug Fix] Fix error that prevented sharing domains across organizations in Apps Manager
  • [Bug Fix] Add optional TTL pruning for TLS routes
  • [Bug Fix] Allow operators to omit backup bucket fields
  • [Bug Fix] diego_brain instances no longer update concurrently with diego_cell VMs to prevent application downtime in case of deployment update failure
  • [Bug Fix] Send Isolation Segment smoke test application requests on port 443
  • Bump ubuntu-xenial stemcell to version 315.45
  • Bump capi to version 1.80.3
  • Bump cf-autoscaling to version 219
  • Bump cf-cli to version 1.16.0
  • Bump cf-smoke-tests to version 40.0.109
  • Bump cflinuxfs3 to version 0.109.0
  • Bump diego to version 2.30.1
  • Bump dotnet-core-offline-buildpack to version 2.2.12
  • Bump go-offline-buildpack to version 1.8.40
  • Bump nginx-offline-buildpack to version 1.0.13
  • Bump nodejs-offline-buildpack to version 1.6.51
  • Bump php-offline-buildpack to version 4.3.77
  • Bump push-apps-manager-release to version 669.0.7
  • Bump python-offline-buildpack to version 1.6.34
  • Bump r-offline-buildpack to version 1.0.10
  • Bump routing to version 0.188.2
  • Bump ruby-offline-buildpack to version 1.7.40
  • Bump staticfile-offline-buildpack to version 1.4.43
Component Version
ubuntu-xenial stemcell315.45
backup-and-restore-sdk1.15.1
binary-offline-buildpack1.0.32
bosh-dns-aliases0.0.3
bosh-system-metrics-forwarder0.0.18
bpm1.0.4
capi1.80.3
cf-autoscaling219
cf-backup-and-restore0.0.11
cf-cli1.16.0
cf-networking2.22.2
cf-smoke-tests40.0.109
cf-syslog-drain10.2
cflinuxfs30.109.0
consul-drain0.0.3
consul198
credhub2.4.0
diego2.30.1
dotnet-core-offline-buildpack2.2.12
garden-runc1.19.1
go-offline-buildpack1.8.40
haproxy9.5.2
istio1.0.1
java-offline-buildpack4.18
leadership-election1.4
log-cache2.1.4
loggregator-agent3.14
loggregator105.5
mapfs1.1.5
metric-registrar1.0.4
mysql-monitoring9.3.0
nats26
nfs-volume2.1.0
nginx-offline-buildpack1.0.13
nodejs-offline-buildpack1.6.51
notifications-ui36
notifications57
php-offline-buildpack4.3.77
push-apps-manager-release669.0.7
push-usage-service-release670.0.6
pxc0.16.0
python-offline-buildpack1.6.34
r-offline-buildpack1.0.10
routing0.188.2
ruby-offline-buildpack1.7.40
silk2.22.2
smb-volume1.1.0
staticfile-offline-buildpack1.4.43
statsd-injector1.10.0
syslog11.4.0
uaa71.0

2.6.0

Component Version
ubuntu-xenial stemcell315.36
backup-and-restore-sdk1.15.1
binary-offline-buildpack1.0.32
bosh-dns-aliases0.0.3
bosh-system-metrics-forwarder0.0.18
bpm1.0.4
capi1.80.0
cf-autoscaling218
cf-backup-and-restore0.0.11
cf-cli1.13.0
cf-networking2.22.2
cf-smoke-tests40.0.108
cf-syslog-drain10.2
cflinuxfs30.101.0
consul-drain0.0.3
consul198
credhub2.4.0
diego2.30.0
dotnet-core-offline-buildpack2.2.11
garden-runc1.19.1
go-offline-buildpack1.8.39
haproxy9.5.2
istio1.0.1
java-offline-buildpack4.18
leadership-election1.4
log-cache2.1.4
loggregator-agent3.14
loggregator105.5
mapfs1.1.5
metric-registrar1.0.4
mysql-monitoring9.3.0
nats26
nfs-volume2.1.0
nginx-offline-buildpack1.0.11
nodejs-offline-buildpack1.6.49
notifications-ui36
notifications57
php-offline-buildpack4.3.76
push-apps-manager-release669.0.4
push-usage-service-release670.0.6
pxc0.16.0
python-offline-buildpack1.6.32
r-offline-buildpack1.0.9
routing0.188.1
ruby-offline-buildpack1.7.38
silk2.22.2
smb-volume1.1.0
staticfile-offline-buildpack1.4.42
statsd-injector1.10.0
syslog11.4.0
uaa71.0

How to Upgrade

The procedure for upgrading to Pivotal Application Service (PAS) v2.6 is documented in the Upgrading Pivotal Cloud Foundry topic.

When upgrading to PAS v2.6, be aware of the following upgrade considerations:

  • If you previously used an earlier version of PAS, you must first upgrade to PAS v2.5 to successfully upgrade to PAS v2.6.

  • Some partner service tiles may be incompatible with PCF v2.6. Pivotal is working with partners to ensure their tiles are updated to work with the latest versions of PCF.

    For information about which partner service releases are currently compatible with PCF v2.6, review the appropriate partners services release documentation at https://docs.pivotal.io, or contact the partner organization that produces the tile.

New Features in PAS v2.6

See the following new features for PAS v2.6:

Monitor System Metrics with System Metrics Agent

System Metrics Agent provides more visibility into VM compute, network, and storage metrics. These VM metrics help with troubleshooting and diagnosing issues for potential infrastructure problems.

When enabled, the metrics are emitted through Loggregator. For more information about viewing logs and metrics, see the Platform components row of Viewing Logs and Metrics.

For a list of the VM metrics that the System Metric Agent emits, see VM Metrics in GitHub.

To enable the System Metric Agent, go to the System Logging pane in the PAS tile and select Enable System Metrics.

Metric Registrar Enabled by Default

In the Metric Registrar pane, Metric Registrar is enabled by default. This allows you to output custom application metrics that can be monitored by platform-provided tooling.

For information about configuring Metric Registrar, see the (Optional) Configure Metric Registrar section of the Configuring PAS topic.

Enable, View, and Rollback Revisions for Apps

PAS supports revisions for apps. A revision is an object that represents code and configuration used by an app at a specific time. Some uses cases for revisions include rolling back your app to a previous version and viewing changes in your app over time. You can also add metadata to revisions.

For more information, see App Revisions.

Push Apps with Sidecar Processes (Beta)

You can run additional processes, or sidecars, in the same container as your app. Sidecars are useful for processes that depend on each other or must run in the same container. This includes processes that must communicate through localhost or share the same filesystem, such as an Application Performance Monitoring (APM) tool.

For more information, see Pushing Apps with Sidecar Processes (Beta).

Garden Delegates Container Creation and Destruction to containerd by Default

Traditionally, Garden uses runc directly to create, delete, and run container processes. Garden v1.15.0 and later support delegating some of the container lifecycle through containerd, which is the industry standard wrapper around runc.

Containerd mode is enabled by default to bring PAS in line with other container providers. To disable containerd mode, deselect the Enable Containerd Delegation checkbox in the Application Containers pane of the PAS tile.

For more information, see opsguide-containerd.md on GitHub.

NFS Legacy Mounter Removed

PAS v2.6 removes the nfs-legacy NFS mounter. Existing nfs-legacy service instances will continue to work, but will use the newer nfs mounter.

PAS v2.3 introduced the nfs-experimental service. PAS v2.4 made this the default nfs service, and the original service became nfs-legacy. For more information, see Experimental NFS Volume Service Supports NFSv4 and NFS-Experimental Service Graduation.

Extra Diego Root CA Added to Prevent Future Expiration Issues

The current Diego root CA included with PAS v2.3 and later expires in mid-2020. To prevent potential expiration issues with intermediate CAs when the root CA expires, PAS v2.6 includes another Diego root CA that extends the root CA expiration date.

The Diego root CA is used to sign intermediate CAs, which are used to sign app identity certificates. In turn, app identity certificates are used to establish trust between applications and various components.

PAS v2.6 includes both the original root CA and the new root CA for Diego. This release note is informational only. No operator intervention is required.

Performance Improvements for Read-Write File Systems

PAS v2.6 improves the speed of read-write mounted file systems that use UID mapping. This improvement addresses performance issues in mapfs, which is enabled when you specify a UID that maps to an NFS volume.

For more information about using external file systems and NFS volume services, see Using an External File System (Volume Services).

Increased CPUs for Router VMs

To improve reliability, PAS v2.6 increases the default and minimum CPU core count from one to two for Router VMs.

PAS migrates Router VMs with a single core to a VM type with two CPU cores during the upgrade.

Loggregator Syslog Agent Increases Scale For Syslog Drains

WARNING: See the following known issue related to this feature: App Syslog Drains Fail After Enabling Agent-Based Syslog.

The Loggregator architecture includes optional Syslog Agents. Syslog Agents run on PCF component VMs and host VMs to manage connections with and write to syslog drains for app logs. The addition of Syslog Agents increases the number of syslog drain service bindings supported by the Loggregator system and reduces the workload for Loggregator VMs.

To enable Syslog Agents, select Enable agent-based syslog egress for app logs in the System Logging PAS configuration pane.

Note: Enabling this feature disables the Syslog Adapter and Syslog Scheduler to avoid log duplication.

This update resolves a known issue where app log loss occurs at 10,000 syslog drain service bindings. For more information about the known issue, see the Known Loggregator Scaling Issues knowledge base article.

For more information about Syslog Agents, see Loggregator Architecture.

Terminate Specific Instances of an App in Apps Manager UI

Apps Manager allows you to terminate a specific instance of an app through the UI. On the list of apps in the Apps Manager UI, a menu option called Terminate Instance appears.

For more information about terminating an instance, see Terminate a process instance in the Cloud Foundry API documentation.

View and Edit Labels and Annotations Associated with an Organization

In the Settings pane of the Apps Manager UI, the Metadata section contains lists of labels and annotations associated with an organization. You can edit these labels and annotations.

For more information, see the Add Metadata section of the Managing Orgs and Spaces Using Apps Manager topic.

Known Issues

App Syslog Drains Fail After Enabling Agent-Based Syslog

If you select Enable agent-based syslog egress for app logs in the System Logging pane of the PAS tile, external syslog drains that are bound to Windows Apps cannot collect logs. For more information, see Enable agent-based syslog egress for app logs" interrupts external log collection for PAS Windows apps in the Pivotal Knowledge Base.

Some Environment Variables Are Missing When Using cflinuxfs3

When using the cflinuxfs3 stack in PAS v2.3 or later, if you provide environment variables containing periods or dashes, the environment variables do not appear in the process environment of the app.

To resolve this issue, ensure that all applications are using environment variables that do not contain periods or dashes.

For more information, see Missing environment variables when using PAS 2.3+ and the cflinuxfs3 stack in the Pivotal Knowledge Base.

CredHub Database Migration Failure

When the CredHub database fails to migrate with a Flyway exception, it may be caused by an issue with the flyway_schema_history table.

For information on how to address this issue, see Database Migration Failure in GitHub.

Intermittent Misrouting of Apps in Large PCF Foundations

Large PCF Foundations can experience intermittent misrouting of apps. These routes can point to non-existent or incorrect app containers and can cause apps to intermittently return HTTP codes 404 or 502.

This issue typically occurs in larger-sized foundations where a single Gorouter instance misses a deregistration message when a user unmaps routes to a running app. As a result, the Gorouter retains stale routes in its routing table.

This issue is fixed in PAS v2.6.1 and later.

If you experience intermittent misrouting in apps, do the following:

  1. Log in to Ops Manager.
  2. Update the PAS tile to PAS v2.6.1.
  3. In the PAS tile, select Application Containers.
  4. Select the Prune Routes on TTL Expiry for TLS Backends checkbox.
  5. Click Review Pending Changes.
  6. Click Apply Changes.

For more information, see Enabling TLS from the Gorouter to application instances results in bad routes in PAS 2.3+.

Apps Manager Shows Blank Page in Internet Explorer

In PAS v2.6.3, Apps Manager includes a change that is not compatible with the Internet Explorer 11 browser. The change results in a blank page being shown when attempting to use Apps Manager.

This issue is fixed in PAS 2.6.3 and later, and does not affect other browsers.

In PAS v2.6.0 to v2.6.4, Apps Manager does not show a custom logo in its header. It instead shows the Pivotal Web Services logo.

In PAS v2.6.5 the correct PCF default logo is displayed and custom logos may be uploaded, however the uploaded logo must be the right size, otherwise it will cause layout problems.

These issues are fixed in PAS v2.6.6 and later.

Apps Manager Does Not Show Spring Mappings Outside of the Application Context

In PAS v2.6.0 to v2.6.3, the Spring Boot mappings in Apps Manager did not account for contexts other than the 'application’ context. For this reason, some Spring Boot mappings may fail to show up in Apps Manager for a given application.

This issue is fixed in PAS v2.6.4 and later.