PCF Isolation Segment v2.3 Release Notes

Known Issues

  • [Known Issue] The NSX-T tile versions 2.3.1 and lower are not compatible with IST. Upcoming release NSX-T 2.3.2 will address this issue.

Releases

Breaking Change: You must upgrade to PAS v2.3.5 or later before upgrading to IST v2.3.5 or later.

2.3.12

  • [Feature Improvement] Update default polling interval and idle connection limits for networking components to reduce resource contention on PAS database
  • Bump cf-networking to version 2.12.5
  • Bump cflinuxfs3 to version 0.88.0
  • Bump silk to version 2.12.4
Component Version
ubuntu-xenial stemcell97.82
bpm1.0.4
cf-networking2.12.5
cflinuxfs21.283.0
cflinuxfs30.88.0
consul195
diego2.20.8
garden-runc1.16.8
haproxy8.9.0
loggregator103.4
mapfs1.1.4
nfs-volume1.5.7
routing0.179.6
silk2.12.4
syslog11.3.2

2.3.11

  • [Bug Fix] Fixes backward compatibility issue with NFS that can prevent apps from binding to service instances created in PAS 2.2 or earlier
  • Bump ubuntu-xenial stemcell to version 97.82
  • Bump cflinuxfs2 to version 1.283.0
  • Bump cflinuxfs3 to version 0.86.0
  • Bump nfs-volume to version 1.5.7
Component Version
ubuntu-xenial stemcell97.82
bpm1.0.4
cf-networking2.12.3
cflinuxfs21.283.0
cflinuxfs30.86.0
consul195
diego2.20.8
garden-runc1.16.8
haproxy8.9.0
loggregator103.4
mapfs1.1.4
nfs-volume1.5.7
routing0.179.6
silk2.12.2
syslog11.3.2

2.3.10

  • [Feature Improvement] Add support for staging Docker images from repositories using schema version 2 manifests
  • [Bug Fix] Prevent potential BPM configuration corruption after a VM restarts which could cause some jobs to fail to start. Knowledge Article 6606
  • Bump ubuntu-xenial stemcell to version 97.74
  • Bump bpm to version 1.0.4
  • Bump cflinuxfs2 to version 1.281.0
  • Bump cflinuxfs3 to version 0.81.0
  • Bump diego to version 2.20.8
Component Version
ubuntu-xenial stemcell97.74
bpm1.0.4
cf-networking2.12.3
cflinuxfs21.281.0
cflinuxfs30.81.0
consul195
diego2.20.8
garden-runc1.16.8
haproxy8.9.0
loggregator103.4
mapfs1.1.4
nfs-volume1.5.6
routing0.179.6
silk2.12.2
syslog11.3.2

2.3.9

  • [Feature Improvement] Add support for TCP hitless reloads in haproxy to avoid connection reset errors
  • [Feature Improvement] Add ability to enable/disable gorouter hairpinning
  • [Bug Fix] Improved clean up logic for stuck or unreachable NFS mounts to prevent cells’ rep and nfsv3driver jobs from hanging during unmount and drain
  • [Bug Fix] Fix feature: “Operator can specify headers to be stripped from the response by the router”
  • [Bug Fix] Properly populate chained CA certificates for backend proxies when route integrity is enabled
  • [Bug Fix] Fix diego rep to always clean up temporary download cache directory
  • Bump ubuntu-xenial stemcell to version 97.71
  • Bump cflinuxfs2 to version 1.279.0
  • Bump cflinuxfs3 to version 0.76.0
  • Bump diego to version 2.20.7
  • Bump nfs-volume to version 1.5.6
  • Bump routing to version 0.179.6
Component Version
ubuntu-xenial stemcell97.71
bpm1.0.3
cf-networking2.12.3
cflinuxfs21.279.0
cflinuxfs30.76.0
consul195
diego2.20.7
garden-runc1.16.8
haproxy8.9.0
loggregator103.4
mapfs1.1.4
nfs-volume1.5.6
routing0.179.6
silk2.12.2
syslog11.3.2

2.3.8

  • [Security Fix] Bump BPM to v1.0.3 for RunC CVE
  • [Bug Fix] Fixes access issue for NFS shares with root_squash enabled and no world read permissions.
  • Bump ubuntu-xenial stemcell to version 97.57
  • Bump bpm to version 1.0.3
  • Bump cflinuxfs2 to version 1.267.0
  • Bump cflinuxfs3 to version 0.62.0
  • Bump loggregator to version 103.4
  • Bump mapfs to version 1.1.4
Component Version
ubuntu-xenial stemcell97.57
bpm1.0.3
cf-networking2.12.3
cflinuxfs21.267.0
cflinuxfs30.62.0
consul195
diego2.20.1
garden-runc1.16.8
haproxy8.9.0
loggregator103.4
mapfs1.1.4
nfs-volume1.5.5
routing0.179.4
silk2.12.2
syslog11.3.2

2.3.7

  • [Bug Fix] Fix concurrency bug in the Router’s route pool, which could manifest as a fatal error: “Unlock of unlocked RWMutex”
  • [Bug Fix] Improve garden init process to avoid edge cases that can lead to zombies
  • [Bug Fix] Fix allowNativePassword error when using external IaaS MySql as a system database
  • Bump ubuntu-xenial stemcell to version 97.53
  • Bump cflinuxfs2 to version 1.260.0
  • Bump cflinuxfs3 to version 0.51.0
  • Bump garden-runc to version 1.16.8
  • Bump nfs-volume to version 1.5.5
  • Bump routing to version 0.179.4
Component Version
ubuntu-xenial stemcell97.53
bpm0.11.0
cf-networking2.12.3
cflinuxfs21.260.0
cflinuxfs30.51.0
consul195
diego2.20.1
garden-runc1.16.8
haproxy8.9.0
loggregator103.3
mapfs1.1.2
nfs-volume1.5.5
routing0.179.4
silk2.12.2
syslog11.3.2

2.3.6

  • Bump ubuntu-xenial stemcell to version 97.43
  • Bump cflinuxfs2 to version 1.258.0
  • Bump cflinuxfs3 to version 0.48.0
  • Bump diego to version 2.20.1
Component Version
ubuntu-xenial stemcell97.43
bpm0.11.0
cf-networking2.12.3
cflinuxfs21.258.0
cflinuxfs30.48.0
consul195
diego2.20.1
garden-runc1.16.1
haproxy8.9.0
loggregator103.3
mapfs1.1.2
nfs-volume1.5.4
routing0.179.3
silk2.12.2
syslog11.3.2

2.3.5

Breaking Change: You must upgrade to PAS v2.3.5 or later before upgrading to PCF Isolation Segment v2.3.5 or later. See Apps Do Not Stage After Upgrading to v2.3.5 or Later.

  • [Bug Fix] Bump Diego to v2.20.0 to resolve DNS timeouts from Golang components
  • Bump ubuntu-xenial stemcell to version 97.42
  • Bump cflinuxfs2 to version 1.255.0
  • Bump cflinuxfs3 to version 0.46.0
  • Bump diego to version 2.20.0
  • Bump silk to version 2.12.2
Component Version
ubuntu-xenial stemcell97.42
bpm0.11.0
cf-networking2.12.3
cflinuxfs21.255.0
cflinuxfs30.46.0
consul195
diego2.20.0
garden-runc1.16.1
haproxy8.9.0
loggregator103.3
mapfs1.1.2
nfs-volume1.5.4
routing0.179.3
silk2.12.2
syslog11.3.2

2.3.4

  • [Security Fix] Address leak of CF admin credentials into NFS broker bosh errand logs
  • [Security Fix] Rotate diego intermediate CA before current certificate expires
  • Bump ubuntu-xenial stemcell to version 97.34
  • Bump cf-networking to version 2.12.3
  • Bump cflinuxfs2 to version 1.249.0
  • Bump cflinuxfs3 to version 0.40.0
  • Bump mapfs to version 1.1.2
  • Bump nfs-volume to version 1.5.4
Component Version
ubuntu-xenial stemcell97.34
bpm0.11.0
cf-networking2.12.3
cflinuxfs21.249.0
cflinuxfs30.40.0
consul195
diego2.18.0
garden-runc1.16.1
haproxy8.9.0
loggregator103.3
mapfs1.1.2
nfs-volume1.5.4
routing0.179.3
silk2.12.1
syslog11.3.2

2.3.3

  • [Feature Improvement] Improve router pruning behavior when route integrity is enabled
  • Bump ubuntu-xenial stemcell to version 97.28
  • Bump cflinuxfs2 to version 1.245.0
  • Bump cflinuxfs3 to version 0.34.0
  • Bump routing to version 0.179.3
Component Version
ubuntu-xenial stemcell97.28
bpm0.11.0
cf-networking2.12.2
cflinuxfs21.245.0
cflinuxfs30.34.0
consul195
diego2.18.0
garden-runc1.16.1
haproxy8.9.0
loggregator103.3
mapfs1.0.1
nfs-volume1.5.3
routing0.179.3
silk2.12.1
syslog11.3.2

2.3.2

  • [Bug Fix] Fix issue in loggregator where availability zone (AZ) names with special characters could cause metron agent job to fail
  • [Bug Fix] Logs marked as “DEBUG” are no longer forwarded by default
  • Bump ubuntu-xenial stemcell to version 97.19
  • Bump cflinuxfs2 to version 1.242.0
  • Bump cflinuxfs3 to version 0.29.0
  • Bump loggregator to version 103.3
Component Version
ubuntu-xenial stemcell97.19
bpm0.11.0
cf-networking2.12.2
cflinuxfs21.242.0
cflinuxfs30.29.0
consul195
diego2.18.0
garden-runc1.16.1
haproxy8.9.0
loggregator103.3
mapfs1.0.1
nfs-volume1.5.3
routing0.179.2
silk2.12.1
syslog11.3.2

2.3.1

  • [Bug Fix] Fixes slow app instance shutdown when using NSX-T for container networking
  • [Bug Fix] Fix parse error for syslog rules when iptables logging is enabled

  • Bump cf-networking to version 2.12.2

  • Bump cflinuxfs2 to version 1.238.0

  • Bump cflinuxfs3 to version 0.25.0

  • Bump diego to version 2.18.0

  • Bump silk to version 2.12.1

  • Bump stemcell ubuntu-xenial to version 97.18

Component Version
stemcell97.18
bpm0.11.0
cf-networking2.12.2
cflinuxfs21.238.0
cflinuxfs30.25.0
consul195
diego2.18.0
garden-runc1.16.1
haproxy8.9.0
loggregator103.0
mapfs1.0.1
nfs-volume1.5.3
routing0.179.2
silk2.12.1
syslog11.3.2
* Components marked with an asterisk have been patched to resolve security vulnerabilities or fix component behavior.

2.3.0

Component Version
stemcell97.16
bpm0.11.0
cf-networking2.12.0
cflinuxfs21.236.0
cflinuxfs30.23.0
consul195
diego2.16.0
garden-runc1.16.1
haproxy8.9.0
loggregator103.0
mapfs1.0.1
nfs-volume1.5.3
routing0.179.2
silk2.12.0
syslog11.3.2
* Components marked with an asterisk have been patched to resolve security vulnerabilities or fix component behavior.

About PCF Isolation Segment

The PCF Isolation Segment v2.3 tile is available for installation with PCF v2.3.

Isolation segments provide dedicated pools of resources where you can deploy apps and isolate workloads. Using isolation segments separates app resources as completely as if they were in different CF deployments but avoids redundant management and network complexity.

For more information about using isolation segments in your deployment, see the Managing Isolation Segments topic.

How to Install

The procedure for installing PCF Isolation Segment v2.3 is documented in the Installing PCF Isolation Segment topic.

To install a PCF Isolation Segment, you must first install PCF v2.3.

New Features in PCF Isolation Segment v2.3

Xenial Stemcells

IST v2.3 uses a stemcell based on Ubuntu 16.04 (Xenial Xerus).

Earlier versions’ stemcells were based on Ubuntu 14.04 (Trusty Tahr). The Trusty distribution will reach end of general support (EOGS) in April 2019 and will no longer receive security updates.

Using Xenial stemcells in v2.3 ensures that IST users continue to have access to secure stemcells based on a Ubuntu distribution maintained by Canonical.

For more information on the impact of using Xenial stemcells in PCF, see Updates for Xenial Stemcell Support.

HTTP Router Uses TLS By Default in New Deployments

In new installations of PCF v2.3, the HTTP router (Gorouter) uses TLS by default to communicate with application containers. This configuration improves resiliency and consistency for application routes. Find this setting in the Isolation Segment tile > Application Containers pane.

For more information, see Increased Resiliency, Consistency, and Security for HTTP Routing in the Isolation Segment v2.1 release notes.

If you upgrade to PCF v2.3, each Isolation Segment tile retains its previous setting for HTTP communication to apps. TLS is not the default setting.

NFS LDAP Configuration Updates

The LDAP configuration for NFS in the Application Containers pane includes the following updates:

  • You can optionally enter a LDAP Server CA Cert if your LDAP server supports TLS and you want to enable secure connections from the NFS driver to your LDAP server.
  • The LDAP User Fully-Qualified Domain Name has been renamed to LDAP User Search Base for clarity and consistency with LDAP UAA configuration in the Authentication and Enterprise SSO pane.

App Service Discovery Updates

App service discovery is enabled by default and no longer configurable. The Enable app service discovery checkbox no longer appears in the Application Developer Controls pane.

You can also configure the internal domain used for service discovery using the Internal Domain field in the Application Developer Controls pane of the PAS tile. This field defaults to apps.internal.

BOSH Process Manager

Starting in v2.3, some IST components use BOSH Process Manager (BPM).

BPM is a layer between BOSH and the jobs running on component VMs. It improves the way processes run on VMs by isolating colocated jobs. With the exception of networking, BPM namespaces operating system resources so a job cannot view or interact with the processes of another job. This provides a security barrier such that if a job on a VM is compromised, the incident is limited to just that job rather than all jobs on the same machine.

BPM also includes resource limiting capability. This prevents any one job from using too much operating system resources and impacting colocated jobs.

For more information about BPM, see the bpm-release repository.

Remove Deprecated Garden Image Plugin Option

WARNING: If you do not have GrootFS enabled, you must enable it before upgrading to PAS v2.3. See How to Upgrade above.

Garden creates app containers in IST and includes an image plugin that prepares the filesystem for the container. PCF v1.12 introduced GrootFS as the default image plugin to replace the previous built-in functionality, garden-shed, which used an obsolete layer filesystem (AUFS) that lacked support from the Linux Kernel community.

Though GrootFS is the default image plugin in v1.12 through v2.2, at one time PAS provided an option in the Application Containers pane for operators to disable GrootFS and use garden-shed. However, garden-shed is deprecated and IST v2.3 removes the option to use it.

For more information about GrootFS in PCF, see the following topics:

HTTP Router Uses TLS By Default in New Deployments

In new installations of IST v2.3, the HTTP router (Gorouter) uses TLS by default to communicate with application containers. This configuration improves resiliency and consistency for application routes. If you are upgrading to IST v2.3, IST retains its previous setting for HTTP communication to apps. TLS is now the default setting.

Configure this setting in the PAS tile > Application Containers pane.

For more information, see Increased Resiliency, Consistency, and Security for HTTP Routing in the PAS v2.1 release notes.

Mutual TLS App Identity Verification

In the Application Containers pane of PAS v2.3, the new option Router and applications use mutual TLS to verify each other’s identity configures the Gorouter and app containers to verify each other’s identities through mutual TLS (mTLS).

With the Router uses TLS to verify application identity option, the Gorouter uses a one-way TLS handshake to verify the identity of the app container, but the app container does not verify the identity of the Gorouter. The new mTLS option increases security over one-way TLS by ensuring that the Gorouter is the only client that can communicate with app instances.

Remove Consul Agent

In IST v2.3, IST component VMs no longer include the consul_agent job. IST component VMs can now only use BOSH DNS for service discovery.

However, the consul_server VM continues to exist in PAS deployments to support any service or partner tiles that still require communication with Consul.

Breaking Change: Enabling mTLS creates certain limitations. For more information, see Limitations with Mutual TLS App Identity Verification in the Known Issues section.

Known issues

Apps Do Not Stage After Upgrading to v2.3.5 or Later

If you upgrade to PCF Isolation Segment v2.3.5 or later before upgrading to PAS v2.3.5 or later, apps do not stage when pushed to an Isolation Segment. For more information, see the following article in the Pivotal Knowledge Base: Diego releases out of sync while upgrading PASW to version 2.3.5 or above.

NSX-T v2.3.1 and Earlier Not Compatible with PCF Isolation Segment

The NSX-T tiles v2.3.1 and earlier are not compatible with PCF Isolation Segment. The Gorouters in an Isolation Segment are not given access in the firewall rules for NSX-T v2.3.1 and earlier, which prevents them from communicating with apps.

NSX-T v2.3.2 and later give access to the Gorouters in an Isolation Segment, and thus are compatible with PCF Isolation Segment.

About Advanced Features

The Advanced Features section of the PCF Isolation Segment v2.3 tile includes new functionality that may have certain constraints.

Although these features are fully supported, Pivotal recommends caution when using them in production.

Create a pull request or raise an issue on the source for this page in GitHub