Pivotal Elastic Runtime v1.9 Release Notes

Page last updated:

Warning: Pivotal Cloud Foundry (PCF) v1.9 is no longer supported because it has reached the End of General Support (EOGS) phase. To stay up to date with the latest software and security updates, upgrade to a supported version.

About Updating to Elastic Runtime v1.9

If you are currently on Elastic Runtime v1.8.32 or earlier, it is recommended that you upgrade to v1.8.33 or later before upgrading to Elastic Runtime v1.9.

Upgrading directly to v1.9 without first upgrading to v1.8.33 will result in the loss of TCP routes if your deployment makes use of them.

Elastic Runtime v1.8.33 contains a migration to move your TCP routing data to MySQL. Please view the Elastic Runtime v1.8 release notes for more details.

Releases

1.9.48

• [Security Fix] Bump stemcell to version 3445.24 to address issues:
  • USN-3534-1
  • USN-3540-2
• [Bug Fix] Bump uaa-release to v24.14

1.9.47

• [Security Fix] Bumps stemcell version to 3445.22 for USN-3544-2 and USN-3544-4

1.9.46

• [Security Fix] Patches Cloud Controller to prevent users from being able to create a private subdomain of a route in an organization they do not have access to.

1.9.45

• [Security Fix] Bumps stemcell version to 3445.19 for USN-3509-2.

1.9.44

• [Security Fix] Bumps cflinuxfs2-release to v1.166.0 to resolve USN-3475-1. Release Notes
• [Security Fix] Patches Golang components in capi-release to pull in Golang v1.8.3.
• [Bug Fix] Patches consul-release to include fix for DNS lock management for Windows deployments.

1.9.43

• [Security Fix] Bumps the stemcell to v3445.16 to resolve several security vulnerabilities:
  • USN-3424-1
  • USN-3432-1
  • USN-3434-1
  • USN-3441-1
  • USN-3444-2
• [Security Fix] Bumps the cflinuxfs2-release to v1.165.0 to resolve several security vulnerabilities:
  • USN-3457-1
  • USN-3458-1
  • USN-3464-1

1.9.42

• [Security Fix] Bumps cflinuxfs2-release to v1.161.0 to resolve multiple security issues. Release Notes

1.9.41

• [Security Fix] Bumps cflinuxfs2-release to v1.158.0 to resolve multiple security issues. Release Notes

1.9.40

• [Security Fix] Bumps cflinuxfs2-release to v1.156.0 to resolve multiple security issues. Release Notes
• [Security Fix] Bumps cf-mysql-release to v36.6 to patch vulnerabilities in Bundler and RubyGems CVE-2016-7954 CVE-2017-0902
• [Security Fix] Resolves a remote code execution security vulnerability when the zip program is executed by the Cloud Controller.
• [Security Fix] Resolves an issue with an incorrect Host header being set on incoming requests through the Router CVE Notice.
• Bumps the following buildpack releases:
  • binary-buildpack
  • dotnet-core-buildpack
  • go-buildpack
  • java-buildpack
  • nodejs-buildpack
  • php-buildpack
  • python-buildpack
  • ruby-buildpack
  • staticfile-buildpack
• [Stability Improvement] Changes the default for Galera MySQL state snapshot transfers (SST). Automatic SST is now enabled by default. Operators can disable this feature by visiting the “Internal MySQL” tab and checking the “Prevent node auto re-join” checkbox.

1.9.39

• [Security Fix] Bumps stemcell to v3445.11 to address USN-3420-2.
• [Security Fix] Bumps cflinuxfs-release to v1.155.0 to address USN-3415-1.

1.9.38

• [Security Fix] Bumps cflinuxfs2-release to v1.150.0 to resolve USN-3398-1.
• [Bug Fix] Bumps apps-manager-release to v659.12.16, including the following fixes:
  • The square logo will now display correctly in the header.
  • Allows the invitations application to install in offline environments.
  • Users can now see shared private domains on the org domains tab.
  • Resolves an erroneous mapping of hidden shared private domains.
• [Bug Fix] Bumps cf-autoscaling-release to v75.12. Release Notes.
• [Bug Fix] Fixes a data migration in Cloud Controller to prune duplicate routes when upgrading to this version from a 1.8.x version of the Elastic Runtime.
• [Bug Fix] Resolves an issues with Loggregator WebSocket control frame timeouts.
• [Feature Improvement] Operators can now configure a “Staging Timeout” to force Cloud Controller to wait for staging of applications that may take a very long time.
• [Feature Improvement] The internal MySQL cluster will now emit metrics via the firehose. You can use cf nozzle to view those metrics as they are emitted.

1.9.37

• [Security Fix] Bumps stemcell to v3363.31 to resolve USN-3392-2.
• [Security Fix] Bumps cflinuxfs2-release to v1.147.0 to resolve USN-3387-1 and USN-3388-1.

1.9.36

• [Security Fix] Bumps stemcell to version 3363.30.
• [Bug Fix] Patches Diego components to resolve race condition in the buildpack and docker launchers, as well as the diego-sshd process that could cause errant process failures.
• The regions listed on the File Storage form for S3-compatible blobstores now includes all available S3 regions.
• Operators can now configure the etcd heartbeat and election timeout values for their Diego Database instance.
• Automated backup for the internal MySQL instances now includes support for GCP and Azure.

1.9.35

• [Security Fix] Bumps stemcell to v3363.29 to resolve USN-3378-2.
• [Security Fix] Bumps cflinuxfs2 to v1.145.0 to resolve multiple CVEs and USNs. Please see the release notes for more details.

1.9.34

• [Bug Fix] Cloud Controller will now correctly verify self-signed certificates on inbound requests. This was broken in a patch applied in Elastic Runtime 1.9.31. If you have an environment with self-signed certificates, you should skip v1.9.31 - v1.9.33.
• [Bug Fix] The smoke test errand will now correctly use the system organization to deploy its canary applications.
• The components included in routing-release (Gorouter, route_registrar, routing-api, tcp_emitter, and tcp_router) have been updated to run on Go v1.8.

1.9.33

• [Security Fix] Bumps apps-manager-release to v659.12.13. This release has the following updates:
  • Bumps nodejs to v6.11.1 to provide security fixes.
  • Includes org_id and org_name in usage report zip file.
• [Security Improvement] Bumps cf-mysql-release to v36. Release Notes

  Note: This version of cf-mysql-release disables mysql client access as administrator from remote hosts. Administrators must bosh ssh into each MySQL VM to connect as the MySQL super user to execute commands like mysql, mysqldump and mysqlimport. This restriction affects the admin and roadmin accounts.

• Bumps mysql-backup-release to v1.33.0. Provides compatibility with v36 of cf-mysql-release.
• Bumps mysql-monitoring-release to v8.3.0. Release Notes.
• [Stability Improvement] Reduces the default for cc.droplets.max_staged_droplets_stored from 5 to 2. This will result in reduced blobstore utilization as the Cloud Controller will only keep 2 historic staged application droplets, in addition to the currently running application droplet. Garbage collection of expired droplets will occur the next time an application is staged.

1.9.32

• [Security Fix] Provides a fix for CVE-2017-8033. This security vulnerability would have allowed attackers to escalate their privileges by pushing an application that could modify the files on the Cloud Controller VM.
• [Security Fix] The Router will now validate the UAA token issuer field. This will prevent users with valid tokens belonging to an Identity Zone other than the default zone from escalating their privileges when making requests against system components.
• [Bug Fix] Resolves an issue with UAA SAML Service Provider Key Password quoting.
• [Stability Improvement] Operators can now configure the Cluster Probe Timeout for their Internal MySQL cluster. This property controls the maximum time a new MySQL node will search for an existing cluster before starting its own. Higher values for this property will help to maintain cluster quorum on slower or more loaded infrastructures.
• Bumps the following buildpacks to their latest version:
  • dotnet-core-buildpack Release Notes
  • go-buildpack Release Notes
  • java-buildpack Release Notes
  • nodejs-buildpack Release Notes
  • php-buildpack Release Notes
  • python-buildpack Release Notes
  • ruby-buildpack Release Notes
  • staticfile-buildpack Release Notes
• Sets the default max-in-flight value for the Diego Cells to 4%. Operators can still use the Ops Manager API to configure this setting to fit their needs. The max-in-flight percentage for the Diego Cell job in the Elastic Runtime has been set to 10% since 1.9.0, but we’ve seen especially in larger environments that having the percentage this high can cause some problems:
  • Many simultaneous VM creates/deletes and BOSH blob updates can place significant stress on the underlying infrastructure, especially on vSphere which has a greater probability of being under-provisioned.
  • The cells that are draining are no longer available for allocation, resulting in a 10% decrease in total memory and disk capacity during the deployment. This can cause deployments to no longer have sufficient total capacity to run all the work, or to have insufficient headroom to place larger workloads successfully.

1.9.31

• [Security Fix] Cloud Controller will now validate the UAA token issuer field. This will prevent users with valid tokens belonging to an Identity Zone other than the default zone from escalating their privileges when making requests against system components.
• [Security Fix] Provides a fix for CVE-2017-8035. This security vulnerability would have allowed arbitrary files on the Cloud Controller VM to be downloaded by external API users.

1.9.30

• [Bug Fix] Bumps uaa-release to v24.13. Resolves an issue where SAML assertions could not be validated.
• Bumps service-backup-release to v18.1.2. Release Notes

1.9.29

• [Security Fix] Bumps cflinuxfs2-rootfs to 1.33.0. Release Notes

1.9.28

• [Security Fix] Bumps stemcell to 3363.26.
• Bumps the following buildpacks to their latest version:
  • binary-buildpack Release Notes
  • dotnet-core-buildpack Release Notes
  • go-buildpack Release Notes
  • java-buildpack Release Notes
  • nodejs-buildpack Release Notes
  • php-buildpack Release Notes
  • python-buildpack Release Notes
  • ruby-buildpack Release Notes
• [Bug Fix] Bumps Pivotal Account to v1.1.14. This release supports large custom-branding images, provides security fixes, and refactors Pivotal Account deployment errand to avoid database issues during migrations on certain versions of MySQL.
• Bumps notifications-release to v36. Release Notes
• [Feature] HttpStartStop metric for Gorouter includes ‘instanceIndex’.
• [Bug Fix] Patches Cloud Controller to prevent a DB error when a staging response contains too many environment variables.
• Removes unnecessary persistent disk from Cloud Controller VM.

1.9.27

• Bumps uaa-release to v24.12.

1.9.26

• Bumps garden-runc to v1.7.0.
• Patches diego-release to allow ICMP and UDP packet logging for security group rules.
• Patches Cloud Controller to resolve migration upgrade issue.

1.9.25

• Bumps stemcell to v3363.25.
• Bumps cflinuxfs rootfs v1.126.0.

1.9.24

• Bumps uaa-release to v24.11.

1.9.23

• Bumps stemcell to v3312.26.
• Bumps cf-mysql-release to v32.9.0.
• Bumps cflinuxfs2 rootfs to v1.123.0.
• Resource Configuration now support custom VM templates that have CPU counts that are not a power of two.
• The SAML Service Provider Certificate/Key Password is not properly obfuscated.
• Patches Cloud Controller to resolve an issue where apps could become orphaned from their spaces.
• Patches Cloud Controller to increase the application healthcheck timeout to 10 minutes.
• Patches Loggregator to prevent a misleading error message appearing when running cf logs.

1.9.22

• Bumps autoscaling to v75.10.
• Allows the Apps Manager access token validity duration to be configured.
• Replaces all buildpack releases with versions that provide cached buildpack assets.
• Allows the UAA password expiry field to be configured via the Ops Manager API.
• Bumps uaa-release to v24.10.
• Bumps Apps Manager to v659.12.9.
• Allows S3 buckets located in regions other than us-east-1 can be used to store Internal MySQL backups.
• Patches Loggregator to resolve CLOSE_WAIT issue.

Warning:
This release includes a misleading error message being returned when running cf logs --recent.

The response incorrectly includes the line ERR WebsocketListener.Start: Error connecting to a doppler but the command is working as expected.

This message will occur once for each doppler instance in your deployment.

1.9.21

• Patches a bug with event pagination in the Cloud Controller API.
• Patches the cachedownloader to improve executor cache resilience and efficiency.
• Bumps all buildpack releases to their latest versions.
• Removes the Password Expiry field from the Authentication and Enterprise SSO form.
• Allows UAA SAML certificates to be configured.
• Allows operators to specify DNS servers that differ from those provided in their BOSH configuration.
• Exposes etcd timeout configurations on the Advanced Features form.

1.9.20

• Bumps stemcell to v3312.24.

1.9.19

• Bumps uaa-release to v24.8.

1.9.18

• Bumps garden-runc to v1.2.0 to fix compatibility with some anti-virus scanning software.
• Bumps staticfile-buildpack-release to v1.4.4 to address CVE-2017-4970. More details can be found at pivotal.io/security.
• Bumps apps-manager to v659.12.5.
• Updates the notifications-ui errand to allow operators to provide large custom branding logos.
• Adds Azure Fault-Domain detection failure logic to rep.
• Patches bug in Gorouter that caused the router to crash when invalid X-CF-APP-INSTANCE headers were sent in a request.
• Fixes the configuration for backup of Internal MySQL instances.
• Bumps uaa-release to v24.7.

1.9.17

• Bumps the stemcell to version 3312.23.

1.9.16

• Bumps rootfs to v1.60.0 with stack 1.111.0 for low/medium security fixes
• Bumps dotnet-core buildpack to v1.0.13
• Bumps service-backup-release to v18.0.3
• Adds fix for Azure Storage blobstore support

1.9.15

• Bumps the stemcell version to 3312.22.

1.9.14

• Configures the notifications service to deploy with a specified buildpack.
• Bumps pivotal-account to v1.1.11 to ensure it specifies a known buildpack during deployment.
• Allows operators to toggle the invitations feature set for AppsManager.
• Allows Space Auditors to once again stream application logs from AppsManager.
• Resolves an issue that prevented MySQL VMs restarted outside of BOSH from rejoining the cluster.
• Resolves a migration issue in the autoscaling service.
• Reduces the notifications polling interval in the autoscaling service to 5 seconds.

1.9.13

Important: This release introduced an upgrade incompatibility to the 1.10.x ERT series. Please see the note below for more details.

This upgrade advisory only applies to customers who have installed Pivotal Cloud Foundry Elastic Runtime v1.9.13 and are planning to upgrade to Pivotal Cloud Foundry Elastic Runtime v1.10.0.

We have uncovered an unfortunate issue when upgrading from Elastic Runtime v1.9.13 to v1.10.0 and are working towards resolving this issue. Brand new deployments of PCF 1.10 are not affected by this issue.

Please refer to the FAQ below for more details

Q: I have Elastic Runtime v1.9.13, should I not upgrade to ER 1.10.0?
A: Yes, please wait to upgrade to Elastic Runtime v1.10.x until the upgrade fix is in place, which is estimated to be ready in several days.

Q: I have Elastic Runtime v1.9.x, should I not upgrade to v1.9.13?
A: Please do upgrade to Elastic Runtime v1.9.13+ as this contains critical fixes. However, please do not upgrade to Elastic Runtime v1.10.0 until the upgrade fix is in place, which is estimated to be ready in several days.

Q: I have Elastic Runtime v1.9.12(or below), can I upgrade to v1.10.0?
A: Yes, you can upgrade to Elastic Runtime v1.10.0 without any issues.

Q: Can I perform a brand new deployment of Elastic Runtime v1.10.0?
A: Yes, you can proceed with a brand new deployment of Elastic Runtime v1.10.0 and will be able to consume any future Elastic Runtime v1.10.x patches

• Bumps the rootfs to v1.56 which contains stack version 1.107.0.
• Bumps uaa-release to v24.6.
• Bumps etcd-release to v97.
• Bumps the buildpacks to the most recent versions. The buildpack versions can now be seen below in the Components table.
• Fixes a bug in the internal MySQL configuration that prevented notifications from being sent when the cluster went into a dataloss-prevention state.

1.9.12

• Bumps the stemcell version to 3312.21.

1.9.11

• Allows operators to specify HTTP headers that will be recorded in the GoRouter access logs.
• Patches the Cloud Controller to correct ordering of API results when using pagination.

• Patches the Cloud Controller workers to ensure CA certificates are validated when communicating with the internal WebDAV blobstore. This resolves an issue causing WebDAV blobstores to grow unbounded due to failed SSL cert validation.

  Component	Version
  Stemcell	3312.20
  binary-buildpack	1.0.5
  capi	1.11.0\*
  cf	246\*
  cf-autoscaling	75.6
  cf-mysql	32
  cflinuxfs2-rootfs	1.46.0
  consul	137
  diego	0.1491.0\*
  dotnet-core-buildpack	1.0.5
  etcd	92
  garden-runc	1.1.1
  go-buildpack	1.7.16
  java-offline-buildpack	3.10
  loggregator	65\*
  mysql-backup	1.28.0
  mysql-monitoring	6
  nats	14
  nodejs-buildpack	1.5.23
  notifications	34
  notifications-ui	26
  php-buildpack	4.3.22
  pivotal-account	1.1.10
  postgres	8
  push-apps-manager-release	659.10
  python-buildpack	1.5.12
  routing	0.143.0\*
  ruby-buildpack	1.6.28
  service-backup	17.2.0
  staticfile-buildpack	1.3.13
  uaa	24.5
  \* Components marked with an asterisk have been patched to resolve security vulnerabilities or fix component behavior.

1.9.10

• Secures the router debug servers by ensuring they bind to the loopback device.
• Patches the router to prevent extra ?s from being appended to requests that already contain one.
• Bumps the autoscaling service to v75.6 to resolve a database timestamp issue.
• Bumps UAA to v3.9.7 to remove a possible denial of service attack vector.

1.9.9

• Bumps the stemcell to version 3312.20.

1.9.8

• Allows external databases to be configured with unique user accounts.
• Bumps uaa-release to v13.10 to redact SAML & OAuth Keys from the ID Zone API.
• Allows a “Max Inflight Container Starts” configuration to be set. This setting will limit the total number of containers that are allowed to be starting at any one time. The default setting is to limit the number of inflight starting containers to 200.
• Bumps etcd-release to v92 to improve startup resilience on VM using IPSec.

1.9.7

• Patches Pivotal Account to prevent an account authorization vulnerability. For more details, please see pivotal.io/security.

1.9.6

• Adds configurable audit logging for the Internal MySQL database. Configuration options can be found on the Internal MySQL page.
• Corrects the documentation for non-RFC-1918 Private Network configuration.
• Bumps etcd-release to v91 for some stability improvements.
• Patches the Diego TPS bridge component to prevent credential mangling.
• Includes proper verification of database presence when configuring the ERT to work with an external database server.

1.9.5

• Bumps the autoscaling release to v75.3, removing the git dependency that was vulnerable, and patching a bug that caused CPU-based scaling to incorrectly scale down if CPU utilization was reported as 0%.
• Bumps etcd-release to v88 to improve stability on environments with higher network latencies.
• Bumps Apps Manager to v659.10 to resolve several outstanding bugs and issues.
• Bumps Notifications to v34, removing logging that included UAA OAuth tokens, and allowing the service to handle validation of UAA tokens signed with different signing keys.
• Bumps the rootfs to 1.97.0 to cover some low and medium vulnerabilities.
• Patches loggregator to resolve an issue where application logs were not correctly displaying application instance indices.

1.9.4

• Bumps the garden-runc release to version 1.1.1 to address CVE-2016-9962. For more details, please see pivotal.io/security.

1.9.3

• Requires user input for the Internal MySQL Monitor Recipient Email Address field. This field previously defaulted to an invalid email address. Users that wish to use the internal MySQL cluster and deploy the monitor should remember to set a valid email recipient.
• Bumps the routing-release to version 0.143.0 to update the component’s golang version to 1.7.
• Moves credentials for the Cloud Controller Bridge components into configuration files to prevent them from being viewable in the process list.
• Fixes race conditions around transactions in the Diego BBS.

1.9.2

• Bumps the stemcell to 3312.12 to address a memory usage issue in rsyslog.
• Adds support for application http health-check types. Specifying an http health-check can be done through the API. More details can be found in the Cloud Foundry API documentation.

1.9.1

• Bumps the stemcell to 3312.9 and the rootfs to 1.94.0 in order to address a vulnerability in API (USN-3156-1).
• Patches a vulnerability in the Notifications service that allowed unprivileged users to impersonate other users with unauthenticated tokens.
• Resolves a bug in the authentication mechanism of the Autoscaling service that prevented some users from viewing the UI.
• Patches Cloud Controller to reduce conditions under which a database deadlock could occur.
• Patches Cloud Controller to remove logging of database credentials.
• Bumps the Golang buildpack to 1.7.16 to bring in support for Golang 1.7.

Component Versions

Versions 1.9.0 and higher versions of Elastic Runtime consist of the following component versions:

How to Upgrade

The procedure for upgrading to Pivotal Cloud Foundry Elastic Runtime v1.9 is documented in the Upgrading Pivotal Cloud Foundry topic.

When upgrading to v1.9, be aware of the following upgrade considerations:

• You must upgrade to a version of Elastic Runtime v1.8.9 or higher in order to successfully upgrade to v1.9.
• If you are upgrading from a PCF deployment that at one point included Elastic Runtime v1.7.16 or earlier, make sure that you have performed the remedial steps outlined in App Usage Data and Events Data Become Corrupted After Upgrade or Install before proceeding with the upgrade. If you fail to perform the remedial steps, the upgrade process may corrupt your existing usage data.
  
• Before upgrading to PCF v1.9, remove any product tiles that are no longer supported in PCF. See Review and Remove Unsupported Products.
  
  Some partner service tiles may currently be incompatible with PCF v1.9. Pivotal is working with partners to ensure their tiles are being updated to work with the latest versions of PCF.
  
  For information about which partner service releases are currently compatible with PCF v1.9, review the appropriate partners services release documentation at http://docs.pivotal.io, or contact the partner organization that produces the tile.
• Before upgrading to PCF v1.9, ensure that you have adequate space available on your database server. See Cloud Foundry API Availability During Upgrade.
• Due to the Cloud Foundry API database migration, the CF API will return 500 status codes while the Cloud Controller VMs are being upgraded. In addition, V3 service bindings are removed without notifying the related broker. Pivotal recommends deleting any V3 service bindings before upgrading. Although existing v2 applications still run normally during the upgrade, some developer interactions with PCF such as pushing apps to PCF may fail. You may want to warn users about potential availability issues during the upgrade window.
• Blobstore usage will increase as part of this release. This change is in preparation for upcoming features, such as application rollbacks, which require multiple versions to be available. Cloud Foundry now stores up to six versions of the latest droplet and package per application. Prior to installing this release, consider current blobstore usage and the implications of storing extra droplets and packages, it is likely necessary to increase the available storage, especially when using the WebDAV blobstore.

New Features in Elastic Runtime v1.9.0

This section describes new features of the release.

Advanced Features

The Advanced Features section of the Elastic Runtime tile includes new functionality that may have certain constraints. Although these features are fully supported, Pivotal recommends caution when using them in production.

Security Improvements for Intercomponent Communication

The etcd cluster now uses TLS for all communication between servers and clients.

To support the migration from unencrypted to encrypted communication, existing etcd servers are converted into proxies. These proxies allow connections from clients that still communicate unencrypted.

Important: The etcd proxy needs to remain deployed until such time as all clients in your deployment have switched to encrypted communications.

Additionally, the Diego Cell VMs now provide a secured API endpoint to communicate with the Diego Brain VM. The Cells provide both encrypted and unencrypted channels of communication in this release.

A complete switch-over to TLS for this communication channel will follow in a subsequent release.

Containers Default to Running Unprivileged

In PCF v1.9, all newly staged containers run as unprivileged containers.

Unprivileged containers are a security technique of mapping the root user inside the container to a regular user who has no privileges at the Linux operating system level. This prevents apps from inheriting root access on the host if an app breaks out of the container.

By using the full set of user-namespacing features in Linux, PCF isolates containers sharing the same host.

Cloud Foundry API Availability During Upgrade

The database backing the CF API performs a backwards-incompatible migration when upgrading from v1.8.x. The purpose of this large database migration is to prepare for the release of the Cloud Controller V3 API.

During the migration, you should expect to see 500 status codes from the API while the Cloud Controller VMs are being deployed.

Here are a couple things to note before performing the upgrade:

• Ensure there is adequate space available on the database server. This depends on the type of database you use and the amount of data you are migrating. For example, Postgres creates copy tables that can temporarily consume double the size of the apps table.

• V3 service bindings are removed without notifying the related broker. Pivotal recommends deleting any V3 service bindings before upgrading.

Support for Running Tasks

In addition to managing long-running processes, PCF now has first-class support for running one-off tasks.

Tasks are useful to application developers looking to perform defined operations with finite lifetimes such as database migrations, batch jobs, or seeding blobstores.

The task itself is guaranteed by the platform to run at most once and results in either a success or failure status.

When a task is executed, it inherits the environment of its associated application, including environment variables and filesystem.

For more information, see the Running Tasks topic.

.NET Core Buildpack

The .NET Core Buildpack is now included as a core buildpack for application developers. This buildpack is used by applications running .NET Core on Linux.

The buildpack is only new for operators who have not already upgraded to Elastic Runtime v1.8.13 or higher.

For more information, see the documentation for the .NET Core Buildpack.

Support for Spring Boot Actuators in Apps Manager

Apps Manager now securely integrates the /info, /health, and /logger Spring Boot actuators for apps that use Spring Boot v1.5 or later.

If an app is configured with these actuators, users will see the following in Apps Manager:

• Info: Git commit information will be available in the app page header, and on the app settings tab. Raw output of the info actuator is also available on the app settings tab.
• Health: The output of the health actuator for each instance will be displayed in the instances table on the summary tab of an app. The instance rows can be expanded to show the full details of the health endpoint for any instance.
• Logger: On the logs tab, the Configure Logging button will appear, allowing the user to temporarily configure log levels for the application’s loggers without a push or restart.

For more information, see the documentation for Spring Boot Actuators and Using Actuators.

New Autoscaling Features

The autoscaling service provides new scaling metrics for application developers.

In addition to CPU Utilization, the service now supports HTTP Throughput and HTTP Latency as metrics to decide application scaling events.

• HTTP Throughput measures the number of incoming HTTP requests per second per application instance.
• HTTP Latency measures application response latency at the 99th and 95th percentiles.

For all metrics, application developers can set upper and lower thresholds to decide whether the service should scale the application.

All metrics are computed as a 30-second rolling average. With the new Standard plan, rules are evaluated every 5 seconds.

The autoscaling service now also allows for multiple metrics to be combined for a single application.

Finally, to facilitate the automation of the interactions with the autoscaling service, the service now supports a REST-based API to manage your application scaling configuration.

For more information, see the Scaling an Application Using Autoscaler topic.

Zipkin Tracing Headers

With the Zipkin tracing option enabled, the router attaches tracing headers to incoming requests.

Enabling this option allows application developers who have Zipkin-compatible instrumentation in their application to trace requests as they traverse the platform.

For more information, see Enabling Zipkin Tracing.

Known Issues

This section lists new and existing known issues for Elastic Runtime.

New Issues

• When BOSH is restarted, LogSearch may fail to reconnect to BOSH when it restarts. As a result, after an upgrade to PCF v1.9, not all expected logs may appear in LogSearch. See Upgrading Pivotal Cloud Foundry for the post-upgrade workaround step.
• There is a naming discrepancy in the .Net Core buildpack between v1.8 and v1.9. The buildpack is referred to as “dotnet-core-buildpack” in v1.8 and “dotnet_core_buildpack” in v1.9. This will result in 2 versions of the buildpack being created after an upgrade to v1.9. Operators should remove the older buildpack after completing the upgrade process, making sure to migrate all applications onto the newer buildpack.

Existing Issues

• If you choose to enable TCP routing, you must also add the TCP routing domain via the cf CLI after deploying Elastic Runtime. The TCP routing domain is not configurable in the Elastic Runtime tile.
• Consul server cluster can fail to recover from quorum loss. See the Consul fails to start during upgrade in PCF knowledge base article for how to recover from this issue.
• etcd and Consul clusters do not self-heal in cases where they happen to enter split-brain.
• Disaster recovery for etcd or Consul clusters still requires manual intervention.
• The cf CLI command for viewing application files, cf files, does not work with apps on Diego.

Truncated Syslog Messages

If the total length of a syslog message transported locally from a PCF system component (for example, the Cloud Controller or a Diego cell) is greater than 1 KB, the packet is truncated before it reaches RSYSLOG installed on every BOSH VM instance.

When a job generates a log message, it typically writes the message in two locations: to the /var/vcap/sys/log directory and to RSYSLOG. For writing log messages directly to RSYSLOG, jobs use logger, an Ubuntu utility. Log messages sent through logger may be truncated as explained below:

• If jobs are using the default version of logger installed on the stemcell, log messages longer than 1 KB are truncated because the utility has a hard-coded message length limit.
• If jobs are using a newer version of logger without this restriction or other tool to communicate with RSYSLOG over UDP, the truncation may not happen.

As mentioned above, jobs write system logs to the /var/vcap/sys/log directory. You can download full log lines from the directory files using Ops Manager.

Mount Location Bug

If you have the VOLUME ["/some/data/dir"] directive in your image, ensure your "mount" section in your bind JSON matches what is in the Dockerfile, such as "mount":"/some/data/dir".

See this example Dockerfile on GitHub for reference.

This only works if the path in question is in an already existing root level folder. In other words, "/var/some/random/path" should work, but "/some/random/path" will cause your app to crash at startup. This limitation is due to an aufs bug.

Using Self-Signed Certs could cause Cloud Controller Downtime

If you have installed Elastic Runtime versions 1.9.31 or higher, and are using self-signed certificates for your system domain, then you will experience downtime. Elastic Runtime v1.9.34 resolves this issue.