PCF Ops Manager v2.2 Release Notes

Pivotal Cloud Foundry is certified by the Cloud Foundry Foundation for 2018.

Read more about the certified provider program and the requirements of providers.

Note: Ops Manager API documentation is now public. For more information, see PCF v2.2 Feature Highlights.


How to Upgrade

The Upgrading Pivotal Cloud Foundry topic contains instructions for upgrading to Pivotal Cloud Foundry (PCF) Ops Manager v2.2.

Releases

2.2.9

  • [Security Fix]: Bumps Nokogiri to 1.8.5 to address CVE-2018-14404.
  • [Security Fix] Bumps UAA to 57.6 to address CVE-2018-15761.
  • [Bug Fix]: Now Application Load Balancers (ALBs) also apply to the Director VM for AWS deployments.

Ops Manager v2.2.9 uses the following component versions:

Component Version
Ops Manager2.2-build.359*
Stemcell3586.52*
BBR SDK1.6
BOSH Director266.13
BOSH DNS1.10.0
Metrics Server0.0.21
CredHub1.9.3
Syslog11.3
UAA57.6*
AWS CPI70
Azure CPI35.4
GCP CPI27.0.1
OpenStack CPI38
vSphere CPI50
* Components marked with an asterisk have been updated.

2.2.8

  • [Security Fix]: Bumps stemcell to 3586.48 to address USN-3777-2.
  • [New Feature]: Operators can tune the swap size as a percent of total memory size per instance group.
  • [Bug Fix]: Bumps Azure CPI up to 35.4 to fix LockTimeoutError issues.
  • [Bug Fix]: Operators can change the Director Hostname without losing connection between BOSH Director and VMs.
  • [Bug Fix]: Stemcells no longer accidentally downgrade when upgrading to a new Ops Manager. This rare bug occurred when a product had a newer stemcell patch than Ops Manager included during the upgrade.
  • [Bug Fix]: Operators can work around an expired SAML service provider cert by disabling and enabling SAML.
  • [Feature Improvement]: The expiring certificates endpoint (/api/v0/deployed/certificates) now includes information about the SAML service provider cert.
  • [Feature Improvement]: When you import products that use the future Unified Syslog feature, you are warned that some syslog features will not be active in this version of Ops Manager.
  • [Bug Fix]: Dynamic JS pages now show the message from server-side errors instead of alert boxes with JavaScript errors (such as [Object object] or t.filter()).
  • [New Feature]: You can now configure custom DNS handlers via the Ops Manager API.
  • [New Feature]: You can now configure recursor timeouts via the Ops Manager API.

Ops Manager v2.2.8 uses the following component versions:

Component Version
Ops Manager2.2-build.339*
Stemcell3586.48*
BBR SDK1.6
BOSH Director266.13*
BOSH DNS1.10.0
Metrics Server0.0.21
CredHub1.9.3
Syslog11.3
UAA57.4
AWS CPI70
Azure CPI35.4
GCP CPI27.0.1
OpenStack CPI38
vSphere CPI50
* Components marked with an asterisk have been updated.

2.2.7

  • [Bug Fix]: You are now only prompted to unlock Ops Manager once when enabling Rescue Mode.
  • [Bug Fix]: Ops Manager sets the storage account type and Director ephemeral disk correctly for Azure deployments.

Ops Manager v2.2.7 uses the following component versions:

Component Version
Ops Manager2.2-build.334*
Stemcell3586.43*
BBR SDK1.6
BOSH Director266.12
BOSH DNS1.10.0*
Metrics Server0.0.21
CredHub1.9.3
Syslog11.3
UAA57.4
AWS CPI70
Azure CPI35.4*
GCP CPI27.0.1
OpenStack CPI38
vSphere CPI50
* Components marked with an asterisk have been updated.

2.2.6

  • [Feature]: Operators can rotate BOSH DNS healthiness certificates to a new certificate authority (CA) that is valid for four years.
  • [UI Enhancement]: An error message appears when Ops Manager fails to import an installation.
  • [UI Enhancement]: An error message appears when a file downloaded from Pivotal Network is invalid or corrupt.

Ops Manager v2.2.6 uses the following component versions:

Component Version
Ops Manager2.2-build.319*
Stemcell3586.40
BBR SDK1.6
BOSH Director266.12*
BOSH DNS1.8
Metrics Server0.0.21
CredHub1.9.3
Syslog11.3
UAA57.4
AWS CPI70
Azure CPI35.2
GCP CPI27.0.1
OpenStack CPI38
vSphere CPI50
* Components marked with an asterisk have been updated.

2.2.5

  • [Feature]: New CAs for BOSH DNS healthiness and DNS API apply automatically on upgrade. These CAs are valid for four years.
  • [Feature]: All DNS healthiness certificates are signed by a Credhub CA.
  • [Feature]: Operators can rotate DNS healthiness certificates using POST/api/v0/certificate_authorities/active/regenerate.
  • [Feature Improvement]: Ops Manager API warns you when you attempt to regenerate certificates without first applying changes to propagate CA changes.
  • [Bug Fix]: Verifiers work in vCenter v6.7. Fixes the Ops Manager “Required Datacenter privileges” Error on vSphere known issue.

Ops Manager v2.2.5 uses the following component versions:

Component Version
Ops Manager2.2-build.316*
Stemcell3586.40
BBR SDK1.6
BOSH Director266.10
BOSH DNS1.8
Metrics Server0.0.21
CredHub1.9.3
Syslog11.3
UAA57.4
AWS CPI70
Azure CPI35.2
GCP CPI27.0.1
OpenStack CPI38
vSphere CPI50
* Components marked with an asterisk have been updated.

2.2.4

  • [Security Fix] Bumps stemcell to 3586.40.
  • [Bug Fix] Pivotal Network integrates successfully with Pivotal Application Service (PAS) tile and Small Footprint PAS.
  • [Feature Improvement] You can use the Ops Manager API to delete individual OpenStack or vCenter Configs. For more information, see Deleting IaaS Configuration in the Ops Manager API documentation.
  • [Bug Fix] You cannot import an installation with no deployed products.

Ops Manager v2.2.4 uses the following component versions:

Component Version
Ops Manager2.2-build.312
Stemcell3586.40*
BBR SDK1.6
BOSH Director266.10
BOSH DNS1.8
Metrics Server0.0.21*
CredHub1.9.3
Syslog11.3
UAA57.4
AWS CPI70
Azure CPI35.2
GCP CPI27.0.1
OpenStack CPI38
vSphere CPI50
* Components marked with an asterisk have been updated.

2.2.3

  • [Security Fix] Bumps stemcell to 3586.36

Ops Manager v2.2.3 uses the following component versions:

Component Version
Ops Manager2.2-build.304
Stemcell3586.36*
BBR SDK1.6
BOSH Director266.10*
BOSH DNS1.8*
Metrics Server0.0.17
CredHub1.9.3
Syslog11.3
UAA57.4
AWS CPI70
Azure CPI35.2
GCP CPI27.0.1*
OpenStack CPI38
vSphere CPI50
* Components marked with an asterisk have been updated.

2.2.2

Ops Manager v2.2.2 uses the following component versions:

Component Version
Ops Manager2.2-build.300*
Stemcell3586.27*
BBR SDK1.6
BOSH Director266.8.0*
BOSH DNS1.6
CredHub1.9.3
Syslog11.3
UAA57.4
AWS CPI70
Azure CPI35.2
GCP CPI27
OpenStack CPI38
vSphere CPI50
* Components marked with an asterisk have been updated.

2.2.1

  • [Bug Fix]: Fixes critical manifest generation grammar issue.
  • [Bug Fix]: You can now delete an unused AZ in an installation after clicking Apply Changes.
  • [Bug Fix]: Certain VM image components no longer write to the persistent disk after reboot.
  • [Bug Fix]: Ops Manager now verfies certificates successfully when connecting to S3 blobstores with TLS. This resolves a known issue in Ops Manager v2.2.0. For more information, see Operations Manager Validation returns TLS error when configuring Bosh Director S3 blobstore.
  • [Security Fix]: Bumps Nokogiri to 1.8.4 to remediate CVE-2017-15412.
  • [Feature Improvement]: Installation Dashboard and deployment status pages may load more quickly.

Ops Manager v2.2.1 uses the following component versions:

Component Version
Ops Manager2.2-build.296
Stemcell3586.25*
BBR SDK1.6
BOSH Director266.6*
BOSH DNS1.6
CredHub1.9.3
Syslog11.3
UAA57.4*
AWS CPI70
Azure CPI35.2
GCP CPI27
OpenStack CPI38
vSphere CPI50
* Components marked with an asterisk have been updated.

2.2.0

Ops Manager v2.2.0 uses the following component versions:

Component Version
Stemcell3586.24
BBR SDK1.6
BOSH Director266.5
BOSH DNS1.6
CredHub1.9.3
Syslog11.3
UAA57.3
AWS CPI70
Azure CPI35.2
GCP CPI27
OpenStack CPI38
vSphere CPI50


New Features in Ops Manager v2.2

Ops Manager v2.2 includes the following major features:

Multiple Data Centers on vSphere

Ops Manager now allows you to configure multiple vSphere vCenters to a single BOSH Director.

You can add additional data centers in the vSphere Config pane of your vSphere BOSH Director tile. For more information about how to add, edit, and delete vCenters, see Managing Multiple vSphere vCenters.

Note: If you use the Ops Manager API and multiple vSphere configs exist, the GET HTTP request for Director properties omits the iaas_configuration key.

Selectively Deploy Tiles in Ops Manager or via an API Endpoint

You can now choose to deploy a selection of tiles rather than all tiles in Ops Manager. If you choose to selectively deploy your environment, you can drastically reduce the time to Apply Changes. This feature is ideal to limit updates to one or more tiles, which reduces the amount of change in any given deployment.

To access this feature, click Review Pending Changes underneath the Apply Changes button in the Ops Manager Installation Dashboard. For more information, see Reviewing Pending Changes with Ops Manager.

In the Ops Manager UI, this feature is in beta. It is generally available as an API endpoint. To selectively deploy tiles via the API, send a POST to /api/v0/installations. For more information, see Triggering an install process in the Ops Manager API documentation.

WARNING: Do not selectively deploy tiles when upgrading to PCF v2.2. Instead, redeploy all product tiles using Apply Changes on the Ops Manager Installation Dashboard. For more information, see Redeploy All Products After Upgrading to Ops Manager v2.2.

Note: Ops Manager is soliciting feedback for this feature. Submit feedback through your product architect or directly by emailing opsmanager-feedback+selective_deploys@pivotal.io.

Ops Manager Stores Past Manifests

Through the Ops Manager API, you can see Ops Manager’s manifest history. Manifest history is helpful for running diff commands on manifests to see changes over time.

For this feature, use the following Ops Manager API endpoints:

Azure Stack is Generally Available

Pivotal officially supports Azure Stack.

Azure Stack is a hybrid cloud platform that lets you deliver Azure services from your own on-premise datacenter. For more information about Azure Stack, see What is Azure Stack? from the Microsoft Azure documentation.

You can configure Azure Stack through the BOSH Director for Azure tile. For more information about Azure Stack-specific configurations, see the steps in the Azure Config Page section of the Configuring BOSH Director on Azure topic.

Ops Manager Supports Azure China

Ops Manager now supports a special region in Azure called Azure China. Azure China is a physically separated instance of cloud services that is located in China and independently operated. For more information about Azure China, see What is Azure China 21Vianet? in the Azure China documentation.

To tell the BOSH Director that you are using an Azure China environment, go to the BOSH Director for Azure tile and select Azure China Cloud from the Azure Environment field. For more information, see Azure Config Page in the Configuring Ops Manager on Azure manual installation topic.

Ops Manager Credentials Stored in CredHub

On each Apply Changes, Ops Manager sends your user-specified credentials to BOSH CredHub. This feature offers greater security for your credentials. For more information about where Ops Manager stores your credentials, see BOSH CredHub.

For information about how this feature affects tile authors, see PCF v2.2 Partners Release Notice in the PCF Tile Developer Guide.

Multi-Line Credentials

Ops Manager v2.2 now supports text areas for any type of multi-line credential. If you want a secret property to use a text area instead of the default single-line text field, you must set display_type to text_area in the property_inputs section of your property blueprint, as in the example below.

property_inputs:
  - reference: secret_meaning
    label: 'Secret Meaning'
    description: 'If you play it backwards...'
    display_type: 'text_area'

For more information, see the Custom Forms and Properties section of the Tile Generator topic.

Specify a Custom Trusted SSL Certificate

Operators can specify a custom trusted SSL certificate and key for the Ops Manager server so that traffic isn’t exposed to man-in-the-middle attacks when using Ops Manager.

By default, Ops Manager uses an auto-generated self-signed certificate. To change this configuration to your own SSL certificate, navigate to Settings from the Ops Manager Installation Dashboard and select the SSL Certificate pane to enter your Certificate and Private Key.

For more information about navigating the Ops Manager Settings page, see Settings Page in the Understanding the Ops Manager Interface topic.

Note: Custom SSL certificate and key is persisted between upgrades. Custom SSL only needs a one-time configuration.

Delete Your Pivotal Network API Token

You can now delete your Pivotal Network API token, along with the Pivotal Network release dashboard and all of the tile metadata from Pivotal Network products.

For more information, see Settings Page in the Understanding the Ops Manager Interface topic.

Configure an Ops Manager Syslog Server

You can configure a syslog server for Ops Manager logs. Logs include rails production logs, audit logs, UAA logs, nginx logs, and upstart logs for Ops Manager processes as well as additional log types. Previous to this change, Ops Manager logs were not centralized in one accessible location. You also have the option to TLS-encrypt your logs.

To configure syslog for Ops Manager, go to Syslog from Ops Manager Settings, select Yes to enable syslog and fill the required fields. Only administrators can view the Syslog pane.

For more information about configuring syslog for Ops Manager, see Settings Page in the Understanding the Ops Manager Interface topic.

Note: When you enter your syslog credentials, Ops Manager does not validate them. You should test your syslog server to ensure that the credentials were entered correctly and the server is receiving Ops Manager logs.

Breaking Change: If you were running scripts to get Ops Manager logs, those scripts break on upgrade to Ops Manager v2.2 and later.

Xenial Stemcell Upgrade Support

As of April 2019, Trusty stemcells will no longer receive support, nor will Pivotal have CVE patches for them. Ops Manager v2.2 allows tile authors to upgrade from Trusty stemcells to Xenial stemcells.

TLS for Internal Blobstore Supported

Ops Manager now supports TLS communications if you choose to use an internal blobstore.

To enable internal blobstore TLS communication, all of your tiles must have stemcell v3586 or later. You can configure internal TLS by clicking Enable TLS in the Director Config pane of the BOSH Director tile.

Custom TLS Certificate for External MySQL Database Supported

Ops Manager now allows you to configure a custom TLS certificate for an external MySQL database.

To configure a custom TLS certificate, navigate to Director Config > Database Location and select External MySQL Database to fill in the relevant fields.

Note: You must select Enable TLS for Director Database to configure the TLS-related fields.

For more information, see the Director Config Page section of the Ops Manager Director installation topic for your IaaS.

UI Improvements to Installation Dashboard

The following lists UI changes to the Ops Manager Installation Dashboard:

  • Stemcell Library is persistently in the page header. You can now access Stemcell Library from anywhere in Ops Manager.
  • Changelog is persistently in the page header. You can now access the changelog from anywhere in Ops Manager.
  • Review Pending Changes BETA button is below Apply Changes. For more information about this feature, see Selectively Deploy Ops Manager Tiles.
  • Azure Logo is updated.
  • BOSH Director tile name is changed to “BOSH Director for YOUR_IAAS”.
  • Changelog page shows tiles which were not changed but were still deployed.

For more information about the Ops Manager UI, see Installation Dashboard Page in the Understanding the Ops Manager Interface topic.

Change Log Includes Products Deployed but Unchanged

The Change Log pane lists products as Unchanged when they remain deployed, but their configuration has not changed from a prior deployment, so Ops Manager did not re-deploy them.

More Detail Available Via Ops Manager API Endpoint

A new API endpoint is available for Ops Manager. Send a GET to /v0/staged/pending_changes to see details about your Ops Manager installation, including tile names, errand names, build version, and deployment status. The API response will show information on all tiles, whether they are deployed or have pending changes.

For more information about setting up the Ops Manager API, see Using the Ops Manager API.

Custom Identification Tags Supported

You can specify a single set of tags that apply to all VMs and disks for your foundation. Identification tags allow you to easily identify which foundation your VMs belong to when viewing your IaaS. You are able to set custom Identification Tags in the Director Config pane of your BOSH Director tile.

For more information about configuring identification tags, see the Director Config Page section of the Ops Manager Director installation topic for your IaaS.

BOSH DNS Enabled By Default

BOSH DNS is enabled by default for both app containers and PCF components in PCF v2.2.

In previous versions, Consul managed service discovery between PCF components, but Consul is being replaced by BOSH DNS.

Note: In PCF v2.2, Consul and BOSH DNS are both available in PCF, but BOSH DNS is the only service used for DNS requests.

You can disable BOSH DNS if instructed to do so by Pivotal support. If you disabled BOSH DNS in PCF v2.1, reenable it before upgrading to PCF v2.2. For more information, see BOSH DNS Enabled By Default.

WARNING: Do not disable BOSH DNS without instructions from Pivotal support. Disabling BOSH DNS will also disable PKS, NSX-T, and several PAS features.

“When Changed” Errand Setting Removed

Ops Manager no longer includes a When Changed option for tile errands. In the Errands pane for a given tile, you can set errands On to run them or Off to not run them. The default setting is On.

Known Issues

DNS Server Hangs or DNS Lookups Fail

With BOSH DNS, every BOSH-deployed VM has a DNS server. In large PCF installations, this DNS server may hang or DNS lookups may fail when the VM experiences too many DNS lookups in a short amount of time.

This error is caused by a race condition and deadlock in the VM’s DNS server.

To fix this problem, run monit on the VM with failing DNS to restart its bosh-dns process.

Error When Importing Xenial Stemcell

Ops Manager v2.2.0 and later support Xenial stemcells. However, the Ops Manager UI returns an error when you attempt to import a Xenial stemcell.

As a workaround, you can upload the stemcell and assign it to a product using the Ops Manager API.

This issue is fixed in Ops Manager v2.2.2.

Ops Manager Validation Returns TLS Error When Configuring BOSH Director S3 Blobstore

If a remote S3 blobstore uses a privately signed SSL certificate, operators see an error when configuring the BOSH Director to use an S3 blobstore.

The error reads: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (OpenSSL::SSL::SSLError) Unable to verify certificate. This may be an issue with the remote host or with Excon. Excon has certificates bundled, but these can be customized:

This error appears because Ops Manager attempts to validate the S3 blobstore by testing the SSL certificate. Ops Manager does not use trusted certificates to make this connection, so the connection fails.

A workaround is available for this issue. Operators can install the public CA certificate directly into the OS config of Ops Manager by following these steps:

  1. SSH into the Ops Manager VM.
  2. Copy the public CA certificate into /etc/ssl/certs.
  3. Run sudo update-ca-certificates -f -v. This installs the new CA certificate.

Upon successful execution, “1 added” displays in the output. For example: Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done.

This indicates the new certificate has been installed.

For more information, see the Knowledge Base article Operations Manager Validation Returns TLS Error When Configuring BOSH Director S3 Blobstore.

Ops Manager Deployment Fails Because Monit Reports Job as Failed

This issue causes Ops Manager deployments to fail with an error indicating one or more jobs are not running after an update.

The error reads: Error: 'cloud_controller/6632bf71-7493-4383-a3f9-9401bafb4710 (1)' is not running after update. Review logs for failed jobs: cloud_controller_ng

Additionally, when you SSH into a VM and run monit summary, monit reports jobs as “Execution Failed”.

To remediate this issue, use monit to restart the affected processes.

For more information, see the Knowledge Base article Deployment Fails Because Monit Reports Job as Failed.

Ops Manager “Required Datacenter privileges” Error on vSphere

Ops Manager on vSphere v6.7 fails with an error message: “Could not log in: Required Datacenter privileges could not be verified: SystemError: A general system error occurred: Authorize Exception”

You can ignore this error message. Click “Ignore errors and start the install” to authenticate.

This issue is fixed in Ops Manager v2.2.5 and v2.3.0 or later.

Create a pull request or raise an issue on the source for this page in GitHub