LATEST VERSION: 1.4 - CHANGELOG
PCF Metrics v1.3

Installing PCF Metrics

This document describes how to install and configure Pivotal Cloud Foundry (PCF) Metrics.

For information about the components deployed as part of this install procedure, see the PCF Metrics Product Architecture topic.

Prerequisites

  • Ensure that you have installed the Elastic Runtime Tile.
  • Ensure that you have installed v1.6 or later of the Redis tile.
  • If you are running PCF on AWS, then ensure that, in Elastic Runtime, you have changed the Loggregator Port to 4443from its value of 443.
  • If you are running PCF on Google Cloud Platform (GCP), then do the following to configure the DNS entries to accommodate web sockets:
    1. Log in to the GCP console.
    2. In the menu, navigate to the Networking tab and click Load Balancing.
    3. Find the load balancer that corresponds to ENVIRONMENT-cf-ws.
    4. Record the IP address.
    5. Click Cloud DNS, then click ENVIRONMENT-zone.
    6. Click Add Record Set.
    7. Enter a DNS name for mysql-logqueue. The DNS name should be mysql-logqueue.SYSTEM_DOMAIN. Refer to your ERT Tile’s configuration of the System Domain under the Domains configuration section.
    8. In the IPv4 address field, enter the IP address of the load balancer that you recorded in Step 4.
    9. Leave the other fields as default.
    10. Repeat Steps 6–9 twice to create DNS records for elasticsearch-logqueue (elasticsearch-logqueue.SYSTEM_DOMAIN) and metrics (metrics.SYSTEM_DOMAIN).

Step 1: Add the PCF Metrics Tile to Ops Manager

Note: PCF Metrics should be installed on the same network as the Elastic Runtime Tile.

  1. Download the PCF Metrics file from Pivotal Network.
  2. Upload the PCF Metrics file to your Ops Manager installation.
  3. Click Add next to the uploaded product description in the Available Products view to add PCF Metrics to your Installation Dashboard.

Step 2: Configure the PCF Metrics Tile

Note: The following procedures offer a standard configuration. To customize PCF Metrics for high capacity, see the Sizing PCF Metrics For Your System topic.

From the Installation Dashboard, click the PCF Metrics tile.

Assign Availability Zones (AZs) and Networks.

  1. Click Assign AZs and Networks.
  2. Select an Availability Zone under Place singleton jobs.
    Ops Manager runs Metrics jobs with a single instance in this Availability Zone.
  3. Select one or more Availability Zones under Balance other jobs.
    Ops Manager balances instances of Metrics jobs with more than one instance across the Availability Zones that you specify.
  4. Use the drop-down menu to select a network.
  5. Click Save.

Data Services Ports

For reference, the following table shows the port associated with each data service.

Service Port
Elasticsearch 9200
MySQL 3306

MySQL Alerts

  1. Click MySQL Alerts.
  2. Set the Email value. Alerts for issues storing metrics into the MySQL cluster will be sent to this email address.

Data Store

  1. Click Data Store.
  2. Review the Elastic Search Heap Size value. Elastic Search memory allocation for Heap use. Set to 50% of the memory allocated to the smallest of the Elasticsearch instances in Resource Config or 31GB, whichever is smaller. Use a unit of M for megabytes or G for gigabytes.
  3. Review the MySQL InnoDB Buffer Size value. Set to 80% of MySQL Server VM memory. Use a unit of M for megabytes or G for gigabytes.
  4. Review the MySQL Logqueue Count value. You can increase this instance count at any time to accommodate higher levels of inbound metrics traffic.
  5. Review the Elasticsearch Logqueue Count value. You can increase this instance count at any time to accommodate higher levels of inbound log traffic.
  6. Review the Ingestor Count value. You can increase this instance count at any time to accommodate higher levels of Loggregator Firehose traffic.
  7. Click Save.

Errands

To properly configure the Errands pane, follow the procedure that corresponds to your Ops Manager version.

Ops Manager v1.9

  1. Click Errands.

    Note: The PCF Metrics tile selects all Post-Deploy Errands by default. Pivotal recommends that you do not deselect any errands as doing so can cause issues with the deployment of the tile. However, you can deselect the Remove Legacy PCF Metrics CF Resources errand after deploying v1.3 of the tile.

  2. Review the Post-Deploy Errands and Pre-Delete Errands:

    1. If you are deploying the tile for the first time, select all Post-Deploy Errands.

      • The following list describes what the Smoke tests errand does. For information about resolving errors discovered by this errand, see the Smoke Test Errors section of the Troubleshooting PCF Metrics topic.
        • Confirms that MySQL ingests metrics
        • Confirms that Elasticsearch ingests logs
        • Confirms that the APIs return metrics and logs

      Note: If you deselect Remove PCF Metrics 1.3 CF Resources under Pre-Delete Errands, artifacts may remain after the PCF Metrics tile uninstalls.

Ops Manager 1.10 and up

  1. Click Errands.

    WARNING: By default, The PCF Metrics tile sets all Post-Deploy Errands to only run when changed. You must make the configuration changes below for the first deployment of the PCF Metrics tile to succeed.

  2. Review the Post-Deploy Errands and Pre-Delete Errands:

    1. If you are deploying the tile for the first time, set Post-Deploy Errands to On.
    2. For deployments after the initial install, configure the Post-Deploy Errands as follows:

      1. Set Remove Legacy PCF Metrics CF Resources to When Changed
      2. Set Remove PCF Metrics 1.2 CF Resources to When Changed
      3. Set Push PCF Metrics components to On
      4. Set Smoke tests for Metrics Data and Apm to On
        • The following list describes what the Smoke tests errand does. For information about resolving errors discovered by this errand, see the Smoke Test Errors section of the Troubleshooting PCF Metrics topic.
          • Confirms that MySQL ingests metrics
          • Confirms that Elasticsearch ingests logs
          • Confirms that the APIs return metrics and logs

      Note: If you set Remove PCF Metrics 1.3 CF Resources under Pre-Delete Errands to Off, artifacts may remain after the PCF Metrics tile uninstalls.

Resource Config

  1. Click Resource Config.
  2. Review the resource configurations. By default, the settings match the instance types that are best suited for each job. For reference, the following table shows the default resource and IP requirements for installing the PCF Metrics tile:

    Note: The actual default values you will see in your PCF Metrics tile will depend on the available resources for your PCF deployment. Refer to Sizing PCF Metrics For Your System to correctly configure Metrics resources for your deployment.

    Resource Instances Persistent CPU RAM Ephemeral Static IP Dynamic IP
    Elasticsearch Master 3 10 GB 4 16 GB 32 GB 3 0
    Elasticsearch Coordinator 2 1 GB 2 16 GB 32 GB 1 0
    Elasticsearch Data 4 100 GB 2 16 GB 32 GB 4 0
    MySQL Server 2 100 GB 2 16 GB 32 GB 3 0
    MySQL Proxy 2 n/a 2 16 GB 32 GB 2 0
    MySQL Monitor 1 (not configurable) n/a 2 16 GB 32 GB 0 1
    Metron 1 (not configurable) n/a 2 4 GB 32 GB 1 0

    If you expect a high level of use, you may need to increase the disk resources available to your instances. For information about sizing, see Sizing PCF Metrics For Your System.

    Note: There have been issues with the Ops Manager Bosh Director correctly partitioning persistent disks larger than 2 TB.

  3. Click Save.

Stemcell

  1. Navigate to Pivotal Network and click Stemcells.
  2. Download the appropriate stemcell version for your IaaS.

    Note: On AWS make sure to use a HVM stemcell if you are using the default instance sizes.

  3. Click Import Stemcell and select the stemcell file you downloaded.

Step 3: Configure the Temporary Datastore

PCF Metrics uses a temporary datastore during Elasticsearch downtime, including upgrades, to significantly reduce log loss by continuing to store app logs from the Loggregator Firehose.

To configure the temporary datastore, follow the instructions in the Configuring the Temporary Datastore section of Sizing PCF Metrics for your System.

Step 4: Deploy PCF Metrics

Click Apply Changes to install the service. If the smoke tests fail, see the Troubleshoot Smoke Test Errors section of the Troubleshooting PCF Metrics topic.

Review the Monitoring and Troubleshooting Apps with PCF Metrics topic for more information about how to log in, use, and interpret data from PCF Metrics.

Create a pull request or raise an issue on the source for this page in GitHub