Registering and Emitting Custom App Metrics

This topic describes how to install the Metric Registrar CLI plugin and emit custom app metrics to the Metric Registrar by registering your app as a metric source.

Overview

Registering your app as a metric source allows you to see your custom metrics in PCF Metrics and configure autoscaling rules with PCF Autoscaler.

For more information, see PCF Metrics and PCF Autoscaler.

Install the Plugin

To install the Metric Registrar CLI plugin, do the following:

  1. Log in to the Cloud Foundry Command Line Interface (cf CLI).

  2. Run the following command:

    cf install-plugin -r CF-Community "metric-registrar"
    

Register Your App

To register your app as a metric source, do one of the following:

Note: If you are migrating from and manually send data to Metrics Forwarder for PCF, Pivotal recommends registering a structured log format. See Register a Structured Log Format.

Register a Public Metrics Endpoint

The Metric Registrar supports custom metrics created with the open-source tool, Prometheus. Prometheus uses a text-based exposition format common in many open-source libraries. It also provides several out-of-the-box metrics for different programming languages.

For more information about Prometheus, see What is Prometheus? in the Prometheus documentation.

For examples of apps that use Prometheus to publish metrics to an endpoint, see metric-registrar-examples in GitHub.

Note: These endpoints must be public for the Metric Registrar to retrieve the data. If you do not want to expose public endpoints for your app, see Register a Structured Log Format below.

Prerequisites

Before registering a public metrics endpoint, you must do the following:

  • For all Spring apps, update the application.yml file to include one or more Prometheus endpoints. For example:

    management:
      endpoints:
        web:
          exposure:
            include: "metrics,prometheus"
      endpoint:
        metrics:
          enabled: true
        prometheus:
          enabled: true
    
  • For all Spring apps, update the security configuration file to permit access to the Prometheus endpoints. For an example, see metric-registrar-examples in GitHub.

  • For all other apps, see [client libraries] provided by the open-source community.

Register a Public Metrics Endpoint

To register a public metrics endpoint for an app, do the following:

  1. Log in to the cf CLI.

  2. For each Prometheus endpoint in your app, run the following command to register the endpoint as a metric source:

     cf register-metrics-endpoint APP-NAME PATH
    

    Where:

    • APP-NAME is the name of the app.
    • PATH is the path to the Prometheus endpoint.

    For example, if the app name is example and the metrics endpoint lives at https://example.com/metrics the command would be:

    ```
    cf register-metrics-endpoint example /metrics
    ```
    

Verify a Public Metrics Endpoint

Pivotal recommends that you install the LogCache cf CLI plugin to view metrics.

To install and use the Log Cache cf CLI plugin, do the following:

  1. To install Log Cache, run the following command:

    cf install-plugin -r CF-Community "log-cache"
    
  2. To view metrics for an app, run the following command:

    cf tail --envelope-class=metrics <APP_NAME> 
    

    For example, if you provided a custom metric called users_per_cpu as a gauge, you should see the following output:

    2019-06-18T10:49:28.27-0600 [app-name/1] GAUGE cpu:0.154763 percentage disk:14000128.000000 bytes disk_quota:33554432.000000 bytes memory:10190848.000000 bytes memory_quota:33554432.000000 bytes
    2019-06-18T10:49:26.42-0600 [app-name/1] GAUGE users_per_cpu:557.500000

    Where the first line is default container metrics and the second line is custom metrics users_per_cpu.

    Note: The Metric Registrar produces the following benign error message at every polling interval: [LGR/] ERR Invalid syslog drain URL: parse failure

Register a Structured Log Format

The Metric Registrar supports metrics emitted in JSON or DogStatsD formats. For more information about these formats, see the JSON and DogStatsD sections below.

To register your app as a metric source, do the following:

  1. Log in to the cf CLI.

  2. Run the following command:

    cf register-log-format APP-NAME FORMAT
    

    Where:

    • APP-NAME is the name of the app.
    • FORMAT is either json or DogStatsD.
  3. In your app, log a structured json or DogStatsD message to represent the custom metric.

JSON

The table below shows the supported JSON format for event, gauge, and counter log types.

Type Format
Events
{
  "type": "event",
  "title": "title",
  "body": "body",
  "tags": {
    "tag1": "tag value"
  }
}
Gauges
{
  "type": "gauge",
  "name": "some-counter",
  "value": ,
  "tags": {
    "tag1": "tag value"
  }
}
Counters
{
  "type": "counter",
  "name": "some-counter",
  "delta": ,
  "tags": {
    "tag1": "tag value"
  }
}

DogStatsD

The table below shows the supported DogStatsD format for event, gauge, and counter log types. It also lists the supported fields. For more information about DogStatsD, see the DogStatsD topic in the Datadog documentation.

Type Format Supported Fields
Events
_e{title.length,text.length}:title|text|d:timestamp|
h:hostname|p:priority|t:alert_type|#tag1,tag2
title
text
Gauges
gauge.name:value|g|@sample_rate|#tag1:value,tag2
gauge.name
value
Counters
counter.name:value|c|@sample_rate|#tag1:value,tag2
counter.name
value