Pivotal Platform Observability Fabric (Beta)

Page last updated:

This topic describes the components of Pivotal Platform Observability Fabric (Beta).

Overview

Pivotal Platform Observability Fabric (Beta) includes components that collect, store, and forward logs and metrics in your Pivotal Platform deployment.

Observability Fabric allows you to access all the same logs and metrics that you can access through the Loggregator system.

The components of Observability Fabric use a shared-nothing architecture that requires several fewer VMs than the Loggregator system.

For more information about the components of Loggregator, see Loggregator Architecture.

Metric Egress Components

The Loggregator system uses a push-based model for forwarding metrics, in which all data is sent though the Firehose.

Observability Fabric includes components that allow you to access the same metrics available through the Loggregator Firehose with a pull-based architecture.

You can also use Observability Fabric to automatically register Prometheus endpoints for metrics in Pivotal Application Service (PAS).

The following Observability Fabric components enable pull-based metrics egress:

  • Metrics Agent:
    The Metrics Agent collects Loggregator V2 envelopes and makes them available on a Prometheus endpoint. The Metrics Agent performs a similar function to the Loggregator Agent in the Loggregator system.

  • Service Metrics Agent:
    The Service Metrics Agent receives metrics from service instances on your Pivotal Platform deployment and makes them available on a Prometheus endpoint.

  • Metrics Discovery Registrar:
    The Metrics Discovery Registrar publishes the location of the Prometheus endpoint defined by the Metrics Agent and Service Metrics Agent to NATs. This is helpful for configuring automation to scrape metric data from the endpoint. For more information about automating metric scraping, see Telegraf Reference Architecture (Beta).

Log Egress Components

Observability Fabric uses additional components to enable log egress. You can also use these components with the Loggregator system. For more information about using these components as part of the Loggregator system, see Loggregator Architecture.

The following Observability Fabric components enable log egress:

  • Syslog Agent:

    Syslog Agents run on Pivotal Platform component VMs and host VMs to collect and forward logs to configured syslog drains. This includes syslog drains for individual apps as well as aggregate drains for all apps in your foundation. You can specify the destination for logs as part of the individual syslog drain or in the PAS tile.

  • Aggregate Syslog Drain:
    The aggregate syslog drain feature allows you to configure all Syslog Agents on your deployment to send logs to a single destination. You can use the aggregate syslog drain feature rather than the Loggregator Firehose to forward all logs for your deployment.

  • Log Cache:

    Log Cache allows you to view logs and metrics over a specified period of time. The Log Cache includes API endpoints and a CLI plugin to query and filter logs and metrics. To download the Log Cache CLI plugin, see Cloud Foundry Plugins. The Log Cache API endpoints are available by default. For more information about using the Log Cache API, see Log Cache on GitHub.

Observability Fabric Reference Architectures

This section includes sample reference architectures for using Observability Fabric with Telegraf or Prometheus to export logs and metrics from Pivotal Platform to an external source.

Telegraf Reference Architecture (Beta)

You can deploy Telegraf with Observability Fabric to use push-based metrics collection formats.

cf-telegraf-operator deploys Telegraf and a sidecar that manages the scrape configuration.

For more information, see cf-telegraf-operator on GitHub.