Log and Metric Agent Architecture (Beta)

Page last updated:

This topic describes the components of the Log and Metric Agent Architecture (Beta).

Note: The Log and Metric Agent Architecture (Beta) does not currently integrate with Healthwatch or App Metrics.

Overview

The Log and Metric Agent Architecture (Beta) contains a group of components that allow you to access all the same logs and metrics that you can access through the Loggregator system. The components of the Log and Metric Agent Architecture use a shared-nothing architecture that requires several fewer VMs than the Loggregator system.

The Log and Metric Agent Architecture includes components that collect, store, and forward logs and metrics in your Ops Manager deployment.

For more information about the components of Loggregator, see Loggregator Architecture.

Metric Components

This section describes the components of the Log and Metric Agent Architecture that allow you to access metrics for your foundation.

These components allow you to access the same metrics available through the Loggregator Firehose with a pull-based architecture. The Loggregator system uses a push-based model for forwarding metrics, in which all data is sent though the Firehose.

The following components of the Log and Metric Agent Architecture enable pull-based access to metrics:

  • Metrics Agent:
    The Metrics Agent collects Loggregator V2 envelopes and makes them available on a Prometheus endpoint. The Metrics Agent performs a similar function to the Loggregator Agent in the Loggregator system.

  • Service Metrics Agent:
    The Service Metrics Agent receives metrics from service instances on your Ops Manager deployment and makes them available on a Prometheus endpoint.

  • Metrics Discovery Registrar:
    The Metrics Discovery Registrar publishes the location of the Prometheus endpoint defined by the Metrics Agent and Service Metrics Agent to NATs. This is helpful for configuring automation to scrape metric data from the endpoint. For more information about automating metric scraping, see Telegraf Reference Architecture (Beta).

Log Components

This section describes the Log and Metric Agent Architecture components that allow you to access logs on your foundation.

These components are also a part of the Loggregator system. For more information about how these components function as part of the Loggregator system, see Loggregator Architecture.

The following components of the Log and Metric Agent Architecture enable access to logs:

  • Syslog Agent:

    Syslog Agents run on Ops Manager component VMs and host VMs to collect and forward logs to configured syslog drains. This includes syslog drains for individual apps as well as aggregate drains for all apps in your foundation. You can specify the destination for logs as part of the individual syslog drain or in the TAS for VMs tile.

  • Aggregate Syslog Drain:
    The aggregate syslog drain feature allows you to configure all Syslog Agents on your deployment to send logs to a single destination. You can use the aggregate syslog drain feature rather than the Loggregator Firehose to forward all logs for your deployment.

  • Log Cache:

    Log Cache allows you to view logs and metrics over a specified period of time. The Log Cache includes API endpoints and a CLI plugin to query and filter logs and metrics. To download the Log Cache CLI plugin, see Cloud Foundry Plugins. The Log Cache API endpoints are available by default. For more information about using the Log Cache API, see Log Cache on GitHub.

Reference Architectures

This section includes sample reference architectures for using the Log and Metric Agent Architecture with Telegraf or Prometheus to export logs and metrics from Ops Manager to an external source.

These reference architectures allow you to use external tools to better manage how the components of the Log and Metric Agent Architecture collect and forward logs and metrics.

Telegraf Reference Architecture (Beta)

You can deploy Telegraf with the Log and Metric Agent Architecture to use push-based metrics collection formats.

cf-telegraf-operator deploys Telegraf and a sidecar that manages the scrape configuration.

For more information, see cf-telegraf-operator on GitHub.