Sink Architecture in Enterprise PKS

Page last updated:

This topic describes how sinks are implemented in Enterprise Pivotal Container Service (Enterprise PKS) workloads.

Overview

Enterprise PKS provides the following types of sinks for collecting logs and metrics:

  • A log sink for monitoring your cluster and namespace logs.
  • A metric sink for monitoring your cluster and namespace metrics.

Enterprise PKS clusters include an observability manager that manages log sink and metric sink configurations within a cluster.

The following diagram details Enterprise PKS observability manager architecture:

Observability Manager architecture in PKS

If you want to use sinks, you must select Enable Log Sink Resources or Enable Metric Sink Resources in the Enterprise PKS tile. If you want to forward additional infrastructure metrics, you must also select Enable Node-Exporter on master. When you select these fields, Ops Manager requests that BOSH enable the observability manager with these configurations.

For more information about enabling log sinks and metrics sinks, see (Optional) Logging in the Installing topic for your IaaS.

Log Sink Architecture

The Enterprise PKS log sink aggregates workload logs and forwards them to a common log destination.

The following diagram details Enterprise PKS log sink architecture:

Log sink architecture in PKS

Logs are monitored and aggregated by a set of fluent-bit daemons running as a pod on a single node.

An event-controller collects Kubernetes API events and sends them to a second fluent-bit daemon pod for aggregation.

All aggregated log entries are marshaled to a common log destination.

Note: When sinks are added or removed, all of the fluent-bit pods are refreshed with new sink information.

Metric Sink Architecture

The Enterprise PKS metric sink aggregates workload metrics and forwards them to a common metrics destination. The metric sink can aggregate metrics from master and worker nodes.

Metric Sink Architecture on Worker Nodes

The following diagram details Enterprise PKS metric sink architecture on a worker node:

Metric Sink architecture in PKS

A metric sink for a worker node collects and writes metrics from a cluster to specified outputs using input and output plugins.

Workload metrics are monitored by a set of third-party plugins. The plugins forward the metrics to a Telegraph service pod.

A pair of kubelets monitor Kubernetes and forward Kubernetes metrics to a pair of Telegraph service pods.

If Node Exporter is enabled on the worker nodes from with the Enterprise PKS a Node Exporter deamonset will be included in all clusters. This exposes a wide variety of node metrics. For more information about these metrics, see the Node Exporter repository in GitHub.

To define the collected unstructured metrics, a metric-controller monitors Kubernetes for custom resource definitions and forwards those definitions to the Telegraph services.

The Telegraph services collect, process, and aggregate gathered metrics. All aggregated metrics are marshaled to an additional plugin for forwarding to a third-party application.

Note: When sinks are added or removed, all of the Telegraph pods are refreshed with new sink information.

Metric Sink Architecture on Master Nodes

The following diagram details Enterprise PKS metric sink architecture on a master node:

BOSH metrics sink architecture in PKS

A metric sink for a master node writes BOSH health metrics from a cluster to specified outputs using output plugins.

The etcd server exports metrics from a /metrics endpoint to the Telegraph services.

If Node Exporter is enabled on the master nodes the Enterprise PKS form within the tile, a Node Exporter job will run on the master VMs and expose a wide variety of node metics. For more information about these metrics, see the Node Exporter repository in GitHub.

Kubernetes forwards custom resource definitions to the Telegraph services.

The Telegraph services collect, process, and aggregate gathered metrics. All aggregated metrics are marshaled to an additional plugin for forwarding to a third-party application.

For more information about metrics sinks for master nodes, see Monitoring Master/etcd Node VMs.

Note: When sinks are added or removed, all of the Telegraph pods are refreshed with new sink information.

For more information about sinks in Enterprise PKS, see the following topics:


Please send any feedback you have to pks-feedback@pivotal.io.