Creating and Managing Sink Resources
Page last updated:
Warning: VMware Enterprise PKS v1.6 is no longer supported because it has reached the End of General Support (EOGS) phase as defined by the Support Lifecycle Policy. To stay up to date with the latest software and security updates, upgrade to a supported version.
This topic describes how to create and manage sink resources for a Kubernetes cluster provisioned with VMware Enterprise PKS (PKS), or for a namespace within a cluster.
Overview
Sinks collect logs and metrics about Kubernetes worker nodes in your PKS deployment and workloads that are running on them.
You can create two types of sinks:
- Log sinks
- Metric sinks
For more conceptual information about sinks, see Sink Architecture in Enterprise PKS.
Prerequisites
Before creating a sink resource:
- Review Sink Types in Sink Architecture in Enterprise PKS.
Configure sink resources in the Enterprise PKS tile > In-Cluster Monitoring:
- If you want to create a
ClusterLogSink
orLogSink
resource, select the Enable Log Sink Resources checkbox. - If you want to create a
ClusterMetricSink
orMetricSink
resource, select the Enable Metric Sink Resources checkbox. - If you want to use Node Exporter to send worker node metrics to
metric sinks of kind
ClusterMetricSink
as described in Create a ClusterMetricSink Resource for Node Exporter Metrics below, select the Enable node exporter on workers checkbox.
For more information about these configuration settings, see the PKS installation topic for your IaaS:
- If you want to create a
Install the Kubernetes CLI,
kubectl
. For installation instructions, see Installing the Kubernetes CLI.
Create Sinks
You can create log and metric sinks for clusters and namespaces.
- If you want to create a
ClusterLogSink
orLogSink
, see ClusterLogSink and LogSink Resources below. - If you want to create a
ClusterMetricSink
orMetricSink
, see ClusterMetricSink and MetricSink Resources below.
ClusterLogSink and LogSink Resources
To create ClusterLogSink
or LogSink
resources, you can:
- Create a Syslog ClusterLogSink or LogSink Resource
- Create a Webhook ClusterLogSink or LogSink Resource
- Create a ClusterLogSink or LogSink Resource with a Fluent Bit Output Plugin
Note: Log sinks created in PKS do not support UDP connections.
Note: PKS requires a secure connection
for log forwarding when using ClusterLogSink
and LogSink
resources of type syslog
or webhook
. To forward logs using an unsecured connection, see
Unsecured ClusterLogSink and LogSink Log Forwarding below.
Create a Syslog ClusterLogSink or LogSink Resource
ClusterLogSink
and LogSink
resources of type syslog
deliver logs using the TCP-based syslog protocol.
To define a syslog ClusterLogSink
or LogSink
resource, perform the following steps:
Create a YAML file that specifies your log destination in the following format:
apiVersion: pksapi.io/v1beta1 kind: YOUR-SINK-RESOURCE metadata: name: YOUR-SINK namespace: YOUR-NAMESPACE spec: type: syslog host: YOUR-LOG-DESTINATION port: YOUR-LOG-DESTINATION-PORT enable_tls: true
Where:
YOUR-SINK-RESOURCE
is the sink resource you want to create. This must be eitherClusterLogSink
orLogSink
. For information about these sink resources, see Overview.YOUR-SINK
is a name you choose for your sink.YOUR-NAMESPACE
is the name of your namespace. Omit this line if creatingClusterLogSink
.YOUR-LOG-DESTINATION
is the URL or IP address of your log management service.YOUR-LOG-DESTINATION-PORT
is the port number of your log management service.Note:
enable_tls
must betrue
.
Save the YAML file with an appropriate file name. For example,
my-cluster-log-sink.yml
.Apply the
ClusterLogSink
orLogSink
resource to your cluster by running the following command:kubectl apply -f YOUR-SINK.yml
Where
YOUR-SINK.yml
is the name of your YAML file. For example:$ kubectl apply -f my-cluster-log-sink.yml
Create a Webhook ClusterLogSink or LogSink Resource
ClusterLogSink
and LogSink
resources of type webhook
batch logs into one-second units,
wrap the resulting payload in JSON, and use the POST
method to deliver the logs to the address of your
log management service.
To define a webhook ClusterLogSink
or LogSink
resource, perform the following steps:
Create a YAML file that specifies your log destination in the following format:
apiVersion: pksapi.io/v1beta1 kind: YOUR-SINK-RESOURCE metadata: name: YOUR-SINK namespace: YOUR-NAMESPACE spec: type: webhook url: YOUR-LOG-DESTINATION
Where:
YOUR-SINK-RESOURCE
is the sink resource you want to create. This must be eitherClusterLogSink
orLogSink
. For information about these sink resources, see Overview.YOUR-SINK
is a name you choose for your sink.YOUR-NAMESPACE
is the name of your namespace. Omit this line if creatingClusterLogSink
.YOUR-LOG-DESTINATION
is the URL or IP address of your log management service.
Save the YAML file with an appropriate filename. For example,
my-cluster-log-sink.yml
.Apply the
ClusterLogSink
orLogSink
resource to your cluster by running the following command:kubectl apply -f YOUR-SINK.yml
Where
YOUR-SINK.yml
is the name of your YAML file. For example:$ kubectl apply -f my-cluster-log-sink.yml
Create a ClusterLogSink or LogSink Resource with a Fluent Bit Output Plugin
ClusterLogSink
and LogSink
resources with a Fluent Bit output plugin deliver logs to the output plugin
that you specify in your resource configuration.
To define a ClusterLogSink
or LogSink
resource with a Fluent Bit output plugin,
perform the following steps:
Create a YAML file that specifies your log destination in the following format:
apiVersion: pksapi.io/v1beta1 kind: YOUR-SINK-RESOURCE metadata: name: YOUR-SINK namespace: YOUR-NAMESPACE spec: type: http output_properties: Host: example.com Format: json Port: 443 tls: on tls.verify: off
Where:
YOUR-SINK-RESOURCE
is the sink resource you want to create. This must be eitherClusterLogSink
orLogSink
. For information about these sink resources, see Overview.YOUR-SINK
is a name you choose for your log sink.YOUR-NAMESPACE
is the name of your namespace. Omit this line if creatingClusterLogSink
.
Note: This is a sample plugin configuration for
http
. For a full list of supported plugins, see the Fluent Bit documentation.Save the YAML file with an appropriate filename. For example,
my-cluster-log-sink.yml
.Apply the
ClusterLogSink
orLogSink
resource to your cluster by running the following command:kubectl apply -f YOUR-SINK.yml
Where
YOUR-SINK.yml
is the name of your YAML file. For example:$ kubectl apply -f my-cluster-log-sink.yml
Unsecured ClusterLogSink and LogSink Log Forwarding
By default, PKS requires a secure connection for log forwarding when using ClusterLogSink
and LogSink
resources of type syslog
or webhook
.
For debugging purposes on a local machine, you may want to temporarily forward logs using an unsecured connection. To do this, you must:
Disable sink forwarding validation by running the following command:
kubectl delete validatingwebhookconfigurations validator.pksapi.io
Set
enable_tls
tofalse
in your log destination YAML file.
Warning: Disabling secure log forwarding is not recommended.
ClusterMetricSink and MetricSink Resources
By default, a ClusterMetricSink
resource collects metrics from a cluster using the Kubernetes Input Plugin and writes them to one or more outputs
that you specify in your ClusterMetricSink
configuration.
Alternatively, you can use Node Exporter as your input plugin.
To create a ClusterMetricSink
resource, see:
- Create a ClusterMetricSink or MetricSink Resource.
Follow these instructions if you want to use the default
ClusterMetricSink
configuration. - Create a ClusterMetricSink Resource for Node Exporter Metrics. Follow these instructions if you want to use Node Exporter as your input plugin. For a list of Node Exporter metrics, see the Node Exporter GitHub repository.
A MetricSink
resource collects metrics from a namespace within a cluster using
prometheus.io/scrape
annotations set to true
and writes them to one or more outputs
that you specify in your MetricSink
configuration.
To create a MetricSink
resource, follow the instructions in Create a ClusterMetricSink or MetricSink Resource.
For a list of supported output plugins, see Output Plugins in the telegraf GitHub repository.
Create a ClusterMetricSink or MetricSink Resource
To define a ClusterMetricSink
or MetricSink
resource, perform the following steps:
Create a YAML file in the following format:
apiVersion: pksapi.io/v1beta1 kind: YOUR-SINK-RESOURCE metadata: name: YOUR-SINK namespace: YOUR-NAMESPACE spec: inputs: outputs: - type: YOUR-OUTPUT-PLUGIN
Where:
YOUR-SINK-RESOURCE
is the sink resource you want to create. This must be eitherClusterMetricSink
orMetricSink
. For information about these sink resources, see Overview.YOUR-SINK
is a name you choose for your sink.YOUR-NAMESPACE
is the name of your namespace. Omit this line if creatingClusterMetricSink
.YOUR-OUTPUT-PLUGIN
is the name of the output plugin you want to use for your metrics.Note: You can leave the
inputs
field blank. ForClusterMetricSink
, this field is configured to include metrics from the kubelet by default. ForMetricSink
, the field includes allprometheus.io/scrape
annotations set totrue
by default.
For example:
apiVersion: pksapi.io/v1beta1 kind: ClusterMetricSink metadata: name: http spec: inputs: outputs: - type: http url: https:example.com method: POST data_format: json
Create a ClusterMetricSink Resource for Node Exporter Metrics
To define a ClusterMetricSink
resource for collecting Node Exporter metrics, perform the following steps:
- Enable Node Exporter on your cluster workers by selecting the Enable node exporter on workers checkbox in the Enterprise PKS tile > In-Cluster Monitoring.
Create a YAML file in the following format:
apiVersion: pksapi.io/v1beta1 kind: ClusterMetricSink metadata: name: YOUR-SINK spec: inputs: - monitor_kubernetes_pods: true type: prometheus outputs: - type: YOUR-OUTPUT-PLUGIN
Where:
YOUR-SINK
is a name you choose for your sink.YOUR-OUTPUT-PLUGIN
is the name of the output plugin you want to use for your metrics.
For example:
apiVersion: pksapi.io/v1beta1 kind: ClusterMetricSink metadata: name: http spec: inputs: - monitor_kubernetes_pods: true type: prometheus outputs: - type: http url: https:example.com method: POST data_format: json
Save the YAML file with an appropriate filename. For example,
my-cluster-metric-sink.yml
.Apply the
ClusterMetricSink
resource to your cluster by running the following command:kubectl apply -f YOUR-SINK.yml
Where
YOUR-SINK.yml
is the name of your YAML file. For example:$ kubectl apply -f my-cluster-metric-sink.yml
Filter Sinks
The LogSink
and ClusterLogSink
resources allow users to set filters to
include or exclude logs or events. For more information, see
Monitoring Clusters with Log Sinks.
To filter log sinks, add a filter properties section to the YAML file that specifies your log destination in the following format:
apiVersion: pksapi.io/v1beta1
kind: YOUR-SINK-RESOURCE
metadata:
name: YOUR-SINK
namespace: YOUR-NAMESPACE
spec:
type: syslog
host: YOUR-LOG-DESTINATION
port: YOUR-LOG-DESTINATION-PORT
enable_tls: true
filters:
include-events: true
include-logs: false
Where:
YOUR-SINK-RESOURCE
is the sink resource type that you created. This must be eitherClusterLogSink
orLogSink
.YOUR-SINK
is the name you chose for your sink.YOUR-NAMESPACE
is the name of your namespace. Omit this line forClusterLogSink
.YOUR-LOG-DESTINATION
is the URL or IP address of your log management service.YOUR-LOG-DESTINATION-PORT
is the port number of your log management service.
The default values for these filter properties is true. If you do not specify filter properties, both logs and events are included.
List Sinks
To list sinks for clusters and namespaces, use the commands in the following sections.
ClusterLogSink and LogSink Resources
To list cluster log sinks, run the following command:
kubectl get clusterlogsinks
To list namespace log sinks, run the following command:
kubectl -n YOUR-NAMESPACE get logsinks
Where YOUR-NAMESPACE
is the name of your namespace.
ClusterMetricSink and MetricSink Resources
To list cluster metric sinks, run the following command:
kubectl get clustermetricsinks
To list namespace metric sinks, run the following command:
kubectl -n YOUR-NAMESPACE get metricsinks
Where YOUR-NAMESPACE
is the name of your namespace.
Delete Sinks
To delete sinks for clusters and namespaces, use the commands in the following sections.
ClusterLogSink and LogSink Resources
To delete a cluster log sink, run the following command:
kubectl delete clusterlogsink YOUR-SINK
Where YOUR-SINK
is the name of your sink.
To delete a namespace log sink, run the following command:
kubectl -n YOUR-NAMESPACE delete logsink YOUR-SINK
Where:
YOUR-NAMESPACE
is the name of your namespace.YOUR-SINK
is the name of your log sink.
ClusterMetricSink and MetricSink Resources
To delete a cluster metric sink, use the following command:
kubectl delete clustermetricsink YOUR-SINK
Where YOUR-SINK
is the name of your sink.
To delete a namespace metric sink, use the following command:
kubectl -n YOUR-NAMESPACE delete metricsink YOUR-SINK
Where:
YOUR-NAMESPACE
is the name of your namespace.YOUR-SINK
is the name of your metric sink.
Please send any feedback you have to pks-feedback@pivotal.io.