Configuring System Logging in Elastic Runtime

Page last updated:

This topic explains how to scale the Pivotal Cloud Foundry (PCF) Loggregator component VMs, and to forward logs to an external aggregator service.

Enable Syslog Forwarding

To forward log messages to an external Reliable Event Logging Protocol (RELP) server, complete the following steps:

  1. From the Elastic Runtime tile, Select System Logging. Sys logging
  2. If you want to include security events in your log stream, select the Enable Cloud Controller security event logging checkbox. This logs all API requests, including the endpoint, user, source IP, and request result, in the Common Event Format (CEF).
  3. Enter the IP address of your syslog server in External Syslog Aggregator Hostname and its port in External Syslog Aggregator Port. The default port for a syslog server is 514.

    Note: The host must be reachable from the Elastic Runtime network, accept TCP connections, and use the RELP protocol. Ensure your syslog server listens on external interfaces.

  4. Select an External Syslog Network Protocol to use when forwarding logs.
  5. For the Syslog Drain Buffer Size, enter the number of messages the Doppler server can hold from Metron agents before the server starts to drop them. See the Loggregator Guide for Cloud Foundry Operators topic for more details.
  6. Click Save.

Scale Loggregator

Apps and PCF system components constantly generate log and metrics data. The Loggregator system handles this data as follows. See Overview of the Loggregator System for more information.

  • The Metron agent running on each component or application VM collects and sends this data out to Doppler components.
  • Doppler components temporarily buffer the data before periodically forwarding it to the Traffic Controller. When the log and metrics data input to a Doppler exceeds its buffer size for a given interval, data can be lost.
  • The Traffic Controller serves the aggregated data stream through the Firehose WebSocket endpoint.

Follow the instructions below to scale the Loggregator system. For guidance on monitoring and capacity planning, see Monitoring Pivotal Cloud Foundry.

Increase buffer size

  1. From the Elastic Runtime tile, select System Logging.
  2. Increase the Drain Buffer Size to prevent loss of log data.
  3. Click Save.
  4. Click Apply Changes.

Add Component VM Instances

  1. From the Elastic Runtime tile, select Resource Config.
  2. Increase the number in the Instances column of the component you want to scale. You can add instances for the following Loggregator components:
    • Loggregator Traffic Controller

      Note: The Reverse Log Proxy (RLP) BOSH job is colocated with the Traffic Controller VM. If you want to scale Loggregator to handle more logs for syslog drains, you can add instances of the Traffic Controller.

    • Syslog Adapter
    • Doppler Server Loggregator vms
  3. Click Save.
  4. Click Apply Changes.
Create a pull request or raise an issue on the source for this page in GitHub