Customizing Platform Log Forwarding

Page last updated:

You can configure Pivotal Application Service (PAS) to forward logs to remote endpoints using the Syslog protocol defined in RFC 5424. For more information, see Enable Syslog Forwarding in Configuring Logging in PAS.

PAS annotates forwarded messages with structured data. This structured data identifies the originating BOSH Director, deployment, instance group, availability zone, and instance ID. All logs forwarded from BOSH jobs have their PRI set to 14 (facility: user, severity: info), which may not reflect the originally intended PRI of the log.

Logs forwarded from other sources, such as kernel logs, retain their original PRI information.

Log lines use the format below:

    [instance@ENTERPRISE_NUMBER director="$DIRECTOR" deployment="$DEPLOYMENT" 

Example log messages:

<14>1 2017-01-25T13:25:03.18377Z etcd - - [instance@47450 
    director="test-env" deployment="cf" group="diego_database" az="us-west1-a" 
    id="83bd66e5-3fdf-44b7-bdd6-508deae7c786"] [INFO] the leader is 
<14>1 2017-01-25T13:25:03.184491Z bbs - - [instance@47450 
    director="test-env" deployment="cf" group="diego_database" az="us-west1-a" 

Change Which Logs Are Forwarded

When you enable log forwarding, all log lines written to disk in the /var/vcap/sys/log directories are forwarded from all Cloud Foundry job virtual machines (VMs).

You can specify a custom rule to modify which logs PAS forwards.

The custom rsyslog rules shown below are written in RainerScript. The custom rules are inserted before the rule that forwards logs. The stop command, stop, prevents logs from reaching the forwarding rule. This filters out these logs.

Logs filtered out before forwarding remain on the local disk, where the BOSH job originally wrote them, and may remain available for download from Ops Manager, or accessible through SSH.

Note: If your custom rule is invalid, PAS forwards no logs.

Forward Only Logs From a Certain Job

This rule filters logs out unless they originate from the uaa job:

if ($app-name != "uaa") then stop

Exclude Logs With Certain Content

This rule filters out logs that contain “DEBUG” in the body.

if ($msg contains "DEBUG") then stop

Note: In the above example, “DEBUG” is in the message body. Not all logs that are originally intended as to be DEBUG logs will have this string in their body.

Create a pull request or raise an issue on the source for this page in GitHub