Configuring the FIM Add-on for PCF

Page last updated:

This topic describes how to configure and use the File Integrity Monitoring Add-on for PCF (FIM Add-on).

Listing Directories to be Monitored

The FIM Add-on monitors a set of critical system directories. The list of directories to be monitored by the FIM Add-on can be configured using the fim.dirs property. The default value of fim.dirs is:

fim:
  dirs:
    # System binaries and configuration
    - /bin
    - /etc
    - /lib
    - /lib64
    - /opt
    - /sbin
    - /srv
    - /usr
    - /var/lib

    # Bosh agent
    - /var/vcap/bosh
    - /var/vcap/monit/job

    # Bosh releases
    - /var/vcap/data/packages
    - /var/vcap/data/jobs

Note: The default fim.dirs are written for Ubuntu Trusty stemcells. On Xenial stemcells, add /lib32 to fim.dirs.

Event Logging

The FIM Add-on logs events that occur. These samples can be used to configure a Security Information and Event Management (SIEM) system to verify regular activity and generate alerts for unauthorized changes to a file.

These logs appear in the following format:

2019-01-02T15:54:01.350053+00:00 localhost filesnitch[5963]: CEF:0|cloud_foundry|fim|1.0.0|OPERATION-TYPE|file integrity
monitoring event|SEVERITY| fname="ABSOLUTE-PATH" hostname="BOSH-VM" opname="OPERATION-NAME" optype=OPERATION-TYPE ts=TIMESTAMP
severity=SEVERITY

Where:

  • OPERATION-TYPE and OPERATION-NAME are one of the following:

    • 1, which is CREATE
    • 2, which is WRITE
    • 4, which is REMOVE
    • 8, which is RENAME
    • 16, which is CHMOD
  • SEVERITY is one of the following severity levels:

    • 1, which is used for heartbeats
    • 3, which is used for low severity events
    • 5, which is used for all other events
  • ABSOLUTE-PATH is the absolute path to the file, for example /directory/file

  • BOSH-VM is the BOSH VM name, for example /directory/vm-name

  • TIMESTAMP is the timestamp in Unix epoch format.

For example:

 2019-01-02T15:54:01.350053+00:00 localhost filesnitch[5963]: CEF:0|cloud_foundry|fim|1.0.0|1|file integrity monitoring event|3|
fname="/test/file01" hostname="fim/a6fbc720-cfd1-47fc-a257-c211704924a6"
opname="CREATE" optype=1 ts=1546535400 severity=3

2019-01-02T15:54:19.388257+00:00 localhost filesnitch[5963]:
CEF:0|cloud_foundry|fim|1.0.0|2|file integrity monitoring event|5| fname="/test/file02" hostname="fim/a6fbc720-cfd1-47fc-a257-c211704924a6"
opname="WRITE" optype=2 ts=1546535520 severity=5

For an example of every type of FIM log message that can be configured, see Examples of FIM Log Messages.

Note: Low severity events can be used to filter out business-as-usual events. This makes unexpected events easy to isolate by looking for severity 5.

Ignoring Events

Some monitored directories may contain files that should not be monitored, for example, files that change frequently. You can configure the Fim Add-on to ignore events using path regexes with fim.ignored_patterns. The default value of fim.ignored_patterns is:

fim:
  ignored_patterns:
    # Temporary files created when an operator or errand invoke `bosh ssh`
    - ^/etc/passwd.+$
    - ^/etc/shadow.+$
    - ^/etc/subgid.+$
    - ^/etc/subuid.+$
    - ^/etc/group.+$
    - ^/etc/gshadow.+$

    # Temporary files created when hosts updated
    - ^/etc/hosts.+$

    # Bosh Agent logs
    - ^/var/vcap/bosh/log/.+$

    # Log rotation
    - ^/var/lib/logrotate/status.*$

Low Severity Events

Some monitored directories may contain files that occasionally change. You can configure The FIM Add-on to log events at a lower severity using path regexes with fim.low_severity_patterns. The default value of fim.low_severity_patterns is:

fim:
  low_severity_patterns:
    # When an operator or errand invoke `bosh ssh` a new user is created
    - ^/etc/passwd$
    - ^/etc/shadow$
    - ^/etc/subgid$
    - ^/etc/subuid$
    - ^/etc/group$
    - ^/etc/gshadow$

    # Bosh-DNS sync and new VM creation update hosts
    - ^/etc/hosts$

    # Attached devices and cgroups
    - ^/etc/mtab$

    # DHCP leases
    - ^/var/lib/dhcp/dhclient.eth\d+.leases$

    # Bosh Agent configuration changes when VM created/modified
    - ^/var/vcap/bosh/settings.json$

    # Bosh Agent CHMODs jobs and packages as part of `bosh deployment`
    - ^/var/vcap/data/jobs$
    - ^/var/vcap/data/packages$

Configure the Output Destination

FIM supports three types of output:

  • stdout: sends messages to /var/vcap/sys/log/fim/fim.stdout.log.
  • stderr: sends messages to /var/vcap/sys/log/fim/fim.stderr.log.
  • syslog: sends messages to /var/log/messages.

The output is configured using the fim.outputs property.

Note: Currently, FIM only supports selecting one output at a time.

The default value of the fim.outputs property:

fim:
  outputs:
    - syslog

Configure the Output Format

By default, the FIM Add-on generates messages in the Common Event Format. Output format can be configured as a Golang text template using the fim.format property. The default value of fim.format is:

fim:
  format: "CEF:0|cloud_foundry|fim|1.0.0|{{.Optype}}|file integrity monitoring event|{{.Severity}}| {{.KeyValues}}"

This example output shows what log lines look like with the default fim.format configuration:

Apr 20 19:12:37 localhost filesnitch[819]: CEF:0|cloud_foundry|fim|1.0.0|0|file integrity monitoring event|3| fname="" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="FILESNITCH_CHECKIN" optype=0 ts=1492715822
Apr 20 19:17:02 localhost filesnitch[819]: CEF:0|cloud_foundry|fim|1.0.0|1|file integrity monitoring event|5| fname="/etc/passwd.lock" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="CREATE" optype=1 ts=1492715822
Apr 20 19:17:02 localhost filesnitch[819]: CEF:0|cloud_foundry|fim|1.0.0|4|file integrity monitoring event|5| fname="/etc/passwd.17721" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="REMOVE" optype=4 ts=1492715822
Apr 20 19:17:02 localhost filesnitch[819]: CEF:0|cloud_foundry|fim|1.0.0|1|file integrity monitoring event|5| fname="/etc/group.lock" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="CREATE" optype=1 ts=1492715822
Apr 20 19:17:02 localhost filesnitch[819]: CEF:0|cloud_foundry|fim|1.0.0|4|file integrity monitoring event|5| fname="/etc/group.17721" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="REMOVE" optype=4 ts=1492715822
Apr 20 19:17:02 localhost filesnitch[819]: CEF:0|cloud_foundry|fim|1.0.0|1|file integrity monitoring event|5| fname="/etc/gshadow.lock" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="CREATE" optype=1 ts=1492715822

Note: The FILESNITCH CHECKIN message is a logging marker that indicates filesnitch is operational in the absence of any filesystem events.

The key-values pairs in the final field of a CEF file log carry the following meaning:

  • fname is the name of the affected file.
  • hostname is the hostname of the VM on which the file event originated.
  • ts is the point in time at which FIM received the file event.
  • optype and opname are the type of file operation in the numeric and textual format, respectively. The possible values of the two fields are described by the table below.

    opname optype Example Trigger
    CREATE 1 touch newfile.txt, echo 'content' > newfile2.txt
    WRITE 2 echo 'hello world' >> file.txt
    REMOVE 4 rm file.txt
    RENAME 8 mv file.txt file.txt.orig
    CHMOD 16 chmod 0400 file.txt, touch existingfile.txt

Other template values are listed below.

JSON

The {{.Json}} string serializes an event into a standard JSON dictionary. Example:

{"fname":"/bin/binary","hostname":"plymouth","opname":"CREATE","optype":1,"ts":1475195084}

Key-Values

The {{.KeyValues}} string serializes an event into a series of key=value fields. Example:

fname="/bin/binary" hostname="plymouth" opname="CREATE" optype=1 ts=1475195258

Individual Fields

template description
{{.Fname}} Name of the affected file
{{.Hostname}} Hostname of the VM on which the file event originated
{{.OpName}} Type of file operation in textual format
{{.OpType}} Type of file operation in numeric format
{{.Severity}} Level of importance attributed to the event
{{.Ts}} Point in time at which FIM received the file event
{{.Digests}} Key-value pairs of hash algorithms and the hash of the modified file

For example, the following variable definition…

fim:
  format: "{{.Fname}} {{.Hostname}} {{.OpName}} {{.OpType}} {{.Digests}} {{.Ts}}"

…produces the following:

/bin/binary plymouth CREATE 1 sha256=da39a3ee5e6b4b0d3255bfef95601890afd80709 1475195574

Calculate File Hashes

The FIM Add-on supports hashing monitored files on WRITE or CREATE events using the sha256 algorithm. Hashing is disabled by default, and can be configured using the fim.digests property, as follows:

fim:
  digests:
    - sha256

File Size Threshold

The FIM Add-on sets a threshold on the size of files to be hashed. Use the fim.digest_threshold property to adjust this threshold. The property takes a value in bytes. The default value is 10000000.

fim:
  digest_threshold: 10000000

Monitor Containers with FIM

You can use FIM to monitor the following:

  • Garden containers on the Diego Cell VMs in Pivotal Application Service (PAS)

  • Containers on the Kubernetes worker node VMs in Enterprise Pivotal Container Service (PKS)

Monitor Garden Containers

To configure FIM to monitor Garden containers, do the following when configuring FIM:

  1. In List Directories to Be Monitored above, add the following directories to the fim.dirs property:

    fim:
      dirs:
        ...
    
        # Garden containers
        - /var/vcap/data/grootfs/store/unprivileged/images/
        - /var/vcap/data/grootfs/store/privileged/images/
    

    For more information about GrootFS volumes, see Volumes.

  2. In Ignore Events, add the following pattern to the ignored_patterns property:

    fim:
      ignored_patterns:
        ...
        # When monitoring Garden containers,
        # ignore duplicate log messages sent to .../UUID/rootfs/
        - ^/var/vcap/data/grootfs/store/(un)?privileged/images/[\w-]+/rootfs/.*$
    

    When files in the Garden containers are modified, changes are made to both the diff and rootfs directories.

    Adding this ignore pattern, means that files and directories in the /var/vcap/data/grootfs/store/unprivileged/images/UUID/rootfs and /var/vcap/data/grootfs/store/privileged/images/UUID/rootfs directories are ignored by FIM. UUID is the ID of the container.

For an example log message, see Examples of Log Messages from Containers.

Monitor Containers in Enterprise PKS

To configure FIM to monitor containers on the Kubernetes worker node VMs in Enterprise PKS, do the following when configuring FIM:

  1. In List Directories to Be Monitored above, add the following directory to the fim.dirs property:

    fim:
      dirs:
        ...
    
        # Containers in Enterprise PKS
        - /var/vcap/store/docker/docker/
    

When files and directories in the /var/vcap/store/docker/docker/overlay2/UUID/diff directory are created, removed, or modified, FIM writes log messages. UUID is the ID of the container.

For an example log message, see Examples of Log Messages from Containers.