Installing and Configuring File Integrity Monitoring

Page last updated:

This topic describes how to install Pivotal File Integrity Monitoring (FIM).

Note: When you install the FIM tile using Ops Manager, FIM does not monitor the files on your BOSH Director. To apply FIM to the BOSH Director VM, see Installing File Integrity Monitoring on BOSH Director.

Prerequisites

  • You must be a Pivotal Platform operator with admin rights. See Operators in the Pivotal Platform documentation.

  • Pivotal Operations Manager (Ops Manager). For compatible versions, see the Product Snapshot.

Install FIM

To install the FIM file on the Ops Manager Installation Dashboard:

Note: If you are upgrading from v1.4 or earlier, you must follow the instructions in Upgrading FIM.

  1. Download the product file from Pivotal Network.

  2. Navigate to the Ops Manager Installation Dashboard and click Import a Product to upload the product file.

  3. Under Import a Product, click + next to the version number of FIM. This adds the tile to your staging area.

  4. Click the newly added FIM tile.

Configure FIM for Linux

To configure FIM for Linux VMs:

  1. Select FIM Configuration for Ubuntu.

    FIM Configuration for Ubuntu section of the FIM configuration in Ops Manager. Click here to view a larger version of this image.

  2. Configure the following fields:

    Field Description
    Watchlist Create a list of file paths to monitor for file system events. Click Add to add file paths to the list, and click the trash can icon to remove file paths from the list.

    For more information, see Watchlist below.

    Note: This field corresponds to fim.dirs in FIM v1.4 and earlier.

    Ignore patterns Create a list of files that you want FIM to ignore. Events for files matching any of the provided regular expressions are not included in the logs. Click Add to add files to the list, and click the trash can icon to remove files from the list.

    The items that you add must use Go-flavored path regular expressions. To test whether a regular expression is valid, you can use Regex101.

    For more information, see Ignore Patterns below.

    Note: This field corresponds to fim.ignored_patterns in FIM v1.4 and earlier.

    Low severity tagging for frequently changed files Create a list of files to be marked as low severity. Click Add to add files to the list, and click the trash can icon to remove files from the list.

    The items that you add must use Go-flavored path regular expressions. To test whether a regular expression is valid, you can use Regex101.

    For more information, see Low Severity Events below.

    Note: This field corresponds to fim.low_severity_patterns in FIM v1.4 and earlier.

    Output log format Enter a template for log lines. This template must be compatible with the golang package text/template.

    For more information about the Output log format field, see Output Log Format below.

    Note: This field corresponds to fim.format in FIM v1.4 and earlier.

    Heartbeat interval (in seconds) Set the heartbeat interval as follows:
    • To enable the heartbeat interval, set the value to an integer greater than 0 . If you set a negative value, an error occurs.
    • To disable the heartbeat interval, set the value to 0.
    The default value is 600.

    Note: This field corresponds to fim.heartbeat_interval in FIM v1.4 and earlier.

    Max memory usage (in bytes) Set a limit in bytes for the maximum amount of memory, including file cache, that FIM can use per VM. The default value is 536870912 (512 MB).

    Note: This field corresponds to fim.memory_limit in FIM v1.4 and earlier.

    CPU limit (percentage) Set the percentage of CPU that the FIM process can use. Integers from 1 to 100 are valid. The limit is set per core. Setting this field to 100 permits the use of one full core. The default value is 10.

    Note: This field corresponds to fim.cpu_limit in FIM v1.4 and earlier.

    Enforce CPU limit Select the enforcement policy for the CPU limit (percentage):
    • Always: Ensures the CPU limit (percentage) is always enforced
    • When other processes are using CPU resources: Permits the CPU usage to exceed the limit set by CPU limit (percentage) if idle CPU cycles are available
    The default setting is When other processes are using CPU resources.

    Warning: If Enforce CPU limit is set Always, verify that the CPU limit (percentage) is set high enough for FIM to execute correctly. If the limit is too strict, FIM fails to start.

    Note: This field corresponds to fim.enforce_cpu_limit in FIM v1.4 and earlier.
    • Always is equivalent to fim.enforce_cpu_limit == true
    • When other processes are using CPU resources is equivalent to fim.enforce_cpu_limit == false
    Log file digest for write/create events Choose whether to enable computing digests for write/create events using the Enable or Disable radio buttons. If you enable digests, a field for A threshold of file size beyond which digests are not calculated (in bytes) appears after you select the option.

    For more information, see File Digests below.

    Note: This field corresponds to fim.digests in FIM v1.4 and earlier. Setting Log file digest for write/create events to Enable is equivalent to fim.digests == [sha256].

    A threshold of file size beyond which digests are not calculated (in bytes) Enter a positive value for the threshold for the maximum size of files for FIM to hash. This field only appears if you have selected Enable for Log file digest for write/create events. The default value is 10000000.

    Note: This field corresponds to fim.digest_threshold in FIM v1.4 and earlier.

    List of instance group names that will be excluded from deployment Enter a comma-separated list of instance groups that you do not want FIM deployed on.
  3. Click Save.

Configure FIM for Windows (Beta)

Warning: FIM for Windows is currently in beta. To disable installing FIM on Windows VMs, follow the steps in Disable Windows below.

To configure FIM for Windows VMs:

  1. Select FIM Configuration for Windows (Beta). FIM Configuration for Windows section of the FIM configuration in Ops Manager. Click here to view a larger version of this image.

  2. Configure the following fields:

    Field Description
    Watchlist Create a list of file paths to monitor for file system events. Click Add to add file paths to the list, and click the trash can icon to remove file paths from the list.

    For more information, see Watchlist below.

    Note: This field corresponds to fim.dirs in FIM v1.4 and earlier.

    Container Watchlist Create a list of file paths to monitor for file system events per container on the Windows Diego Cell. Click Add to add file paths to the list, and click the trash can icon to remove file paths from the list. For example, to monitor file system events for app files, enter C:\Users\vcap\app.

    If you do not enter a file path, FIM does not monitor any file system events for containers.

    Note: The file path for generated logs from container events is relative to the file system for the Diego Cell, rather than the container.

    For example, a container event for the container file path C:\Users\vcap\app\test.html appears as a file system event in C:\proc\PID\root\Users\vcap\app\test.html, where PID is the process ID of the container.

    Ignore patterns Create a list of files that you want FIM to ignore. Click Add to add files to the list, and click the trash can icon to remove files from the list.

    The items that you add must use Go-flavored path regular expressions. When defining Ignore patterns for Windows, you must replace all single back slashes with double back slashes. To test whether a regular expression is valid, you can use Regex101. Events for files matching any of the provided regular expressions are not included in the logs.

    For more information, see Ignore Patterns below.

    Note: To ignore events for files in containers, you must enter regular expressions that are relative to the file system for the Diego Cell, rather than the container. To do this, enter regular expressions that start with ^C:\\proc\\[^\\]+\\root.

    For example, to ignore all files in containers in the directory C:\Users\vcap\app, enter ^C:\\proc\\[^\\]+\\root\\Users\\vcap\\app\\.*$.

    Note: This field corresponds to fim.ignored_patterns in FIM v1.4 and earlier.

    Low severity tagging for frequently changed files Create a list of files to be marked as low severity. Click Add to add files to the list, and click the trash can icon to remove files from the list.

    The items that you add must use Go-flavored path regular expressions. When defining Low severity tagging for frequently changed files for Windows, you must replace all single back slashes with double back slashes. To test whether a regular expression is valid, you can use Regex101.

    For more information, see Low Severity Events below.

    Note: This field corresponds to fim.low_severity_patterns in FIM v1.4 and earlier.

    Output log format Enter a template for log lines. This template must be compatible with the golang package text/template.

    For more information, see Output Log Format below.

    Note: This field corresponds to fim.format in FIM v1.4 and earlier.

    Heartbeat interval (in seconds) Set the heartbeat interval as follows:
    • To enable the heartbeat interval, set the value to an integer greater than 0 . If you set a negative value, an error occurs.
    • To disable the heartbeat interval, set the value to 0.
    The default value is 600.

    Note: This field corresponds to fim.heartbeat_interval in FIM v1.4 and earlier.

    Log file digest for write/create events Choose whether to enable computing digests for write/create events using the Enable or Disable radio buttons. If you enable digests, a field for A threshold of file size beyond which digests are not calculated (in bytes) appears after you select the option.

    For more information, see File Digests below.

    Note: This field corresponds to fim.digests in FIM v1.4 and earlier. Setting Log file digest for write/create events to Enable is equivalent to fim.digests == [sha256].

    A threshold of file size beyond which digests are not calculated (in bytes) Enter a positive value for the threshold for the maximum size of files for FIM to hash. This field only appears if you have selected Enable for Log file digest for write/create events. The default value is 10000000.

    Note: This field corresponds to fim.digest_threshold in FIM v1.4 and earlier.

    List of instance group names that will be excluded from deployment Enter a comma-separated list of instance groups that you do not want FIM deployed on.
  3. Click Save.

Disable Windows

To disable installing FIM on Windows VMs:

  1. In the FIM tile, select FIM Configuration for Windows (Beta).

  2. Add the instance group windows_diego_cell to the field List of instance group names that will be excluded from deployment.

  3. Click Save.

Monitor Containers with FIM

You can use FIM to monitor:

  • Garden containers on the Diego Cell VMs in Pivotal Application Service (PAS)
  • Containers on the Diego Windows Cell VMs in Pivotal Application Service for Windows (PASW)
  • Containers on the Kubernetes worker node VMs in Enterprise Pivotal Container Service (PKS)

For an example log message, see Examples of Log Messages from Containers.

Monitor Garden Containers

To configure FIM to monitor Garden containers:

  1. In the FIM tile, select FIM Configuration for Ubuntu.

  2. Add the Garden container directories to the Watchlist section:

    • /var/vcap/data/grootfs/store/unprivileged/images/
    • /var/vcap/data/grootfs/store/privileged/images/

    For more information about GrootFS volumes, see Volumes.

  3. Add the following pattern to the Ignore patterns section:

    • ^/var/vcap/data/grootfs/store/(un)?privileged/images/[\w-]+/rootfs/.*$

    Note: When files in the Garden containers are modified, changes are made to both the diff and rootfs directories. Adding this ignore pattern means that FIM ignores files and directories in the /var/vcap/data/grootfs/store/unprivileged/images/UUID/diff directory, where UUID is the ID of the container.

  4. Click Save.

Monitor Windows Garden Containers

To configure FIM to monitor Windows Garden containers:

  1. In the FIM tile, select FIM Configuration for Windows (Beta).

  2. Add at least one directory to the Container Watchlist section. Pivotal recommends that you add C:\Users\vcap\app, which is the directory for app files.

  3. Click Save.

Monitor Containers in PKS

To configure FIM to monitor containers on the Kubernetes worker node VMs in PKS:

  1. In the FIM tile, select FIM Configuration for Ubuntu.

  2. Add the container directory /var/vcap/store/docker/docker/ to the Watchlist section.

    Note: FIM writes log messages when files and directories in the /var/vcap/store/docker/docker/overlay2/UUID/diff directory are created, removed, or modified. UUID is the ID of the container.

  3. Click Save.

Configure Forwarding for FIM Alerts

FIM writes all alerts to the BOSH logs for the VMs in your deployment.

  • In Linux, these logs are located in /var/vcap/sys/log/fim/fim.stdout.log.
  • In Windows, these logs are located in C:\var\vcap\sys\log\fim-windows\filesnitch\job-service-wrapper.out.log.

You can use syslog forwarding to forward the alerts to a syslog aggregator.

  • If you are using the Pivotal Application Service (PAS) tile: The syslog aggregator that you specify receives all alerts generated on PAS, including the FIM alerts. To configure system logging, follow the procedure in Configuring Logging in PAS.

  • If you are using the syslog BOSH release: You can use the syslog BOSH release to forward system logs. For more information, see syslog-release in GitHub.

Note: When you configure syslog forwarding, ensure there is enough disk space for the logs, and that they rotate frequently. If you are not sure how often to rotate the logs, configure the rotation to occur either hourly, or when they reach a certain configured size. VMware recommends forwarding logs to a remote syslog aggregation system.

Apply Changes from Your Configuration

Your installation is not complete until you apply your configuration changes:

  1. Navigate to the Installation Dashboard in Ops Manager.

  2. Click Review Pending Changes.

  3. Click Apply Changes to complete the FIM installation.

Verify the Installation

To verify the installation for Linux:

  1. bosh ssh into the VMs in your deployment. For more information, see BOSH SSH.

  2. Enter this command:

    touch /bin/hackertool
    
  3. Enter this command:

    grep hackertool /var/vcap/sys/log/fim/fim.stdout.log
    
  4. Verify in the logs that a new file has been created. For example:

    CEF:0|cloud_foundry|fim|1.0.0|1|file integrity monitoring event|5| fname="/bin/hackertool" hostname="fim_1/3ad6ff1f-37e0-4b8a-80bd-d16b7f79c149" opname="CREATE" optype=1 ts=1574098829 severity=5
    


To verify the installation for Windows:

  1. bosh ssh into the VMs in your deployment. For more information, see BOSH SSH.

  2. Enter this command:

    powershell New-Item -type File /var/vcap/data/jobs/sample_file
    
  3. Enter this command:

    powershell "Get-Content C:\var\vcap\sys\log\fim-windows\filesnitch\job-service-wrapper.out.log | Select-String sample_file"
    
  4. Verify in the logs that a new file has been created. For example:

    CEF:0|cloud_foundry|fim|1.0.0|1|file integrity monitoring event|5| fname="C:\var\vcap\data\jobs\sample_file" hostname="no-job_1/ebee34c1-3300-4f5d-9557-bbef845d608c" opname="CREATE" optype=1 ts=1569953512 severity=5
    

Reference the following sections when configuring FIM.

Watchlist

FIM monitors a set of critical system directories. You can configure the directories that FIM monitors by adding and removing items in the Watchlist section.

Watchlist for Linux

Below is the default list of file paths in Watchlist section of FIM Configuration for Ubuntu.

Component File Paths
System binaries and configuration /boot/grub
/root
/bin
/etc
/lib
/lib64
/opt
/sbin
/srv
/usr
/var/lib
BOSH agent /var/vcap/bosh
/var/vcap/monit/job
BOSH releases /var/vcap/data/packages
/var/vcap/data/jobs

Watchlist for Windows

Below is the default list of file paths in Watchlist section of FIM Configuration for Windows.

  • C:\Windows\System32
  • C:\Program Files
  • C:\Program Files (x86)
  • C:\var\vcap\bosh
  • C:\var\vcap\data\packages
  • C:\var\vcap\data\jobs

Ignore Patterns

Some monitored directories might contain files that you do not want FIM to monitor, such as files that change frequently. You can configure FIM to ignore these events by adding and removing items in the Ignore patterns section. Use path regular expressions.

Ignore Patterns for Linux

Below is the default list in Ignore Patterns section of FIM Configuration for Ubuntu.

Scenario List
Temporary files created when an operator or errand runs bosh ssh ^/etc/passwd.+$
^/etc/shadow.+$
^/etc/subgid.+$
^/etc/subuid.+$
^/etc/group.+$
^/etc/gshadow.+$
Temporary files created when hosts are updated ^/etc/hosts.+$
BOSH agent logs ^/var/vcap/bosh/log/.+$
Log rotation ^/var/lib/logrotate/status.*$
Monit state ^/root/\.monit\.state$

Ignore Patterns for Windows

Note: There is currently no default value for Ignored patterns for Windows.

When defining Ignore patterns for Windows, you must replace all single back slashes with double back slashes. For example, to ignore all files in the directory C:\var\vcap\bosh\ignore_me\, use:

^C:\\var\\vcap\\bosh\\ignore_me\\.*$

Low Severity Events

Some monitored directories might contain files that only change occasionally or files that update frequently but are low impact. You can configure FIM to log events at a lower severity by adding and removing items in the Low severity tagging for frequently changed files section. Use path regular expressions.

Severity can be one of the following severity levels:

  • 0: Used for heartbeats.
  • 3: Used for low severity events. These events are for files that match any of the provided regular expressions. This can be useful to filter out business-as-usual events.
  • 5: Used for all other events. This is the default severity.

Low Severity Tagging for Frequently Changed Files for Linux

Below is the default list in Low severity tagging for frequently changed files section of FIM Configuration for Ubuntu.

Scenario List
When an operator or errand runs the bosh ssh a new user is created ^/etc/passwd$
^/etc/shadow$
^/etc/subgid$
^/etc/subuid$
^/etc/group$
^/etc/gshadow$
BOSH-DNS sync and new VM creation update hosts ^/etc/hosts$
Attached devices and cgroups ^/etc/mtab$
DHCP leases ^/var/lib/dhcp/dhclient.eth\d+.leases$
BOSH agent configuration changes when VM created/modified ^/var/vcap/bosh/settings.json$
BOSH agent CHMODs jobs and packages as part of bosh deployment ^/var/vcap/data/jobs$
^/var/vcap/data/packages$

Low Severity Tagging for Frequently Changed Files for Windows

Note: There is currently no default value for Low severity tagging for frequently changed files for Windows.

When defining Low severity tagging for frequently changed files for Windows, you must replace all single back slashes with double back slashes. For example, to mark all files in the directory C:\var\vcap\bosh\ignore_me\ as low severity, use:

^C:\\var\\vcap\\bosh\\ignore_me\\.*$

Output Log Format

By default, FIM generates messages in the Common Event Format. You can configure the output format as a Go text template using the Output log format field. For more information and examples of FIM log messages, see Log Messages.

Default Format

The default value of Output log format is:

"CEF:0|cloud_foundry|fim|1.0.0|{{.Optype}}|file integrity monitoring event|{{.Severity}}| {{.KeyValues}}"

Example output using the default Output log format configuration:

CEF:0|cloud_foundry|fim|1.0.0|0|file integrity monitoring event|0| fname="" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="FILESNITCH CHECKIN" optype=0 ts=1492715822 severity=0
CEF:0|cloud_foundry|fim|1.0.0|1|file integrity monitoring event|5| fname="/etc/passwd.lock" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="CREATE" optype=1 ts=1492715822 severity=5
CEF:0|cloud_foundry|fim|1.0.0|4|file integrity monitoring event|5| fname="/etc/passwd.17721" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="REMOVE" optype=4 ts=1492715822 severity=5
CEF:0|cloud_foundry|fim|1.0.0|1|file integrity monitoring event|5| fname="/etc/group.lock" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="CREATE" optype=1 ts=1492715822 severity=5
CEF:0|cloud_foundry|fim|1.0.0|4|file integrity monitoring event|5| fname="/etc/group.17721" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="REMOVE" optype=4 ts=1492715822 severity=5
CEF:0|cloud_foundry|fim|1.0.0|1|file integrity monitoring event|5| fname="/etc/gshadow.lock" hostname="diego_cell/8279dfa8-9f86-4bb1-8b92-65457d2ae989" opname="CREATE" optype=1 ts=1492715822 severity=5

Note: The FILESNITCH CHECKIN message is a logging marker that indicates filesnitch is operational in the absence of any file system events.

Custom Format

You can use individual fields to configure the log format. Each individual field, is a named property provided by FIM that will be replaced during the logging action.

For example:

"{{.Fname}} {{.Hostname}} {{.OpName}} {{.OpType}} {{.Digests}} {{.Ts}}"

Example output using the above configuration:

/bin/binary plymouth CREATE 1 sha256=da39a3ee5e6b4b0d3255bfef95601890afd80709 1475195574

The table below lists the template values you can use:

Template Description
{{.Fname}} The name of the affected file.
{{.Hostname}} The hostname of the VM on which the file event originated.
{{.OpName}} The type of file operation in textual format. For more information about opname, see Opname and Optype below.
{{.OpType}} The type of file operation in numeric format. For more information about optype, see Opname and Optype below.
{{.Severity}} The level of importance attributed to the event. For the severity levels, see Low Severity Events above.
{{.Ts}} The point in time at which FIM received the file event in Unix epoch format.
{{.Digests}} Key-value pairs of hash algorithms and the hash of the modified file. For more information, see File Digests below.
{{.Json}} This string serializes an event into a standard JSON dictionary. The string is in the following format:
{"fname":"ABSOLUTE-PATH","hostname":"BOSH-VM","opname":"OPERATION-NAME","optype":OPERATION-TYPE,"ts":TIMESTAMP}
For example:
{"fname":"/bin/binary","hostname":"plymouth","opname":"CREATE","optype":1,"ts":1475195084}
{{.KeyValues}} This string serializes an event into a series of key-value pairs. The string is in the following format:
fname="ABSOLUTE-PATH" hostname="BOSH-VM" opname="OPERATION-NAME" optype=OPERATION-TYPE ts=TIMESTAMP
For example:
fname="/bin/binary" hostname="plymouth" opname="CREATE" optype=1 ts=1475195258

Opname and Optype

Opname and optype are the type of file operation in textual and numeric format, respectively. For the possible values of the two fields see the table below:

opname optype Example Linux Trigger Example Windows Trigger
FILESNITCH CHECKIN 0 This is a heartbeat message written to the log. This occurs during every Heartbeat interval.

The default interval is 600 seconds. To configure this property, see Configure FIM for Linux.
This is a heartbeat message written to the log. This occurs during every Heartbeat interval.

The default interval is 600 seconds. To configure this property, see Configure FIM for Windows (Beta).
CREATE 1 touch newfile.txt

echo 'content' > newfile2.txt
Powershell New-Item -type File newfile.txt

Powershell Add-Content -Path newfile.txt -Value 'content'
WRITE 2 echo 'hello world' >> file.txt Powershell Add-Content -Path newfile.txt -Value 'content'
REMOVE 4 rm file.txt Powershell rm file.txt
RENAME 8 mv file.txt file.txt.orig Powershell mv file.txt file.txt.orig
CHMOD 16 chmod 0400 file.txt Powershell icacls file.txt /grant administrators:F

Note: FIM on Windows reports WRITE and CHMOD together as WRITE|CHMOD. The two operations are indistinguishable.

File Digests

FIM supports hashing monitored files on WRITE or CREATE events using the sha256 algorithm. If you enable digests, FIM includes the calculated hash for the file in the logs.

If you want to show that content has changed or check which version of the file is mapped to a log entry, you can calculate the sha256 value of a file and compare it to the value in the log.

Hashing is disabled by default.

FIM sets a threshold on the size of files, in bytes, to be hashed.