Using Splunk Firehose Nozzle for PCF
This topic describes how to use Splunk Firehose Nozzle for Pivotal Cloud Foundry (PCF).
After installing and configuring Splunk Firehose Nozzle for PCF, PCF operators can navigate to the URL of their existing Splunk Enterprise deployment to immediately search, report, visualize, and alert on PCF Firehose data.
The following assumes basic familiarity on how to run searches, save reports, and create dashboards in Splunk Enterprise. If you are new to Splunk, start with the Search Tutorial which shows you how to search data and create simple dashboards.
The following is an example Splunk dashboard for operational intelligence.
The Firehose event types forwarded by the Splunk Nozzle are assigned the following Splunk sourcetypes by default:
|Firehose event type||Splunk sourcetype|
In addition, logs from the Nozzle are of sourcetype
You can use these Splunk sourcetypes to search and retrieve the relevant events from the index, use statistical commands to calculate metrics and generate reports, search for specific conditions within a rolling time window, identify patterns in your data, predict future trends, and so on. Here are some sample search commands that you can use:
Search for any errors from your deployment:
Search for router bad gateways:
Search for metrics of remaining cell disk capacity or memory:
You must have access to the index specified to receive these Firehose events. You might need to explicitly specify the index in your search if that index is not part of the indexes searched by default for your Splunk role. These searches can be saved as reports and used to power dashboard panels, as shown in the next section.
To monitor your environment operational health, you can build on these same searches to generate reports and visualizations. For example:
Report available memory per cell:
sourcetype=cf:valuemetric name=CapacityRemainingMemory | eval valueGB=round(case(unit=="MiB", value/1024, unit=="KiB", value/(1024*1024), unit=="GiB", value),2) | stats min(valueGB) as mem by job_instance | rename mem as "Available Memory (GB)", job_instance as "Job Instance"
Report available memory per cell over time:
sourcetype=cf:valuemetric name=CapacityRemainingMemory | eval valueGB=round(case(unit=="MiB", value/1024, unit=="KiB", value/(1024*1024), unit=="GiB", value),2) | timechart min(valueGB) by job_instance
Report number of routes registered with trend indicator:
sourcetype=cf:valuemetric name=RoutesTotal | timechart avg(value) as numRoutes
For more information about searching and reporting with Splunk, see Splunk Docs