Installing and Configuring Splunk Firehose Nozzle for PCF

This topic describes how to install and configure Splunk Firehose Nozzle for PCF.

Install and Configure Splunk Firehose Nozzle for PCF

Prerequisites

  • For information about the Ops Manager Installation Dashboard, see Using Ops Manager.
  • There is no upgrade path from the previous release of Splunk Nozzle for PCF (Beta). Install this version to an environment without the Splunk Firehose Nozzle for PCF (Beta).

Steps

  1. Download the Splunk file from Pivotal Network.

  2. Navigate to the Ops Manager Installation Dashboard and click Import a Product to upload the product file.

  3. Click Add next to the uploaded Splunk Firehose Nozzle for PCF tile in the Ops Manager Available Products view to add it to your staging area.

  4. Click the newly added Splunk Firehose Nozzle for PCF tile. Report Example: Ops Manager Dashboard View

  5. Configure the tile as described below.

  6. Click Save.

  7. Return to the Ops Manager Installation Dashboard and click Apply Changes to install Splunk Firehose Nozzle for PCF tile.

Configuration

  1. Click Assign AZs and Networks: Choose placement accordingly.

  2. Click Splunk Settings: Report Example: Tile Splunk Settings

    Complete the fields as follows:

    • HTTP Event Collector Endpoint (HEC) URL: HTTP Event Collector endpoint URL.<http(s)>://<address>:<port>, where <address> is either FQDN or IP address, and <port> is the port on which Splunk HEC is listening. In a clustered environment this can also be the address and port of a load balancer that distributes requests to multiple configured HEC endpoints. These endpoints can be across both Splunk Heavy Forwarders or Splunk Indexers.
    • HTTP Event Collector (HEC) Token: Splunk generated HEC token. More Information
    • Skip SSL validation: Skip SSL certificate validation for connections to Splunk. Secure communications will not check SSL certificates against a trusted Certificate Authority. Do not use for a production environment.
    • Index: The Splunk index that events are sent to.

    WARNING: Setting an invalid index causes events to be lost. This index must match one of the selected indexes for the Splunk HTTP event collector token used for the HTTP Event Collector Token parameter.

  3. Click Cloud Foundry Settings: Report Example: Cloud Foundry Settings View

    Complete the fields as follows:

    • API Endpoint: Cloud Foundry API endpoint.
    • API user/API password: Splunk Nozzle’s credentials. Best practice is providing a user with “Rotate, Repave, Repair” credentials, so that credentials can be easily deactivated, rotated, and have minimal access privileges. The Nozzle’s user needs scopes cloud_controller.admin_read_only and doppler.firehose (older releases without cloud_controller.admin_read_only have to use cloud_controller.admin).

      • For Elastic Runtime versions v1.8.9 and v1.7.29 and later, a splunk-firehose user is pre-created. Look up this user by:
        1. In the Elastic Runtime tile in Ops Manager, click the “Credentials” tab.
        2. In the UAA section, retrieve the “Splunk Firehose Credentials”
      • For earlier versions of Elastic Runtime, use the uaac tool to create the appropriate user. The uaac commands to add a user are as follows:
      1. uaac target https://uaa.[system domain url]
      2. uaac token client get admin -s [admin client credentials secret]
      3. uaac -t user add splunk-nozzle --password password123 --emails na
      4. uaac -t member add cloud_controller.admin_read_only splunk-nozzle
      5. uaac -t member add doppler.firehose splunk-nozzle
    • Skip SSL validation: Skip SSL certificate validation for connections to Cloud Foundry. Secure communications do not check SSL certificates against a trusted Certificate Authority. Do not use for a production environment.

    • Event Types: Choose which events the nozzle forwards to the Splunk indexer(s). For more information about the following events, see dropsonde-protocol

      • HttpStartStop
      • LogMessage
      • ValueMetric
      • CounterEvent
      • Error
      • ContainerMetric
  4. Click Advanced. Report Example: Cloud Foundry Settings View

    Complete the fields as follows:

    • Scale Out Nozzle: Scale out Splunk Nozzle. Splunk recommends running two or more nozzles for high availability.
    • Firehose Subscription ID: The Loggregator does a round-robin of events across connections having the same ID. An operator would only need to change this if some other connection is also using the default value.
    • Additional Fields: Arbitrary set of key:value pairs in the form key1:value1,key2:value2,key3:value3 that do not occur in event payload itself. These fields are indexed with every event payload, and are used as metadata for indexing and searching. One situation in which this might be useful is differentiating logs from multiple foundations going into the same Splunk Enterprise index.
    • Add App Information: When true, the nozzle calls back to the Cloud Foundry API and queries events that have an app GUID to add additional information, such as app name, space name, space GUID, organization name, organization GUID. This causes a one-time load increase on the API machines proportional the number of running apps, because each Nozzle populates a cache of app metadata.
    • Enable Event Tracing: Enables event trace logging. Splunk events now contain a UUID, Splunk Nozzle Event Counts, and a Subscription-ID for Splunk correlation searches.
  5. Resource Config: Select default options.

  6. Stemcell: Ensure the proper stemcell is available.

Create a pull request or raise an issue on the source for this page in GitHub