LATEST VERSION: 1.5.5 - CHANGELOG
Service Metrics v1.5.5

Deploying a Release with Service Metrics

Upload Required Releases

Upload the following releases to your BOSH director:

  • Your service release
  • The service-metrics release
  • Your service-name-metrics release

Write a BOSH Manifest

The service manifest must have a non-errand instance group that co-locates the following jobs:

  • The <YOUR-SERVICE-NAME> job from the service release
  • The service-metrics job from the service metrics release
  • The <YOUR-SERVICE-NAME>-metrics job from -metrics release
  • The metron_agent job from the Loggregator release
instance_groups:
- name: service-metrics
  instances: 1
  jobs:
  - name: <YOUR-SERVICE-NAME>
    release: service-release
  - name: service_metrics
    release: service-metrics
  - name: <YOUR-SERVICE-NAME>-metrics
    release: <YOUR-SERVICE-NAME>-metrics
  - name: metron_agent
    release: loggregator
  stemcell: trusty
  vm_type: medium
  networks:
  - name: service-metrics
  azs: [eu-west-1c]

Properties

The service metrics job can be configured with the following properties:

Field Type Description Required Default Value
origin string the name of the service, so it can reference metrics originating from that service in the logs yes n/a
metrics_command string the command to generate the metrics no see Metrics command
metrics_command_args array of strings any args provided to metrics_command no []
execution_interval_seconds int how often the metric generation command runs no 60
debug boolean turn verbose mode on/off no false
monit_dependencies array of strings jobs that must run before monit attempts to start the service metrics job. This is a way to define job dependencies, which are not supported by BOSH. no []

An example snippet is shown below:

properties:
  service_metrics: #the service metrics release template expects the property key to be service_metrics, even though the job is called service-metrics
    origin: <YOUR-SERVICE-NAME>
    metrics_command: /bin/echo # optional
    metrics_command_args: # optional
    - "-n"
    - '[{"key":"service-dummy","value":99,"unit":"metric"}]'
    execution_interval_seconds: 5 # optional
    debug: true # optional
    monit_dependencies: # optional
    - service-broker
  metron_agent:
    zone: z1
    deployment: <YOUR-SERVICE-NAME>
    dropsonde_incoming_port: 3457 # optional
    protocols:
    - udp  # default protocol
    - tls  # optional, to connect to doppler over TLS
    tls:  # optional, to connect to doppler over TLS
      client_cert: <REPLACE_WITH_METRON_AGENT_CERTIFICATE>
      client_key: <REPLACE_WITH_METRON_AGENT_PRIVATE_KEY>
  metron_endpoint:
    shared_secret: <REPLACE-WITH-SECRET>
  loggregator:
    etcd:
      machines:
        - 10.0.1.110
    tls: # optional, to connect to doppler over TLS
      ca_cert: <REPLACE_WITH_LOGGREGATOR_CA_CERTIFICATE>
  loggregator_endpoint:
    shared_secret: <REPLACE-WITH-SECRET>

Metrics command

The default metrics_command is /var/vcap/jobs/service-metrics-adapter/bin/collect-service-metrics.

We recommend creating a service-metrics-adapter job in your service release that templates this script. The collect-service-metrics script can encapsulate any commands and arguments required to capture metrics, instead of using the metrics_command and metrics_command_args properties in the manifest.

If the metrics_command fails, for example if the [MY-SERVICE]-metrics binary exits with a non-zero exit code, the service-metrics job will not start, or will exit with 0 if it was already running. In this case, the BOSH instance shows as failing and monit will try to restart the service-metrics job.

Notes

The service metrics release does not currently support the BOSH v2 manifest format, which allows job level properties.

The service metrics release supports enabling TLS between Metron and Doppler, which is documented on the loggregator release.

Logging

Service metrics logs to files in /var/vcap/sys/log/service-metrics, and also to syslog.

For forwarding syslog to a third party syslog drain (e.g. papertrail) we recommend co-locating the syslog-release.

Create a pull request or raise an issue on the source for this page in GitHub