Creating New Variables in CredHub

Page last updated:

This topic explains how CredHub manages variables in the context of a larger deployment, and how to create new variables for use in CredHub.

Background

When a tile author defines a top-level variables section in the product template, Ops Manager passes the variables section to the product manifest. tile authors can define variables in the product template as follows:

variables:
  - name: EXAMPLE-CREDHUB-PASSWORD
    type: password

You can reference these variables in the manifest snippets in their tile metadata using a triple parentheses syntax: 

((( EXAMPLE-CREDHUB-PASSWORD )))

Using triple parentheses lets Ops Manager identify CredHub variables while still supporting the BOSH double parentheses syntax. A variable referenced within triple parentheses is replaced by double parentheses in the generated manifest. After contacting CredHub, BOSH populates that variable value internally.

The benefit of this approach is that the Ops Manager YAML file does not contain sensitive credentials when the metadata manifest snippets have triple parentheses. The resulting manifest file contains variables within double parentheses, rather than unobscured credentials.

For example, a tile author adds credentials to a manifest snippet in the following format:

  key: ((( EXAMPLE-CREDHUB-PASSWORD )))
  key: prefix-((( ANOTHER-CREDHUB-PASSWORD )))-suffix

Ops Manager evaluates the above example to generate the following section in the product manifest:

  (( EXAMPLE-CREDHUB-PASSWORD ))
  prefix-(( ANOTHER-CREDHUB-PASSWORD ))-suffix

How CredHub Works Within a Deployment

CredHub is distributed as a BOSH release. As part of this installation, Ops Manager co-locates the CredHub release on the BOSH Director, including the CredHub job configurations, and the Director is configured to point to the CredHub API.

Once CredHub has been deployed and configured on the Director, any Director deployment can use CredHub variables in place of credential values. Using variables, rather than values, provides an extra layer of security when transmitting credentials within your deployment.

Changing Your Deployment Manifest to Include CredHub Variables

The BOSH Director interpolates credential values into manifests that use the ((variables)) syntax. When the Director encounters a variable using this syntax, it requests the credential value from CredHub. If the credential does not exist and the release or manifest contains generation properties, the credential value is generated automatically.

The manifest excerpt below includes references to two credentials, EXAMPLE-PASSWORD and EXAMPLE-TLS.

When this manifest is deployed, the BOSH Director retrieves the stored variables and replaces them with the credential values associated with each variable. The EXAMPLE-TLS variables include property accessors, so only the certificate and private_key components are interpolated.

name: demo-deploy

instance_groups:
  jobs: 
  - name: demo 
    release: demo
    properties:
      demo:
        password: ((EXAMPLE-PASSWORD))
        tls: 
          certificate: ((EXAMPLE-TLS.certificate))
          private_key: ((EXAMPLE-TLS.private_key))

Ops Manager configures the Director to generate a credential if it does not exist. The manifest includes generation parameters that define how the credential should be generated. These generation parameters are defined in the variables section as shown below.

---
name: demo deploy 

variables: 
- name: EXAMPLE-PASSWORD
  type: password
- name: EXAMPLE-CA
  type: certificate
  options: 
    is_ca: true
    common_name: 'Example Certificate Authority'
- name: EXAMPLE-TLS
  type: certificate
  options: 
    ca: EXAMPLE-CA
    common_name: example.com

instance_groups:
  jobs: 
  - name: demo 
    release: demo
    properties:
      demo:
        password: (( EXAMPLE-PASSWORD ))
        tls: 
          certificate: (( EXAMPLE-TLS.certificate ))
          private_key: (( EXAMPLE-TLS.private_key ))

Variable Namespacing

Deployment manifests often use common variable names; for example, (( PASSWORD )). To avoid variable name collisions between deployments, the BOSH Director automatically stores variables with the BOSH Director name and deployment name. For example, the variable (( EXAMPLE-PASSWORD )) is stored in CredHub as /BOSH-Director-name/deployment-name/example-password.

Other Namespacing Options

Use a BOSH link to share credentials across deployments. You can read about BOSH links in the v1.11 Release Notice. Alternatively, if you want to use an exact name, prefixing the variable with a forward slash (/) will cause the Director to use the exact name you type. An example of a precisely typed variable follows.

((/EXAMPLE-PASSWORD))

Reference Existing CAs in CredHub Variables

This section describes how to reference existing CAs stored in CredHub correctly in your tile’s property configuration.

In Ops Manager v2.9 and later, operators can perform a bulk rotation of all CAs and certificates in a foundation, which may include leaf certificates used by individual service tiles. Ops Manager invokes CredHub Maestro to perform this operation.

CredHub Maestro requires that any triple parentheses references to CAs that sign leaf certificates must return a concatenated version of the CA. The concatenated version, which includes the older and newer CA, ensures that jobs using leaf certificates do not lose trusted state during CA rotation. This translates to the least amount of downtime of your tile’s services during certificate rotation.

When referencing a CA stored in CredHub, use the format LEAF-CERTIFICATE-NAME.ca to ensure that a concatenated version of the CA is returned. Do not reference the CA directly with the format CA-CERTIFICATE-NAME.certificate.

The following table presents examples of the correct and incorrect way to reference CAs and leaf certificates in order to support certificate rotation by CredHub Maestro.

Correct Format Incorrect Format 
templates:
 - name: bpm
   release: bpm
 - manifest: |
...
.properties.routing_backends_client_cert_with_san.cert_pem ))
    private_key: (( .properties.routing_backends_client_cert_with_san.private_key_pem ))
    ca_certs: |
      (( .properties.routing_custom_ca_certificates.value ))
      (( $ops_manager.ca_certificate ))
      ((( /cf/some-diego-leaf-cert.ca )))
      ((( /cf/some-diego-leaf-2-6.ca)))
      forwarded_client_cert: (( .properties.routing_tls_termination.selected_option.parsed_manifest(gorouter_forwarded_client_cert) ))
variables: 
 - name: /cf/diego-instance-identity-root-ca
    options:
      common_name: Diego Instance Identity Root CA
      duration: 1095
      is_ca: true
      type: certificate
 - name: /cf/diego-instance-identity-root-ca-2-6
    options:
      common_name: Diego Instance Identity Root CA
      duration: 1095
      is_ca: true
      type: certificate 
  - name: /cf/some-diego-leaf-cert.ca
    options:
      ca: /cf/diego-instance-identity-root-ca
      type: certificate
  - name: /cf/some-diego-leaf-2-6.ca
    options:
      ca: /cf/diego-instance-identity-root-ca-2-6
      type: certificate 
templates:
 - name: bpm
   release: bpm
 - manifest: |
...
.properties.routing_backends_client_cert_with_san.cert_pem ))
    private_key: (( .properties.routing_backends_client_cert_with_san.private_key_pem ))
    ca_certs: |
      (( .properties.routing_custom_ca_certificates.value ))
      (( $ops_manager.ca_certificate ))
      ((( /cf/diego-instance-identity-root-ca.certificate )))
      ((( /cf/diego-instance-identity-root-ca-2-6.certificate )))
      forwarded_client_cert: (( .properties.routing_tls_termination.selected_option.parsed_manifest(gorouter_forwarded_client_cert) ))
variables: 
 - name: /cf/diego-instance-identity-root-ca
   options:
     common_name: Diego Instance Identity Root CA
     duration: 1095
     is_ca: true
     type: certificate
- name: /cf/diego-instance-identity-root-ca-2-6
  options:
     common_name: Diego Instance Identity Root CA
     duration: 1095
     is_ca: true
     type: certificate