Encrypting Cloud Controller Database and Rotating Keys

Page last updated:

This topic describes how operators can encrypt sensitive data in the Cloud Controller (CC) database using operator-provided keys.

Enabling Encryption

In the following steps, the examples show a single encryption key being configured and set as the current key, but multiple keys can be stored under keys. Multiple keys are allowed so that CC can decrypt your previously-encrypted sensitive data before rotating to a new key. Only the key with its label set as the current_key_label will be used for encryption.

  1. Choose a value for the encryption key (e.g. "example-random-key-string").

  2. Choose a label for the encryption key (e.g. encryption_key_2019_06_07). The label must abide by the following restrictions:

    • 127 character limit
    • Cannot contain a : (colon)
  3. Format the encryption key values into YAML as follows:

    current_key_label: "encryption_key_2019_06_07"
    keys:
      encryption_key_2019_06_07: "example-random-key-string"
    

    Note: If your CC instance groups have the db_encryption_key key, ensure that you add db_encryption_key to the keys block, and that you set its corresponding key label to value of current_key_label. For example:

    db_encryption_key: "example-random-key-string"
    current_key_label: "encryption_key_2019_06_07"
    keys:
      encryption_key_2019_06_07: "example-random-key-string"
    
  4. Make the changes by either creating and applying an ops file or directly modifying the manifest.

  5. Deploy the changes with the command that corresponds to your use case:

    • With an ops file
      $ bosh deploy -o YOUR_OPS_FILE_NAME PATH_TO_YOUR_MANIFEST
      
    • Direct modification of the manifest
      $ bosh deploy PATH_TO_YOUR_MANIFEST
      
  6. Run the bosh errand to encrypt the fields.

    $ bosh run-errand rotate-cc-database-key
  7. Once the errand has run successfully, you can safely remove the db_encryption_key from the manifest.

    Warning: If you remove db_encryption_key from the manifest before running the errand, CC will no longer be able to decrypt your previously-encrypted sensitive data.

Configuring via an Ops File

The recommended way of enabling encryption is to use an ops file. YAML anchors (e.g. encryption_info in the example below) can also be used to reduce repetition and ensure consistency of encryption key properties across the instance groups.

For example, the following could be used as an ops file:

- type: replace
  path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/database_encryption?
  value: &encryption_info
    keys:
      encryption_key_2019_06_07: "example-random-key-string"
    current_key_label: "encryption_key_2019_06_07"

- type: replace
  path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/database_encryption?
  value: *encryption_info

- type: replace
  path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/database_encryption?
  value: *encryption_info

Configuring via the Manifest

For each CC instance group (api, cc-worker, scheduler) in the manifest, insert the database_encryption YAML block so that the final YAML looks like the following:

instance_groups:
  api:
    jobs:
      cloud_controller_ng:
        properties:
          database_encryption: &encryption_info
            keys:
              encryption_key_2019_06_07: "example-random-key-string"
            current_key_label: "encryption_key_2019_06_07"
  ...
  cc-worker:
    jobs:
      cloud_controller_ng:
        properties:
          database_encryption: *encryption_info
  ...
  scheduler:
    jobs:
      cloud_controller_ng:
        properties:
          database_encryption: *encryption_info

Note: The keys and current_key_label field must match across all instance groups - our example uses YAML anchors to achieve this.

Rotating Encryption Keys

In order to rotate an encryption key:

  1. Add a new encryption key to the list of keys (e.g. encryption_key_2019_06_08 below).
  2. Update the current_key_label to be the label of the new encryption key (e.g. updating the current_key_label to be "encryption_key_2019_06_08" below).

    The following is an example of an ops file used for rotation:

    - type: replace
      path: /instance_groups/name=api/jobs/name=cloud_controller_ng/properties/cc/database_encryption?
      value: &encryption_info
        keys:
          encryption_key_2019_06_07: "example-random-key-string"
          encryption_key_2019_06_08: "other-random-key-string"
        current_key_label: "encryption_key_2019_06_08"
    
    - type: replace
      path: /instance_groups/name=cc-worker/jobs/name=cloud_controller_worker/properties/cc/database_encryption?
      value: *encryption_info
    
    - type: replace
      path: /instance_groups/name=scheduler/jobs/name=cloud_controller_clock/properties/cc/database_encryption?
      value: *encryption_info
    
  3. Deploy the changes with the following command.

    $ bosh deploy -o YOUR_OPS_FILE_NAME PATH_TO_YOUR_MANIFEST

    Note: Regardless of whether the errand is run or not, any updates to sensitive data will be encrypted with the new key.

  4. Run the bosh errand to rotate the encryption key.

    $ bosh run-errand rotate-cc-database-key
  5. Once the errand has run successfully, you can safely remove the old key from the manifest.

    Warning: If you remove the old key from the manifest before running the errand, CC will no longer be able to decrypt your previously-encrypted sensitive data.

Create a pull request or raise an issue on the source for this page in GitHub