Creating a Product Config File
Extracting a product configuration file, an externalized config that lives outside of Ops Manager, can make it easier to manage multiple foundations as well as help with:
- avoiding configuration drift
- configuration promotion
A configuration file can be generated from the tile metadata directly from Pivotal Network.
- A token for the Pivotal Network API is required.
- You'll need the Platform Automation Docker Image imported and ready to run.
- For products that have multiple
.pivotalfiles, you'll need a glob pattern uniquely matching one of them.
Generate the Config Template Directory
sourceit to avoid credentials in your bash history.)
1 2 3 4 5 6 7
This will create or update a directory at
cd into the directory to get started creating your config.
Interpolate a Config
The directory will contain a product.yml file.
This is the template for the product configuration you're about to build.
Open it in an editor of your choice.
Get familiar with what's in there.
The values will be variables intended to be interpolated from other sources,
designated with the
You can find the value for any property with a default in the
This file serves as a good example of a variable source.
You'll need to create a vars file of your own for variables without default values.
For the base template, you can get a list of required variables by running
1 2 3 4 5 6
Put all those vars in a file and give them the appropriate values.
Once you've included all the variables,
the output will be the finished template.
For the rest of this guide,
we will refer to these vars as
There may be situations that call for splitting your vars across multiple files. This can be useful if there are vars that need to be interpolated when you apply the configuration, rather than when you create the final template. You might consider creating a seperate vars file for each of the following cases:
- credentials (these vars can then be persisted separately/securely)
- foundation-specific variables when using the same template for multiple foundations
You can use the
--skip-missing flag when creating your final template
om interpolate to leave such vars to be rendered later.
If you're having trouble figuring out what the values should be, here are some approaches you can use:
- Look in the template where the variable appears for some additional context of its value.
- Look at the tile's online documentation
Upload the tile to an Ops Manager and visit the tile in the Ops Manager UI to see if that provides any hints.
If you are still struggling, inspecting the html of the Ops Manager webpage can more accurately map the value names to the associated UI element.
When Using The Ops Manager Docs and UI
Be aware that the field names in the UI do not necessarily map directly to property names.
The above process will get you a default installation, with no optional features or variables, that is entirely deployed in a single Availability Zone (AZ).
In order to provide non-required variables, use multiple AZs, or make non-default selections for some options, you'll need to use some of the ops files in one of the following four directories:
|features||allow the enabling of selectors for a product. For example, enabling/disabling of an s3 bucket|
|network||contains options for enabling 2-3 availability zones for network configuration|
|optional||contains optional properties without defaults. For optional values that can be provided more than once, there's an ops file for each param count|
|resource||contains configuration that can be applied to resource configuration. For example, BOSH vm extensions|
For more information on BOSH VM Extensions, refer to the Creating a Director Config File How-to Guide.
To use an ops file, add
with the path to the ops file you want to use to your
So, to enable TCP routing in Pivotal Application Service, you would add
For the rest of this guide, the vars for this feature
are referred to as
If you run your complete command, you should again get a list of any newly-required variables.
1 2 3 4 5 6 7 8 9
Finalize Your Configuration
Once you've selected your ops files and created your vars files, decide which vars you want in the template and which you want to have interpolated later.
Create a final template and write it to a file,
using only the vars you want to in the template,
--skip-missing to allow the rest to remain as variables.
1 2 3 4 5 6 7 8 9 10 11
You can check-in the resulting configuration to a git repo. For vars that do not include credentials, you can check those vars files in, as well. Handle vars that are secret more carefully.
You can then dispose of the config template directory.
From A Staged Product
A configuration can be generated from a staged product on an already existing Ops Manager.
To extract the configuration for a product, you will first need to do the following:
- Upload and stage your desired product(s) to a fully deployed Ops Manager. For example, let's use Pivotal Application Service on Vsphere with NSX-T
- Configure your product manually according to the product's install instructions.
om has a command called staged-config, which is used to extract staged product configuration present in the Ops Manager UI of the targeted foundation.
Sample usage, using
om directly and assuming the Pivotal Application Service product:
om --env env.yml staged-config --include-placeholders --product-name cf > tile-config.yml
Most products will contain the following high level keys:
You can check the file in to git.
For convenience, Platform Automation provides you with two ways to use the
om staged-config command. The command can be run as a task
inside of your pipeline. As an example of how to invoke this for the Pivotal Application Service product
in your pipeline.yml(resources not listed):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
env.yml, download the current staged configuration of your product, and put it into a
generated-configfolder in the concourse job. The
putin concourse allows you to persist this config outside the concourse container.
Alternatively, this can be run external to concourse by using docker. An example of how to do this using on the linux/mac command line:
1 2 3
Using Ops Files for Multi-Foundation
--include-placeholders in the
om command is a vital first step to externalizing
your configuration for multiple foundations. This will search the Ops Manager product
for fields marked as "secrets", and replace those values with
In order to fully support multiple foundations, however, a bit more work is necessary. There are two ways to do this: using secrets management or ops files. This section will explain how to support multiple foundations using ops files.
Starting with an incomplete Pivotal Application Service config from vSphere as an example:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
For a single foundation deploy, leaving values such as
".cloud_controller.apps_domain" as-is would work fine. For multiple
foundations, this value will be different per deployed foundation. Other values,
.cloud_controller.encrypt_key have a secret that
already have a placeholder from
om. If different foundations have different
load requirements, even the values in
resource-config can be edited using
Using the example above, let's try filling in the existing placeholder for
cloud_controller.apps_domain in our first foundation.
1 2 3 4
To test that the ops file will work in your
base.yml, this can be done locally using
This will output
base.yml with the replaced(interpolated) values:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
Anything that needs to be different per deployment can be replaced via ops files as long as the
path: is correct.
Upgrading products to new patch versions:
- Configuration settings should not differ between successive patch versions within the same minor version line. Underlying properties or property names may change, but the tile's upgrade process automatically translates properties to the new fields and values.
- Pivotal cannot guarantee the functionality of upgrade scripts in third-party Pivotal Platform products.
Replicating configuration settings from one product to the same product on a different foundation:
- Because properties and property names can change between patch versions of a product, you can only safely apply configuration settings across products if their versions exactly match.