Task Reference
Pivotal Platform Automation Tasks
This document lists each Pivotal Platform Automation task, and provides information about their intentions, inputs, and outputs.
The tasks are presented, in their entirety, as they are found in the product.
The docker image can be used to invoke the commands in each task locally.
Use --help
for more information. To learn more see the running-commands-locally section.
apply-changes
Triggers an install on the Ops Manager described by the auth file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
1 2 3 |
|
1 2 3 4 5 |
|
apply-director-changes
apply-changes
can also be used to trigger an install for just the BOSH Director
with the --skip-deploy-products
/-sdp
flag.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
1 2 3 4 |
|
1 2 3 4 5 |
|
assign-multi-stemcell
assign-multi-stemcell
assigns multiple stemcells to a provided product.
This feature is only available in OpsMan 2.6+.
For more information on how to utilize this workflow,
check out the Stemcell Handling topic.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
|
1 2 3 4 |
|
1 2 3 4 5 |
|
assign-stemcell
assign-stemcell
assigns a stemcell to a provided product. For more information on how to utilize
this workflow, check out the Stemcell Handling topic.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
|
1 2 3 4 |
|
1 2 3 4 5 |
|
configure-authentication
Configures Ops Manager with an internal userstore and admin user account. See configure-saml-authentication to configure an external SAML user store, and configure-ldap-authentication to configure with LDAP.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
1 2 3 4 |
|
1 2 3 4 5 6 7 |
|
For details on the config file expected in the config
input,
please see Generating an Auth File.
configure-director
Configures the BOSH Director with settings from a config file. See staged-director-config, which can extract a config file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
1 2 3 4 5 6 |
|
GCP with service account
For GCP, if service account is used, the property associated_service_account has to be set explicitly in the iaas_configuration
section.
configure-ldap-authentication
Configures Ops Manager with an external LDAP user store and admin user account. See configure-authentication to configure an internal user store, and configure-saml-authentication to configure with SAML.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
1 2 3 4 |
|
1 2 3 4 5 6 |
|
For more details on using LDAP, please refer to the Ops Manager documentation.
For details on the config file expected in the config
input,
please see Generating an Auth File.
configure-product
Configures an individual, staged product with settings from a config file.
Not to be confused with Ops Manager's built-in import, which reads all deployed products and configurations from a single opaque file, intended for import as part of backup/restore and upgrade lifecycle processes.
See staged-config, which can extract a config file, and upload-and-stage-product, which can stage a product that's been uploaded.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
1 2 3 4 5 6 7 8 |
|
configure-saml-authentication
Configures Ops Manager with an external SAML user store and admin user account. See configure-authentication to configure an internal user store, and configure-ldap-authentication to configure with LDAP.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
1 2 3 4 |
|
1 2 3 4 5 6 |
|
Bosh Admin Client
By default, this task creates a bosh admin client. This is helpful for some advanced workflows that involve communicating directly with the BOSH Director. It is possible to disable this behavior; see our config file documentation for details.
Configuring SAML has two different auth flows for the UI and the task.
The UI will have a browser based login flow.
The CLI will require client-id
and client-secret
as it cannot do a browser login flow.
For more details on using SAML, please refer to the Ops Manager documentation
For details on the config file expected in the config
input,
please see Generating an Auth File.
create-vm
Creates an unconfigured Ops Manager VM.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
|
1 2 3 4 5 6 7 |
|
This task requires a config file specific to the IaaS being deployed to. Please see the configuration page for more specific examples.
The task does specific CLI commands for the creation of the Ops Manager VM on each IAAS. See below for more information:
AWS
- Requires the image YAML file from Pivnet
- Validates the existence of the VM if defined in the statefile, if so do nothing
- Chooses the correct ami to use based on the provided image YAML file from Pivnet
- Creates the vm configured via opsman config and the image YAML. This only attaches existing infrastructure to a newly created VM. This does not create any new resources
- The public IP address, if provided, is assigned after successful creation
Azure
- Requires the image YAML file from Pivnet
- Validates the existence of the VM if defined in the statefile, if so do nothing
- Copies the image (of the OpsMan VM from the specified region) as a blob into the specified storage account
- Creates the Ops Manager image
- Creates a VM from the image. This will use unmanaged disk (if specified), and assign a public and/or private IP. This only attaches existing infrastructure to a newly createdVM. This does not create any new resources.
GCP
- Requires the image YAML file from Pivnet
- Validates the existence of the VM if defined in the statefile, if so do nothing
- Creates a compute image based on the region specific Ops Manager source URI in the specified Ops Manager account
- Creates a VM from the image. This will assign a public and/or private IP address, VM sizing, and tags. This does not create any new resources.
Openstack
- Requires the image YAML file from Pivnet
- Validates the existence of the VM if defined in the statefile, if so do nothing
- Recreates the image in openstack if it already exists to validate we are using the correct version of the image
- Creates a VM from the image. This does not create any new resources
- The public IP address, if provided, is assigned after successful creation
Vsphere
- Requires the OVA image from Pivnet
- Validates the existence of the VM if defined in the statefile, if so do nothing
- Build ipath from the provided datacenter, folder, and vmname provided in the config file. The created VM is stored on the generated path. If folder is not provided, the vm will be placed in the datacenter.
- Creates a VM from the image provided to the
create-vm
command. This does not create any new resources
credhub-interpolate
Interpolate credhub entries into configuration files
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
This task requires a valid credhub with UAA client and secret. For information on how to set this up, see Secrets Handling
delete-installation
Delete the Ops Manager Installation
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
1 2 3 |
|
1 2 3 4 5 |
|
delete-vm
Deletes the Ops Manager VM instantiated by create-vm.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
|
1 2 3 4 5 6 7 8 9 10 |
|
This task requires the state file generated create-vm.
The task does specific CLI commands for the deletion of the Ops Manager VM and resources on each IAAS. See below for more information:
AWS
- Deletes the VM
Azure
- Deletes the VM
- Attempts to delete the associated disk
- Attempts to delete the associated nic
- Attempts to delete the associated image
GCP
- Deletes the VM
- Attempts to delete the associated image
Openstack
- Deletes the VM
- Attempts to delete the associated image
vSphere
- Deletes the VM
download-product
Ops Manager 2.5
The filename for the artifact downloaded from Ops Manager is changed! If your resources or pipelines have a regex for the Ops Manager filename, you may be affected. (Please see Ops Manager's official notice for more information)
Downloads a product specified in a config file from Pivnet. Optionally, also downloads the latest stemcell for that product.
Downloads are cached, so files are not re-downloaded each time. When downloading from Pivnet, the cached file is verified using the Pivnet checksum to validate the integrity of that file. If it does not, the file is re-downloaded. When downloading from s3, the cached file is not-verified, as there is no checksum from the s3 API to use.
Outputs can be persisted to an S3-compatible blobstore using a put
to an appropriate resource
for later use with the download-product-s3
,
or used directly as inputs to upload-and-stage-product
and upload-stemcell tasks.
This task requires a download-product config file.
If stemcell-iaas is specified in the download-product config file,
and the specified product is a .pivotal
file,
download-product
will attempt to download the stemcell for the product.
It will retrieve the latest compatible stemcell for the specified IaaS.
The valid IaaSs are:
aws
azure
google
openstack
vsphere
If S3 configuration is present in the download-product config file,
the slug and version of the downloaded product file will be prepended in brackets to the filename.
For example:
-
original-pivnet-filenames:
1 2
ops-manager-aws-2.5.0-build.123.yml cf-2.5.0-build.45.pivotal
-
download-product-filenames if S3 configuration is present:
1 2
[ops-manager,2.5.0]ops-manager-aws-2.5.0-build.123.yml [elastic-runtime,2.5.0]cf-2.5.0-build.45.pivotal
This is to allow the same config parameters that let us select a file from Pivnet select it again when pulling from S3. Note that the filename will be unchanged if S3 keys are not present in the configuration file. This avoids breaking current pipelines.
When using the s3 resource in concourse
If you are using a regexp
in your s3 resource definition that explicitly requires the pivnet filename to be the start of the
regex, (i.e., the pattern starts with ^
) this won't work when using S3 config.
The new file format preserves the original filename, so it is still possible to match on that -
but if you need to match from the beginning of the filename, that will have been replaced by the prefix described above.
When specifying Pivotal Application Service-Windows
This task will automatically download and inject the winfs for pas-windows.
When specifying Pivotal Application Service-Windows on Vsphere
This task cannot download the stemcell for pas-windows on vSphere. To build this stemcell manually, please reference the Creating a vSphere Windows Stemcell guide in Pivotal Documentation.
When only downloading from Pivnet
When the download product config only has Pivnet credentials,
it will not add the prefix to the downloaded product.
For example, example-product.pivotal
from Pivnet will be outputed
as example-product.pivotal
.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
|
1 2 3 4 5 6 |
|
download-product-s3
Downloads a product specified in a config file from an S3-compatible blobstore. This is useful when retrieving assets in an offline environment.
Downloads are cached, so files are not re-downloaded each time.
This is intended to be used with files downloaded from Pivnet by download-product
and then persisted to a blobstore using a put
step.
Outputs can be used directly as an input to upload-and-stage-product and upload-stemcell tasks.
This task requires a download-product config file.
The same configuration file should be used with both this task and download-product
.
This ensures that the same file
is being captured with both tasks.
The product files uploaded to s3 for download with this task require a specific prefix:
[product-slug,semantic-version]
.
This prefix is added by the download-product
task
when S3 keys are present in the configuration file.
This is the meta information about the product from Pivnet,
which is not guaranteed to be in the original filename.
This tasks uses the meta information to be able to perform
consistent downloads from s3
as defined in the provided download config.
For example:
-
original-pivnet-filenames:
1 2
ops-manager-aws-2.5.0-build.123.yml cf-2.5.0-build.45.pivotal
-
filenames expected by
download-product-s3
in a bucket:1 2
[ops-manager,2.5.0]ops-manager-aws-2.5.0-build.123.yml [elastic-runtime,2.5.0]cf-2.5.0-build.45.pivotal
When only downloading from Pivnet
When the download product config only has Pivnet credentials,
it will not add the prefix to the downloaded product.
For example, example-product.pivotal
from Pivnet will be outputed
as example-product.pivotal
.
Info
It's possible to use IAM instance credentials instead of providing S3 creds in the config file. See download-product config file for details.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
|
1 2 3 4 5 6 7 8 9 10 |
|
export-installation
Exports an existing Ops Manager to a file.
This is the first part of the backup/restore and upgrade lifecycle processes. This task is used on a fully installed and healthy Ops Manager to export settings to an upgraded version of Ops Manager.
To use with non-versioned blobstore, you can override INSTALLATION_FILE
param
to include $timestamp
, then the generated installation file will include a sortable
timestamp in the filename.
example:
1 2 |
|
Info
The timestamp is generated using the time on concourse worker. If the time is different on different workers, the generated timestamp may fail to sort correctly. Changing the time or timezone on workers might interfere with ordering.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
1 2 3 4 5 6 7 |
|
Warning
It is recommended to persist the zip file exported from export-installation to an external file store (eg S3) on a regular basis. The exported installation can restore the Ops Manager to a working state if it is non-functional.
import-installation
Imports a previously exported installation to Ops Manager.
This is a part of the backup/restore and upgrade lifecycle processes. This task is used after an installation has been exported and a new Ops Manager has been deployed, but before the new Ops Manager is configured.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
1 2 3 4 5 6 7 |
|
1 2 3 4 5 6 7 8 |
|
make-git-commit
Copies a single file into a repo and makes a commit. Useful for persisting the state output of tasks that manage the vm, such as:
Also useful for persisting the configuration output from:
Info
This commits all changes present
in the repo used for the repository
input,
in addition to copying in a single file.
Info
This does not perform a git push
!
You will need to put
the output of this task to a git resource to persist it.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
pre-deploy-check
Checks if the Ops Manager director is configured properly and validates the configuration.
This feature is only available in Ops Manager 2.6+.
Additionally, checks each of the staged products
and validates they are configured correctly.
This task can be run at any time
and can be used a a pre-check for apply-changes
.
The checks that this task executes are:
- is configuration complete and valid
- is the network assigned
- is the availability zone assigned
- is the stemcell assigned
- what stemcell type/version is required
- are there any unset/invalid properties
- did any ops manager verifiers fail
If any of the above checks fail
the task will fail.
The failed task will provide a list of errors that need to be fixed
before an apply-changes
could start.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
1 2 3 4 |
|
1 2 3 4 5 |
|
stage-product
Staged a product to the Ops Manager specified in the config file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
1 2 3 4 5 6 |
|
staged-config
Downloads the configuration for a product from Ops Manager.
Not to be confused with Ops Manager's built-in export, which puts all deployed products and configurations into a single file, intended for import as part of backup/restore and upgrade lifecycle processes.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
1 2 3 4 5 6 7 8 9 10 11 12 |
|
1 2 3 4 5 6 7 8 |
|
staged-director-config
Ops Manager 2.5
The filename for the artifact downloaded from Ops Manager is changed! If your resources or pipelines have a regex for the Ops Manager filename, you may be affected. (Please see Ops Manager's official notice for more information)
Downloads configuration for the BOSH director from Ops Manager.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
1 2 3 4 |
|
1 2 3 4 5 6 |
|
The configuration is exported to the generated-config
output.
It does not extract credentials from Ops Manager
and replaced them all with YAML interpolation (())
placeholders.
This is to ensure that credentials are never written to disk.
The credentials need to be provided from an external configuration when invoking configure-director.
Info
staged-director-config will not be able to grab all sensitive fields in your Ops Manager installation (for example: vcenter_username and vcenter_password if using vsphere). To find these missing fields, please refer to the Ops Manager API Documentation
test
An example task to ensure the assets and docker image are setup correctly in your concourse pipeline.
1 2 3 4 5 6 7 8 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
1 2 3 |
|
test-interpolate
An example task to ensure that all required vars are present when interpolating into a base file. For more instruction on this topic, see the variables section
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
1 2 3 4 5 6 7 8 |
|
upgrade-opsman
Upgrades an existing Ops Manager to a new given Ops Manager version
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
|
1 2 3 4 5 6 7 8 |
|
For more information about this task and how it works, see the upgrade page.
upload-and-stage-product
Uploads and stages product to the Ops Manager specified in the config file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
1 2 3 4 5 6 |
|
upload-product
Uploads a product to the Ops Manager specified in the config file.
If a shasum is provided in the config.yml, the integrity product will be verified with that shasum before uploading.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
1 2 3 4 5 6 7 8 9 10 11 |
|
1 2 3 4 5 6 |
|
upload-stemcell
Uploads a stemcell to Ops Manager.
Note that the filename of the stemcell must be exactly as downloaded from Pivnet. Ops Manager parses this filename to determine the version and OS of the stemcell.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
|
1 2 3 4 5 6 7 8 9 10 11 12 |
|
1 2 3 4 5 6 |
|