Skip to content
Support Downloads Contact Us Sign In

Inputs

These are the inputs that can be provided to the tasks. Each task can only take a specific set, indicated under the inputs property of the YAML.

env

The env input for a task expects to have a env.yml file. This file contains properties for targeting and logging into the Ops Manager API.

basic authentication

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
---
target: https://pcf.example.com
connect-timeout: 30            # default 5
request-timeout: 1800          # default 1800
skip-ssl-validation: false     # default false
username: username
password: password
# decryption-passphrase is optional,
# except for use with `import-installation`.
# OpsMan depends on the passphrase
# to decrypt the imported installation.
# For other commands, providing this key allows
# decryption of the OpsMan VM after reboot,
# which would otherwise need to be done manually.
decryption-passphrase: passphrase

uaa authentication

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
---
target: https://pcf.example.com
connect-timeout: 30          # default 5
request-timeout: 1800        # default 1800
skip-ssl-validation: false   # default false
client-id: client_id
client-secret: client_secret
# decryption-passphrase is optional,
# except for use with `import-installation`.
# OpsMan depends on the passphrase
# to decrypt the imported installation.
# For other commands, providing this key allows
# decryption of the OpsMan VM after reboot,
# which would otherwise need to be done manually.
decryption-passphrase: passphrase
Getting the client-id and client-secret

Ops Manager will by preference use Client ID and Client Secret if provided. To create a Client ID and Client Secret

  1. uaac target https://YOUR_OPSMANAGER/uaa
  2. uaac token sso get if using SAML or uaac token owner get if using basic auth. Specify the Client ID as opsman and leave Client Secret blank.
  3. Generate a client ID and secret
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
uaac client add -i
Client ID:  NEW_CLIENT_NAME
New client secret:  DESIRED_PASSWORD
Verify new client secret:  DESIRED_PASSWORD
scope (list):  opsman.admin
authorized grant types (list):  client_credentials
authorities (list):  opsman.admin
access token validity (seconds):  43200
refresh token validity (seconds):  43200
redirect uri (list):
autoapprove (list):
signup redirect url (url):

Ops Manager config

The config for an Ops Manager described IAAS specific information for creating the VM -- i.e. VM flavor (size), IP addresses

The config input for opsman task expects to have a opsman.yml file. The configuration of the opsman.yml is IAAS specific.

Specific examples for each IaaS are as follows:

AWS

These required properties are adapted from the instructions outlined in Launching an Ops Manager Director Instance on AWS

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
---
opsman-configuration:
  aws:
    region: us-west-2
    vm_name: ops-manager-vm # defaults ops-manager-vm
    boot_disk_size: 100 # default 200
    vpc_subnet_id: subnet-0292bc845215c2cbf
    security_group_id: sg-0354f804ba7c4bc41
    key_pair_name: ops-manager-key
    iam_instance_profile_name: ops-manager-iam
    instance_type: m5.large
    # At least one IP address (public or private)
    # needs to be assigned to the VM.
    # It is also permissable to assign both.
    public_ip: 1.2.3.4
    private_ip: 10.0.0.2

    # Required if use_instance_profile is false
    # omit if using Instance Profiles
    access_key_id: sample-access-id
    secret_access_key: sample-secret-access-key

    # If using Instance Profiles (omit if using AWS Credentials)
    use_instance_profile: true # default false

    # Optional, necessary if a role is needed to authorize the instance profile
    assume_role: arn:aws:iam::123456789:role/test

Info

At least one IP address (public or private) must be assigned to the Ops Manager VM. Both can be assigned, too.

Using instance_profile to Avoid Secrets

For authentication you must either set use_instance_profile: true or provide a secret_key_id and secret_access_key. You must remove key information if you're using an instance profile. Using an instance profile allows you to avoid interpolation, as this file then contains no secrets.

Azure

These required properties are adapted from the instructions outlined in Launching an Ops Manager Director Instance on Azure

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
---
opsman-configuration:
  azure:
    subscription_id: 90f35f10-ea9e-4e80-aac4-d6778b995532
    resource_group: res-group
    tenant_id: 3e52862f-a01e-4b97-98d5-f31a409df682
    client_id: 5782deb6-9195-4827-83ae-a13fda90aa0d
    client_secret: 6Iaue71Lqxfq
    location: westus
    container: opsmanagerimage                    # container for opsman image
    network_security_group: ops-manager-security-group

    # Note that there are several environment-specific details in this path
    vpc_subnet: /subscriptions/<MY_SUBSCRIPTION_ID>/resourceGroups/<MY_RESOURCE_GROUP>/providers/Microsoft.Network/virtualNetworks/<MY_VNET>/subnets/<MY_SUBNET>

    storage_account: opsman                       # account name of container

    # Optional
    # only needed if your client doesn't have the needed storage permissions
    storage_key: pEuXDaDK/WWo...

    ssh_public_key: ssh-rsa AAAAB3NzaC1yc2EAZ...  # ssh key to access VM
    vm_name: ops-manager-vm                       # default: ops-manager-vm
    boot_disk_size: 100                           # default: 200
    cloud_name: AzureCloud                        # default: AzureCloud

    # This flag is only respected by the create-vm & upgrade-opsman commands
    # set to true if you want to create the new opsman vm with unmanaged disk
    # delete-vm discovers the disk type from the VM
    use_unmanaged_disk: false

    # At least one IP address (public or private)
    # needs to be assigned to the VM.
    # It is also permissable to assign both.
    public_ip: 1.2.3.4
    private_ip: 10.0.0.3

Info

At least one IP address (public or private) must be assigned to the Ops Manager VM. Both can be assigned, too.

GCP

These required properties are adapted from the instructions outlined in Launching an Ops Manager Director Instance on GCP

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
---
opsman-configuration:
  gcp:
    # either gcp_service_account json is required
    # or a gcp_service_account_name associated with the project
    # you must remove whichever you don't use
    gcp_service_account: |
      {
        "type": "service_account",
        "project_id": "project-id",
        "private_key_id": "af719b1ca48f7b6ac67ca9c5319cb175",
        "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
        "client_email": "user@project-id.iam.gserviceaccount.com",
        "client_id": "1234567890",
        "auth_uri": "https://accounts.google.com/o/oauth2/auth",
        "token_uri": "https://accounts.google.com/o/oauth2/token",
        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
        "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/user%40project-id.iam.gserviceaccount.com"
      }
    gcp_service_account_name: user@project-id.iam.gserviceaccount.com
    project: project-id
    region: us-central1
    zone: us-central1-b
    vm_name: ops-manager-vm               # default: ops-manager-vm
    # For SharedVPC: projects/[HOST_PROJECT_ID]/regions/[REGION]/subnetworks/[SUBNET]
    vpc_subnet: infrastructure-subnet
    tags: ops-manager
    # This CPU, Memory and disk size demonstrated here
    # match the defaults, and needn't be included if these are the desired values
    custom_cpu: 2
    custom_memory: 8
    boot_disk_size: 100
    # At least one IP address (public or private) needs to be assigned to the VM.
    public_ip: 1.2.3.4.
    private_ip: 10.0.0.2
    # scopes to add to created vm
    scopes: ["my-scope"]
    # key to ssh onto the Ops Manger VM.
    # used to manage non-tile BOSH add-ons
    ssh_public_key: ssh-rsa some-public-key...

Info

At least one IP address (public or private) must be assigned to the Ops Manager VM. Both can be assigned, too.

Using a Service Account Name to Avoid Secrets

For authentication either gcp_service_account or gcp_service_account_name is required. You must remove the one you are not using note that using gcp_service_account_name allows you to avoid interpolation, as this file then contains no secrets.

Support for Shared VPC is done via configuring the vpc_subnet path to include the host project id, region of the subnet, and the subnet name.

For example:

projects/[HOST_PROJECT_ID]/regions/[REGION]/subnetworks/[SUBNET]

Openstack

These required properties are adapted from the instructions outlined in Launching an Ops Manager Director Instance on Openstack

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
---
opsman-configuration:
  openstack:
    auth_url: http://os.example.com:5000/v2.0
    project_name: project
    net_id: 26a13112-b6c2-11e8-96f8-529269fb1459
    username: admin
    password: password
    key_pair_name: opsman-keypair
    security_group_name: opsman-sec-group
    vm_name: ops-manager-vm      # default: ops-manager-vm
    # At least one IP address (public or private) needs to be assigned to the VM.
    public_ip: 1.2.3.4
    private_ip: 10.0.0.3
    flavor: m1.xlarge            # default: m1.xlarge
    project_domain_name: default
    user_domain_name: default
    identity_api_version: 3 # default: 2
    insecure: true # default: false
    availability_zone: zone-01

Info

At least one IP address (public or private) must be assigned to the Ops Manager VM. Both can be assigned, too.

vSphere

These required properties are adapted from the instructions outlined in Deploying BOSH and Ops Manager to vSphere

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
---
opsman-configuration:
  vsphere:
    vcenter:
      url: vcenter.example.com
      username: admin
      password: password
      datastore: exmple-ds-1
      ca_cert: certificate
      host: example-host                     # vCenter host to deploy Ops Manager in
      datacenter: example-dc
      resource_pool: /example-dc/host/example-host/Resources/ResPool # or /<Data Center Name>/host/<Cluster Name>
      folder: /example-dc/vm/Folder
      insecure: 1                            # default: 0 (secure); 1 (insecure)
    disk_type: thin                          # example: thin|thick
    private_ip: 10.0.0.2
    dns: 8.8.8.8
    ntp: ntp.example.com                     # example: ntp.ubuntu.com
    ssh_password: password                   
    ssh_public_key: ssh-rsa ......           # for Ops Manager >= 2.3, required for Ops Manager >= 2.6
    hostname: pcf.example.com
    network: virtual-network                 # vcenter network to deploy to
    netmask: 255.255.255.192
    gateway: 192.168.10.1
    vm_name: ops-manager-vm                  # default: ops-manager-vm
    memory: 8                                # default: 8 GB
    cpu: 1                                   # default: 1

director config

The config director will set the bosh tile (director) on Ops Manager.

The config input for a director task expects to have a director.yml file. The configuration of the director.yml is IAAS specific for some properties -- i.e. networking.

There are two ways to build a director config.

  1. Using an already deployed Ops Manager, you can extract the config using staged-director-config.
  2. Deploying a brand new Ops Manager requires more effort for a director.yml. The configuration of director is variables based on the features enabled. For brevity, this director.yml is a basic example for vsphere.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
---
az-configuration:
- clusters:
  - cluster: cluster-name
    resource_pool: resource-pool-name
  name: AZ01

properties-configuration:
  iaas_configuration:
    vcenter_host: vcenter.example.com
    vcenter_username: admin
    vcenter_password: password
    ......
  director_configuration:
    blobstore_type: local
    bosh_recreate_on_next_deploy: false
    custom_ssh_banner: null
    ......
  security_configuration:
    generate_vm_passwords: true
    trusted_certificates:
  syslog_configuration:
    enabled: false

network-assignment:
  network:
    name: INFRASTRUCTURE
  other_availability_zones: []
  singleton_availability_zone:
    name: AZ01

networks-configuration:
  icmp_checks_enabled: false
  networks:
  - name: NETWORK-NAME
  ......

resource-configuration:
  compilation:
    instance_type:
      id: automatic
    instances: automatic
  ......

The IAAS specific configuration can be found in the Ops Manager API documentation.

Included below is a list of properties that can be set in the director.yml and a link to the API documentation explaining any IAAS specific properties.

  • az-configuration - a list of availability zones Ops Manager API
  • network-assignment - the network the bosh director is deployed to Ops Manager API
  • networks-configuration - a list of named networks Ops Manager API
  • properties-configuration
    • iaas_configuration - configuration for the bosh IAAS CPI Ops Manager API
    • director_configuration - properties for the bosh director Ops Manager API
    • security_configuration - security properties for the bosh director Ops Manager API
    • syslog_configuration - configure the syslog sinks for the bosh director Ops Manager API
  • resource-configuration - IAAS VM flavor for the bosh director Ops Manager API
  • vmextensions-configuration - create/update/delete vm extensions Ops Manager API

GCP Shared VPC

Support for Shared VPC is done via configuring the iaas_identifier path for the infrastructure subnet, which includes the host project id, region of the subnet, and the subnet name.

For example:

[HOST_PROJECT_ID]/[NETWORK]/[SUBNET]/[REGION]

product config

There are two ways to build a product config.

  1. Using an already deployed product (tile), you can extract the config using staged-config.
  2. Use an example and fill in the values based on the meta information from the tile. For brevity, this product.yml is a basic example for healthwatch.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
---
product-properties:
  .healthwatch-forwarder.bosh_taskcheck_username:
    value: admin
  .healthwatch-forwarder.boshhealth_instance_count:
    value: 1
  .healthwatch-forwarder.boshtasks_instance_count:
    value: 2
  .healthwatch-forwarder.canary_instance_count:
    value: 2
  .healthwatch-forwarder.cli_instance_count:
    value: 2
  .healthwatch-forwarder.health_check_az:
    value: AZ01
  .healthwatch-forwarder.ingestor_instance_count:
    value: 4
  .healthwatch-forwarder.opsman_instance_count:
    value: 2
  .healthwatch-forwarder.publish_to_eva:
    value: true
  .healthwatch-forwarder.worker_instance_count:
    value: 4
  .mysql.skip_name_resolve:
    value: true
  .properties.opsman:
    value: enable
  .properties.opsman.enable.url:
    value: https://pcf.example.com/
network-properties:
  network:
    name: DEPLOYMENT
  other_availability_zones:
  - name: AZ01
  - name: AZ02
  service_network:
    name: SERVICES
  singleton_availability_zone:
    name: AZ01
resource-config:
  healthwatch-forwarder:
    instances: automatic
    persistent_disk:
      size_mb: automatic
    instance_type:
      id: automatic
  migrate-v1.1-v1.2:
    instances: automatic
    instance_type:
      id: automatic
  mysql:
    instances: automatic
    persistent_disk:
      size_mb: automatic
    instance_type:
      id: automatic
  redis:
    instances: automatic
    persistent_disk:
      size_mb: automatic
    instance_type:
      id: automatic

Included below is a list of properties that can be set in the product.yml and a link to the API documentation explaining the properties.

state

This file contains that meta-information needed to manage the Ops Manager VM. The state input for a opsman VM task expects to have a state.yml file.

The state.yml file contains two properties:

  1. iaas is the IAAS the ops manager vm is hosted on. (gcp, vsphere, aws, azure, openstack)
  2. vm_id is the VM unique identifier for the VM. For some IAAS, the vm ID is the VM name.

Different IaaS uniquely identify VMs differently; here are examples for what this file should look like, depending on your IAAS:

``` yaml tab="AWS" iaas: aws

Instance ID of the AWS VM

vm_id: i-12345678987654321 

1
2
3
4
``` yaml tab="Azure"
iaas: azure
# Computer Name of the Azure VM
vm_id: vm_name

``` yaml tab="GCP" iaas: gcp

Name of the VM in GCP

vm_id: vm_name 

1
2
3
4
``` yaml tab="OpenStack"
iaas: openstack
# Instance ID from the OpenStack Overview
vm_id: 12345678-9876-5432-1abc-defghijklmno

``` yaml tab="vSphere" iaas: vsphere

Path to the VM in vCenter

vm_id: /datacenter/vm/folder/vm_name 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
### opsman image

This file is an [artifact from Pivnet](https://network.pivotal.io/products/ops-manager), which contains the VM image on an IAAS.
For vsphere and openstack, it is a full disk image.
For AWS, GCP, and Azure, it is the YAML file of the image locations.

An example on how to pull the AWS image resource using the [Pivnet Concourse Resource](https://github.com/pivotal-cf/pivnet-resource).

```yaml
resource_types:
- name: pivnet
  type: docker-image
  source:
    repository: pivotalcf/pivnet-resource
    tag: latest-final
resources:
- name: opsman-image
  type: pivnet
  source:
    api_token: ((pivnet_token))
    product_slug: ops-manager
    product_version: 2.*
    sort_by: semver
jobs:
- name: get-the-resource
  plan:
  - get: opsman-image
    params:
      globs: ["*AWS*.yml"]

installation

The file contains the information to restore an Ops Manager VM. The installation input for a opsman VM task expects to have a installation.zip file.

This file can be exported from an Ops Manager VM using the export-installation. This file can be imported to an Ops Manager VM using the import-installation.

Warning

This file cannot be manually created. It is a file that must be generated via the export function of Ops Manager.

stemcell

This stemcell input requires the stemcell tarball (.tgz) as downloaded from Pivnet. It must be in the original filename as that is used by Ops Manager to parse metadata. The filename could look like bosh-stemcell-3541.48-vsphere-esxi-ubuntu-trusty-go_agent.tgz.

Warning

This file cannot be manually created. It is a file that must retrieved from Pivnet.

An example on how to pull the vSphere stemcell using the Pivnet Concourse Resource.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
resource_types:
- name: pivnet
  type: docker-image
  source:
    repository: pivotalcf/pivnet-resource
    tag: latest-final
resources:
- name: stemcell
  type: pivnet
  source:
    api_token: ((pivnet_token))
    product_slug: stemcells
    product_version: 3541.*
    sort_by: semver
jobs:
- name: get-the-resource
  plan:
  - get: stemcell
    params:
      globs: ["*vsphere*.tgz"]

product

The product input requires a single tile file (.pivotal) as downloaded from Pivnet.

An example on how to pull the Pivotal Application Service tile using the Pivnet Concourse Resource.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
resource_types:
- name: pivnet
  type: docker-image
  source:
    repository: pivotalcf/pivnet-resource
    tag: latest-final
resources:
- name: stemcell
  type: pivnet
  source:
    api_token: ((pivnet_token))
    product_slug: elastic-runtime
    product_version: 2.*
    sort_by: semver
jobs:
- name: get-the-resource
  plan:
  - get: product
    params:
      globs: ["*cf*.pivotal"]

Warning

This file cannot be manually created. It is a file that must retrieved from Pivnet.

download-product-config

The config input for a download product task can be used with a download-config.yml file to download a tile. The configuration of the download-config.yml looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
---
pivnet-api-token: token
## Note that file globs must be quoted if they start with *;
## otherwise they'll be interpreted as a YAML anchor.
pivnet-file-glob: "*.pivotal"
pivnet-product-slug: product-slugs

## defaults to false and should be excluded if not set to true
# pivnet-disable-ssl: true

## Either product-version OR product-version-regex is required
# product-version: 1.2.3

## Note that the regex mustn't be quoted,
## as escape characters for the regex will confuse yaml parsers.
# product-version-regex: ^1\.2\..*$

## If set, will attempt to download the latest stemcell for the product.
## Ignored if the specified file is not a `.pivotal` file.
## Valid options are: aws, azure, google, openstack, vsphere.
# stemcell-iaas: google

## The following are required only if using download-product-s3.
## Any key marked required above is still required when using S3.
## If s3-bucket is set,
## downloaded product files will have their slug and version prepended.
s3-bucket: s3-bucket
s3-region-name: us-west-1      # required; sufficient for AWS
s3-endpoint: s3.endpoint.com   # if not using AWS, this is required

## Required unless `s3-auth-method` is `iam`
s3-access-key-id: aws-or-minio-key-id
s3-secret-access-key: aws-or-minio-secret-key

## Optional paths for both the product and the associated stemcell
## defaults to the root path of the specified bucket
# s3-product-path: /path/to/product
# s3-stemcell-path: /path/to/stemcell

## defaults to false and should be excluded if not set to true
# s3-disable-ssl: true

## defaults to false;
## made available only because sometimes necessary for compatibility
# s3-enable-v2-signing: true

## defaults to accesskey;
## allows use of AWS instance IAM creds, if available
# s3-auth-type: iam

download-stemcell-product-config

The config input for a download product task can be used with a download-config.yml file to download a stemcell. The configuration of the download-config.yml looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
---
pivnet-api-token: token
## Note that file globs must be quoted if they start with *;
## otherwise they'll be interpreted as a YAML anchor.
pivnet-file-glob: "*vsphere*"
pivnet-product-slug: stemcells-ubuntu-xenial

## defaults to false and should be excluded if not set to true
# pivnet-disable-ssl: true

## Either product-version OR product-version-regex is required
# product-version: "250.82"

## Note that the regex mustn't be quoted,
## as escape characters for the regex will confuse yaml parsers.
# product-version-regex: ^250\..*$


## The following are required only if using download-product-s3.
## Any key marked required above is still required when using S3.
## If s3-bucket is set,
## downloaded product files will have their slug and version prepended.
s3-bucket: s3-bucket
s3-region-name: us-west-1      # required; sufficient for AWS
s3-endpoint: s3.endpoint.com   # if not using AWS, this is required

## Required unless `s3-auth-method` is `iam`
s3-access-key-id: aws-or-minio-key-id
s3-secret-access-key: aws-or-minio-secret-key

## Optional paths for both the product and the associated stemcell
## defaults to the root path of the specified bucket
# s3-product-path: /path/to/product
# s3-stemcell-path: /path/to/stemcell

## defaults to false and should be excluded if not set to true
# s3-disable-ssl: true

## defaults to false;
## made available only because sometimes necessary for compatibility
# s3-enable-v2-signing: true

## defaults to accesskey;
## allows use of AWS instance IAM creds, if available
# s3-auth-type: iam