PCF v1.8 Partners Release Notice

Note: In September 2016, prior to the release of Pivotal Cloud Foundry (PCF) 1.8, a version of this notice was emailed to PCF partners.

PCF v1.8 latest release candidates (RCs) for Ops Manager and Elastic Runtime (OM RC4 and ERT RC7) have been released. You should have access now.

We’d also like to remind you about the importance of metadata versioning and having compatible versions of your tile with the different PCF versions.

Lastly, our IPsec Add-on is now supported on ERT and our internal service tiles. To ensure that your service works with IPsec, you should test and verify. For more information, see Supporting the IPsec Add-On below.

Test Your Tile on PCF v1.8

If you haven’t done so already with earlier RCs, use your tiles with the current RCs and fix any issues you find. We expect these latest RCs to be our GA candidates. To ensure that your tile works with PCF v1.8, please check for three specific things:

  • Most critically, we removed the double-paren az-accessor in Ops Manager. For a high-level explanation, see Changes in Ops Manager v1.8.0 for Tile Authors in the Pivotal Cloud Foundry Ops Manager v1.8 Release Notes. For additional details and example code, refer to the AZ Accessor Changes section below.

  • The second issue is an inconsistency between Ops Manager 1.7 and 1.8 in how they parse one particular syntax for selectors. It’s unlikely that this inconsistency will affect you, but you should double-check your tile for this usage. For general information about how 1.8 tiles use the selector property, see Selector in the About PCF Tiles topic. For a full rundown, with code examples, of how Ops Manager 1.7 and 1.8 interpret specific selector syntax differently, refer to the Selector Inconsistencies section below.

  • Lastly, to make sure your tile takes advantage of all the benefits of PCF v1.8, change its metadata_version property to v1.8. Changing the version number can unlock new properties and also break properties that changed from previous versions. For more information, see the Product Template Reference topic.

Note: Changing the value ofsingle_az_only for jobs launched by your tile can cause data loss for customers who upgrade to Ops Manager v1.8 versions older than v1.8.12. Contact Pivotal Support for help avoiding this.

For general information about the PCF v1.8 release, see the Pivotal Cloud Foundry Release Notes.

As part of our latest release process, we reviewed our documentation for tile authors and made some changes. With the v1.8 release, we now have an About PCF Tiles topic that we’ll keep up-to-date with the changes in Ops Manager that affect tile authors. In addition, we now let partners access our documentation staging area so that they can preview changes directly and provide feedback. You can see the details, including some code snippet examples on the Changes in Ops Manager v1.8.0 for Tile Authors section of the Ops Manager Release notes.

Reach out to your Pivotal contact for help with any questions or issues with PCF v1.8.


Pivotal Cloud Foundry 1.7 Partners Release Notice

Supporting the IPsec Add-On

As of PCF v1.8, we certify that all of our internal service tiles except RabbitMQ work with our IPsec Add-on to provide a better security experience for our customers.

Services that support Transport Layer Security (TLS) do not necessarily need to support IPsec. This is a question of addressable market.

Operators may choose to deploy services to a non-IPsec subnet. However, based on any individual customer’s unique constraints or policy preferences, services may also be expected to be deployable to the same subnet as ERT, and the IPsec Add-on may be present there for the benefit it brings to the ERT. In this case, your service may be expected to operate in an IPsec environment. By testing that your service operates in a deployment where IPsec is present, you ensure the maximum compatibility with potential customer deployment choices.

Here are three configurations to test with your service:

ERT SubnetServices Subnet
Uses IPsec Does not use IPsec
Uses IPsec Uses IPsec
Uses IPsec Shares the same subnet as ERT (which uses IPsec)

AZ Accessor Changes

PCF tiles are configured and deployed by Ops Manager. Tile authors build BOSH releases, Ops Manager metadata, and upgrade migrations.

You can read more about how to build a PCF tile here.

Ops Manager metadata renders forms and substitutes user-entered values into a generated BOSH manifest. Here’s an excerpt from the PCF example tile, which deploys an Nginx webserver and prints properties as static text.

form_types:
  - name: example-form
    label: Configurable Properties
    description: All the properties that you can configure!
    markdown: |
      ## I am markdown, hear me _roar_.
      ![Alt text](http://placekitten.com/g/400/200)
      Things to do:
      1. Learn [markdown](https://daringfireball.net/projects/markdown/).
      1. ...
      1. Profit!
    property_inputs:
      - reference: .web_server.example_string
        label: Example string

This creates a form in the example tile.

Farther down in the metadata is a definition of the property web_server.example_string:

- name: web_server
    resource_label: Web Server
    templates:
      - name: web_server
        release: example-release

property_blueprints:
    - name: example_string
type: string
configurable: true
default: 'Hello world'
constraints:
- must_match_regex: '\A[^!@#$%^&*()]*\z'
        error_message: 'Cannot contain special characters.'
      - must_match_regex: '\A[^0-9]*\z'
        error_message: 'Cannot contain digits.'

Finally, this value is evaluated in the BOSH manifest:

manifest: |
     configured:
string: (( example_string.value ))

The variable in this manifest section (( example_string.value )) is called an accessor.

When a user clicks the Apply button in Ops Manager, the accessor is replaced with the value that a user entered. If a user types “foo” into the form, the BOSH manifest would look like this:

---
name: example-product
director_uuid: 503e2ec5-a272-40b0-a27d-00452240024f
...
- name: web_server-partition-da395ea0213891b86553
  templates:
  - name: web_server
    release: example-release
...
  properties:
    configured:
string: foo

Balancing Jobs across AZs

Implementation in Ops Manager 1.7

Ops Manager 1.7 releases named each job according to the zone it is placed in. You can see a list of these jobs in the code snippet below. The long GUID at the end of each job name (8d8d7ae8534f0a743a7e) is an internal reference for Ops Manager to keep track of the AZ placements.

| VM
| web_server-partition-8d8d7ae8534f0a743a7e/0
| web_server-partition-8d8d7ae8534f0a743a7e/1
      | IPs        |
| 10.0.32.11 |
| 10.0.32.12 |

| web_server-partition-da395ea0213891b86553/0      | 10.0.16.25 |
| web_server-partition-da395ea0213891b86553/1      | 10.0.16.26 |

To BOSH, these are actually different jobs, because they have different names, and are indexed (/0,/1) separately.

Here’s the problem: an operator would expect these to be the same jobs. They are all instances of the single web server job. But since BOSH once had no way of placing the jobs, Ops Manager compensated with the reference in the name. Ops Manager does present the web server as the same job by hiding the complexity underneath.

Implementation in Ops Manager 1.8

For the Ops Manager / BOSH split, these jobs are named logically. In our example, both BOSH and Ops Manager use the name web_server. This naming is possible because of new BOSH AZ features. Jobs no longer require the GUID.

The fate of AZ accessors

In version 1.7 there is a handy accessor to print the availability zone names into the manifest. An example of where this is used is the CF metron agent, which sends metrics to doppler endpoints in the same availability zone as the job instance. The manifest template from the Pivotal Elastic Runtime tile uses this accessor as follows:

manifest: |
  ...
  metron_agent:
    zone: (( availability_zone ))

While this accessor worked in Ops Manager 1.7, it is unavailable in version 1.8 due to BOSH becoming the decision maker for instance placement.

Instead of using the accessor, tile authors should use a BOSH ERB variable in the BOSH release’s spec file. As stated in that doc, the spec file can include spec.az which will be replaced at deploy time by the instance’s AZ name.

Partition accessor

There is an additional accessor defined in the PCF Partner docs: bosh_job_partition_stats. This accessor returns info about the instance’s availability zone placement. Since that information is outside of Ops Manager’s purview, this accessor will also be unavailable in Ops Manager 1.8.

The Bottom Line

A tile that uses spec.az in its release will not work in 1.7. But the tile must use spec.az in 1.8 since the accessor is gone. So what does the upgrade flow look like?

Here are the steps a user must go through:

  1. User exports the Ops Manager 1.7 installation with tiles that support the old accessors. 2. User imports the installation into 1.8 Ops Manager.

  2. User upgrades tiles to those that do not use the accessor.

  3. User clicks “Apply Changes.”

If the user clicks “Apply Changes” without upgrading their tiles to use spec.az, they will see some error about incompatible tiles requiring upgrades.

While this is not a source of happiness for tile authors, it is a step towards a brighter future. It is also a step towards human-readable BOSH manifests. Moving properties out of the manifest and into the release will meet that standard.

Selector Inconsistencies

There are some inconsistencies between particular accessors in 1.7 vs. 1.8.

A selector is an Ops Manager property input that displays different sets of options beneath radio buttons. Operators use selectors to make small decisions within the context of a larger one. Here’s a selector from the PCF Example Tile:

Selector Example Form

Using a Selector

Tile authors can use selectors by creating forms with this type of property input. The following code creates the food choice selector above. Notice that the selector_property_input references different options like pizza_option and nested sub-options like pizza_option_pepperoni:

  - name: example_selector_form
    label: 'Selector Example'
    description: 'A selector example form'
    property_inputs:
      - reference: .properties.example_selector
        label: 'Food Choices'
        selector_property_inputs:
          - reference: .properties.example_selector.pizza_option
            label: 'Pizza'
            property_inputs:
              - reference: .properties.example_selector.pizza_option.pepperoni
                label: 'Add Pepperoni'
              - reference: .properties.example_selector.pizza_option.pineapple
                label: 'Add Pineapple'
                description: 'Nothing like an apple, would not buy again'
              - reference: .properties.example_selector.pizza_option.other_toppings
                label: 'Other toppings'
          - reference: .properties.example_selector.filet_mignon_option
            label: 'Filet Mignon'
            property_inputs:
              - reference: .properties.example_selector.filet_mignon_option.rarity_dropdown
                label: 'How rare?'
                description: 'Extinct.'
              - reference: .properties.example_selector.filet_mignon_option.review
                label: 'Write your review'
                description: '"Contained live jaguar, would not buy again."'
              - reference: .properties.example_selector.filet_mignon_option.secret_sauce
                label: 'Secret Sauce'
                description: 'Probably mayonnaise'
          - reference: .properties.example_selector.beverage_option
            label: 'Beverage'
            property_inputs:
              - reference: .properties.example_selector.beverage_option.cola
                label: 'What kind of soda?'

The selector definition referenced in the code above is defined farther down in the metadata file. The selector has various nested option_templates:

property_blueprints:
  - name: example_selector
    type: selector
    configurable: true
    default: Pizza
    option_templates:
      - name: pizza_option
        select_value: Pizza
        named_manifests:
          - name: my_snippet
            manifest: |
              pizza_toppings:
                pepperoni: (( .properties.example_selector.pizza_option.pepperoni.value ))
                pineapple: (( .properties.example_selector.pizza_option.pineapple.value ))
                other: (( .properties.example_selector.pizza_option.other_toppings.value ))
          - name: provides_section
            manifest: |
              as: 'pizza_link_web_server_job'
          - name: consumes_section
            manifest: |
              from: 'pizza_link_web_server_job'
        property_blueprints:
          - name: pepperoni
            type: boolean
            configurable: true
          - name: pineapple
            type: boolean
            configurable: true
          - name: other_toppings
            type: string
            configurable: true
            optional: true
            constraints:
            - must_match_regex: '\A[^!@#$%^&*()]*\z'
              error_message: 'This name cannot contain special characters.'
      - name: filet_mignon_option
        select_value: Filet Mignon
        named_manifests:
          - name: my_snippet
            manifest: |
              rarity: (( .properties.example_selector.filet_mignon_option.rarity_dropdown.value ))
              review: (( .properties.example_selector.filet_mignon_option.review.value ))
              secret_sauce: (( .properties.example_selector.filet_mignon_option.secret_sauce.value ))
          - name: provides_section
            manifest: |
              as: 'filet_mignon_link_web_server_job'
          - name: consumes_section
            manifest: |
              from: 'filet_mignon_link_web_server_job'
        property_blueprints:
          - name: rarity_dropdown
            type: dropdown_select
            configurable: true
            default: rare
            options:
              - name: rare
                label: 'Rare'
              - name: medium
                label: 'Medium'
              - name: well-done
                label: 'Well done'
          - name: review
            type: string
            configurable: true
            default: A+++++ power seller of mail order steak
            optional: false
          - name: secret_sauce
            type: secret
            configurable: true
            optional: true
      - name: beverage_option
        select_value: Beverage
        property_blueprints:
          - name: cola
            label: Cola
            type: string
            configurable: true
            optional: true
        named_manifests:
          - name: my_snippet
            manifest: |
              beverage: (( .properties.example_selector.beverage_option.cola.value ))
          - name: provides_section
            manifest: |
              as: 'beverage_link_web_server_job'
          - name: consumes_section
            manifest: |
              from: 'beverage_link_web_server_job'

Accessors print values into a BOSH manifest. They are used in the manifest snippets of the [job_types](section. In the case of selectors, though, there are also manifest snippets in the property blueprint that defines the selector.

The reason there are manifest snippets in the definitions of a selector is because the final manifest snippet in will be different based on which option the user selects.

The final snippet in the job_types section looks like this:

 selector: ((
 .properties.example_selector.selected_option.parsed_manifest(my_snippet) ))

The Three-Stage Journey of Manifest Output

A selector’s journey from user input to manifest output has three stages:

  1. The selector is placed into a form.
form_types
      food form
        property inputs
          reference: example_selector
            property input
            reference: example_selector.pizza_option
              property input
                reference: example_selector.pizza_option.pepperoni
                reference: example_selector.pizza_option.pineapple
  1. The selector and options are defined including all of manifest snippets that could be generated.
property_blueprints
    example_selector
      option_templates
        pizza_option
          my_snippet
            manifest: |
              pizza_toppings:
                pepperoni: (( .properties.example_selector.pizza_option.pepperoni.value ))
                pineapple: (( .properties.example_selector.pizza_option.pineapple.value ))
  1. The final manifest snippet is placed into a job definition.
job_types
    some bosh job
      selector: (( .properties.example_selector.selected_option.parsed_manifest(my_snippet) ))

Inconsistencies between 1.7 and 1.8

Ops Manager version 1.7 and 1.8 have different behavior in how the accessor in stage 2 above is evaluated.

In Ops Manager version 1.7, the values in stage 2 will be nil if the user chooses a different radio group, such as the Filet Mignon group, rather than the pizza group we are using in the example.

In Ops Manager version 1.8, the values in stage 2 will be the value the user chose, or the default value. Thus pizza_option_pineapple.value would be pineapple even if the user chose Filet Mignon, and not Pizza.

Major Edge Case

This inconsistency is an edge case. In the tiles we checked, there would be no weird behavior. The problem will only surface if the stage 2 accessor that references the value is used in stage 3 of the journey, which seems unlikely.

So if you use the selector syntax in stage 3, which uses parsed_manifest then your tile would behave the exact same way in versions 1.7, and 1.8. There would be no problem.

But if, for some reason, your tile uses the stage 2 accessors (( option.value )) in stage 3 job_type manifest snippets, then your tile may behave unpredictably.

Create a pull request or raise an issue on the source for this page in GitHub