PCF v1.8 Partners Release Notice
Note: In September 2016, prior to the release of Pivotal Cloud Foundry (PCF) 1.8, a version of this notice was emailed to PCF partners.
PCF v1.8 latest release candidates (RCs) for Ops Manager and Elastic Runtime (OM RC4 and ERT RC7) have been released. You should have access now.
We’d also like to remind you about the importance of metadata versioning and having compatible versions of your tile with the different PCF versions.
Lastly, our IPsec Add-on is now supported on ERT and our internal service tiles. To ensure that your service works with IPsec, you should test and verify. For more information, see Supporting the IPsec Add-On below.
If you haven’t done so already with earlier RCs, use your tiles with the current RCs and fix any issues you find. We expect these latest RCs to be our GA candidates. To ensure that your tile works with PCF v1.8, please check for three specific things:
Most critically, we removed the double-paren az-accessor in Ops Manager. For a high-level explanation, see Changes in Ops Manager v1.8.0 for Tile Authors in the Pivotal Cloud Foundry Ops Manager v1.8 Release Notes. For additional details and example code, refer to the AZ Accessor Changes section below.
The second issue is an inconsistency between Ops Manager 1.7 and 1.8 in how they parse one particular syntax for selectors. It’s unlikely that this inconsistency will affect you, but you should double-check your tile for this usage. For general information about how 1.8 tiles use the
selectorproperty, see Selector in the About PCF Tiles topic. For a full rundown, with code examples, of how Ops Manager 1.7 and 1.8 interpret specific selector syntax differently, refer to the Selector Inconsistencies section below.
Lastly, to make sure your tile takes advantage of all the benefits of PCF v1.8, change its
v1.8. Changing the version number can unlock new properties and also break properties that changed from previous versions. For more information, see the Product Template Reference topic.
Note: Changing the value of
single_az_only for jobs launched by your tile can cause data loss for customers who upgrade to Ops Manager v1.8 versions older than v1.8.12. Contact Pivotal Support for help avoiding this.
For general information about the PCF v1.8 release, see the Pivotal Cloud Foundry Release Notes.
As part of our latest release process, we reviewed our documentation for tile authors and made some changes. With the v1.8 release, we now have an About PCF Tiles topic that we’ll keep up-to-date with the changes in Ops Manager that affect tile authors. In addition, we now let partners access our documentation staging area so that they can preview changes directly and provide feedback. You can see the details, including some code snippet examples on the Changes in Ops Manager v1.8.0 for Tile Authors section of the Ops Manager Release notes.
Reach out to your Pivotal contact for help with any questions or issues with PCF v1.8.
As of PCF v1.8, we certify that all of our internal service tiles except RabbitMQ work with our IPsec Add-on to provide a better security experience for our customers.
Services that support Transport Layer Security (TLS) do not necessarily need to support IPsec. This is a question of addressable market.
Operators may choose to deploy services to a non-IPsec subnet. However, based on any individual customer’s unique constraints or policy preferences, services may also be expected to be deployable to the same subnet as ERT, and the IPsec Add-on may be present there for the benefit it brings to the ERT. In this case, your service may be expected to operate in an IPsec environment. By testing that your service operates in a deployment where IPsec is present, you ensure the maximum compatibility with potential customer deployment choices.
Here are three configurations to test with your service:
|ERT Subnet||Services Subnet|
|Uses IPsec||Does not use IPsec|
|Uses IPsec||Uses IPsec|
|Uses IPsec||Shares the same subnet as ERT (which uses IPsec)|
PCF tiles are configured and deployed by Ops Manager. Tile authors build BOSH releases, Ops Manager metadata, and upgrade migrations.
You can read more about how to build a PCF tile here.
Ops Manager metadata renders forms and substitutes user-entered values into a generated BOSH manifest. Here’s an excerpt from the PCF example tile, which deploys an Nginx webserver and prints properties as static text.
form_types: - name: example-form label: Configurable Properties description: All the properties that you can configure! markdown: | ## I am markdown, hear me _roar_. ![Alt text](http://placekitten.com/g/400/200) Things to do: 1. Learn [markdown](https://daringfireball.net/projects/markdown/). 1. ... 1. Profit! property_inputs: - reference: .web_server.example_string label: Example string
This creates a form in the example tile.
Farther down in the metadata is a definition of the property
- name: web_server resource_label: Web Server templates: - name: web_server release: example-release ￼￼￼￼ property_blueprints: - name: example_string type: string configurable: true default: 'Hello world' constraints: - must_match_regex: '\A[^!@#$%^&*()]*\z' error_message: 'Cannot contain special characters.' - must_match_regex: '\A[^0-9]*\z' error_message: 'Cannot contain digits.'
Finally, this value is evaluated in the BOSH manifest:
manifest: | configured: string: (( example_string.value ))
The variable in this manifest section
(( example_string.value )) is called an accessor.
When a user clicks the Apply button in Ops Manager, the accessor is replaced with the value that a user entered. If a user types “foo” into the form, the BOSH manifest would look like this:
--- name: example-product director_uuid: 503e2ec5-a272-40b0-a27d-00452240024f ... - name: web_server-partition-da395ea0213891b86553 templates: - name: web_server release: example-release ... properties: configured: string: foo
Balancing Jobs across AZs
Implementation in Ops Manager 1.7
Ops Manager 1.7 releases named each job according to the zone it is placed in. You can see a list of these jobs in the code snippet below. The long GUID at the end of each job name (8d8d7ae8534f0a743a7e) is an internal reference for Ops Manager to keep track of the AZ placements.
| VM | web_server-partition-8d8d7ae8534f0a743a7e/0 | web_server-partition-8d8d7ae8534f0a743a7e/1 | IPs | | 10.0.32.11 | | 10.0.32.12 | ￼￼￼￼ | web_server-partition-da395ea0213891b86553/0 | 10.0.16.25 | | web_server-partition-da395ea0213891b86553/1 | 10.0.16.26 |
To BOSH, these are actually different jobs, because they have different names, and are indexed (
Here’s the problem: an operator would expect these to be the same jobs. They are all instances of the single web server job. But since BOSH once had no way of placing the jobs, Ops Manager compensated with the reference in the name. Ops Manager does present the web server as the same job by hiding the complexity underneath.
Implementation in Ops Manager 1.8
For the Ops Manager / BOSH split, these jobs are named logically. In our example, both BOSH and Ops Manager use the name
web_server. This naming is possible because of new BOSH AZ features. Jobs no longer require the GUID.
The fate of AZ accessors
In version 1.7 there is a handy accessor to print the availability zone names into the manifest. An example of where this is used is the CF metron agent, which sends metrics to doppler endpoints in the same availability zone as the job instance. The manifest template from the Pivotal Elastic Runtime tile uses this accessor as follows:
manifest: | ... metron_agent: zone: (( availability_zone ))
While this accessor worked in Ops Manager 1.7, it is unavailable in version 1.8 due to BOSH becoming the decision maker for instance placement.
Instead of using the accessor, tile authors should use a BOSH ERB variable in the BOSH release’s spec file. As stated in that doc, the spec file can include
spec.az which will be replaced at deploy time by the instance’s AZ name.
There is an additional accessor defined in the PCF Partner docs:
bosh_job_partition_stats. This accessor returns info about the instance’s availability zone placement. Since that information is outside of Ops Manager’s purview, this accessor will also be unavailable in Ops Manager 1.8.
The Bottom Line
A tile that uses
spec.az in its release will not work in 1.7. But the tile must use
spec.az in 1.8 since the accessor is gone. So what does the upgrade flow look like?
Here are the steps a user must go through:
User exports the Ops Manager 1.7 installation with tiles that support the old accessors. 2. User imports the installation into 1.8 Ops Manager.
User upgrades tiles to those that do not use the accessor.
User clicks “Apply Changes.”
If the user clicks “Apply Changes” without upgrading their tiles to use
spec.az, they will see some error about incompatible tiles requiring upgrades.
While this is not a source of happiness for tile authors, it is a step towards a brighter future. It is also a step towards human-readable BOSH manifests. Moving properties out of the manifest and into the release will meet that standard.
There are some inconsistencies between particular accessors in 1.7 vs. 1.8.
A selector is an Ops Manager property input that displays different sets of options beneath radio buttons. Operators use selectors to make small decisions within the context of a larger one. Here’s a selector from the PCF Example Tile:
Using a Selector
Tile authors can use selectors by creating forms with this type of property input. The following code
creates the food choice selector above. Notice that the
different options like
pizza_option and nested sub-options like
- name: example_selector_form label: 'Selector Example' description: 'A selector example form' property_inputs: - reference: .properties.example_selector label: 'Food Choices' selector_property_inputs: - reference: .properties.example_selector.pizza_option label: 'Pizza' property_inputs: - reference: .properties.example_selector.pizza_option.pepperoni label: 'Add Pepperoni' - reference: .properties.example_selector.pizza_option.pineapple label: 'Add Pineapple' description: 'Nothing like an apple, would not buy again' - reference: .properties.example_selector.pizza_option.other_toppings label: 'Other toppings' - reference: .properties.example_selector.filet_mignon_option label: 'Filet Mignon' property_inputs: - reference: .properties.example_selector.filet_mignon_option.rarity_dropdown label: 'How rare?' description: 'Extinct.' - reference: .properties.example_selector.filet_mignon_option.review label: 'Write your review' description: '"Contained live jaguar, would not buy again."' - reference: .properties.example_selector.filet_mignon_option.secret_sauce label: 'Secret Sauce' description: 'Probably mayonnaise' - reference: .properties.example_selector.beverage_option label: 'Beverage' property_inputs: - reference: .properties.example_selector.beverage_option.cola label: 'What kind of soda?'
The selector definition referenced in the code above is defined farther down in the metadata file. The selector has various nested
property_blueprints: - name: example_selector type: selector configurable: true default: Pizza option_templates: - name: pizza_option select_value: Pizza named_manifests: - name: my_snippet manifest: | pizza_toppings: pepperoni: (( .properties.example_selector.pizza_option.pepperoni.value )) pineapple: (( .properties.example_selector.pizza_option.pineapple.value )) other: (( .properties.example_selector.pizza_option.other_toppings.value )) - name: provides_section manifest: | as: 'pizza_link_web_server_job' - name: consumes_section manifest: | from: 'pizza_link_web_server_job' property_blueprints: - name: pepperoni type: boolean configurable: true - name: pineapple type: boolean configurable: true - name: other_toppings type: string configurable: true optional: true constraints: - must_match_regex: '\A[^!@#$%^&*()]*\z' error_message: 'This name cannot contain special characters.' - name: filet_mignon_option select_value: Filet Mignon named_manifests: - name: my_snippet manifest: | rarity: (( .properties.example_selector.filet_mignon_option.rarity_dropdown.value )) review: (( .properties.example_selector.filet_mignon_option.review.value )) secret_sauce: (( .properties.example_selector.filet_mignon_option.secret_sauce.value )) - name: provides_section manifest: | as: 'filet_mignon_link_web_server_job' - name: consumes_section manifest: | from: 'filet_mignon_link_web_server_job' property_blueprints: - name: rarity_dropdown type: dropdown_select configurable: true default: rare options: - name: rare label: 'Rare' - name: medium label: 'Medium' - name: well-done label: 'Well done' - name: review type: string configurable: true default: A+++++ power seller of mail order steak optional: false - name: secret_sauce type: secret configurable: true optional: true - name: beverage_option select_value: Beverage property_blueprints: - name: cola label: Cola type: string configurable: true optional: true named_manifests: - name: my_snippet manifest: | beverage: (( .properties.example_selector.beverage_option.cola.value )) - name: provides_section manifest: | as: 'beverage_link_web_server_job' - name: consumes_section manifest: | from: 'beverage_link_web_server_job'
Accessors print values into a BOSH manifest. They are used in the
manifest snippets of the [job_types](section. In the case of selectors, though, there are also manifest snippets in the property blueprint that defines the selector.
The reason there are manifest snippets in the definitions of a selector is because the final manifest snippet in will be different based on which option the user selects.
The final snippet in the
job_types section looks like this:
selector: (( .properties.example_selector.selected_option.parsed_manifest(my_snippet) ))
The Three-Stage Journey of Manifest Output
A selector’s journey from user input to manifest output has three stages:
- The selector is placed into a form.
form_types food form property inputs reference: example_selector property input reference: example_selector.pizza_option property input reference: example_selector.pizza_option.pepperoni reference: example_selector.pizza_option.pineapple
- The selector and options are defined including all of manifest snippets that could be generated.
property_blueprints example_selector option_templates pizza_option my_snippet manifest: | pizza_toppings: pepperoni: (( .properties.example_selector.pizza_option.pepperoni.value )) pineapple: (( .properties.example_selector.pizza_option.pineapple.value ))
- The final manifest snippet is placed into a job definition.
job_types some bosh job selector: (( .properties.example_selector.selected_option.parsed_manifest(my_snippet) ))
Inconsistencies between 1.7 and 1.8
Ops Manager version 1.7 and 1.8 have different behavior in how the accessor in stage 2 above is evaluated.
In Ops Manager version 1.7, the values in stage 2 will be
nil if the user chooses a different radio group,
such as the Filet Mignon group, rather than the pizza group we are using in the example.
In Ops Manager version 1.8, the values in stage 2 will be the value the user chose, or the default value.
pizza_option_pineapple.value would be
pineapple even if the user chose Filet Mignon, and not Pizza.
Major Edge Case
This inconsistency is an edge case. In the tiles we checked, there would be no weird behavior. The problem will only surface if the stage 2 accessor that references the value is used in stage 3 of the journey, which seems unlikely.
So if you use the selector syntax in stage 3, which uses
parsed_manifest then your tile would behave
the exact same way in versions 1.7, and 1.8. There would be no problem.
But if, for some reason, your tile uses the stage 2 accessors
(( option.value )) in stage 3 job_type
manifest snippets, then your tile may behave unpredictably.