Installing and Configuring Pivotal Spring Cloud Data Flow

Follow the below steps to install Spring Cloud Data Flow for Pivotal Cloud Foundry (PCF).

Prerequisites

Pivotal Spring Cloud Data Flow is built using Spring Boot 1.5, which requires version 3.8 or later of the Java Cloud Foundry buildpack. The default Java buildpack—the buildpack at the lowest position of all Java buildpacks—on your PCF installation must therefore be at version 3.8 or later.

You can use the Cloud Foundry Command Line Interface tool (cf CLI) to see the version of the Java buildpack that is currently installed.

$ cf buildpacks
Getting buildpacks...

buildpack                position   enabled   locked   filename
java_buildpack_offline   1          true      false    java-buildpack-offline-v3.8.1.zip
ruby_buildpack           2          true      false    ruby_buildpack-cached-v1.6.19.zip
nodejs_buildpack         3          true      false    nodejs_buildpack-cached-v1.5.15.zip
go_buildpack             4          true      false    go_buildpack-cached-v1.7.10.zip

If the default Java buildpack is older than version 3.8, you can download a newer version from Pivotal Network and update Pivotal Cloud Foundry by following the instructions in the Managing Custom Buildpacks topic. To ensure that the newer buildpack is the default Java buildpack, you may delete or disable the older buildpack or make sure that the newer buildpack is in a lower position.

Dependent Services

Spring Cloud Data Flow relies on other PCF services for its service broker, for the Data Flow server created for each service instance, and for the Spring Cloud Skipper application created for each service instance. You can configure these in the tile settings. When following the installation steps below, review all of these settings, ensuring that you have previously installed the specified services and that the specified service plans are active.

Important: If any of the services or service plans configured in the Data Flow tile settings are not available in your PCF deployment, the tile installation may fail.

Using TLS with MySQL for PCF v2.4.x

If you have configured Pivotal Spring Cloud Data Flow to use MySQL for PCF v2.4.x for the Data Flow service broker’s backing MySQL database service instance and you are using TLS with MySQL for PCF, the installation of Spring Cloud Data Flow will fail. This is because a MySQL for PCF v2.4.x service instance cannot be created with TLS encryption enabled; instead, after creating the service instance, you must update it in order to enable TLS.

Pivotal recommends using MySQL for PCF v2.5.x with Pivotal Spring Cloud Data Flow. If you must use MySQL for PCF v2.4.x, follow the below steps to complete the installation of Spring Cloud Data Flow.

  1. Begin the installation as described in Installation Steps. The installation will fail on the “deploy-all” errand with a message similar to the following:

    cf bind-service p-dataflow-1.3.2-build.1 p-dataflow-db
    Binding service p-dataflow-db to app p-dataflow-1.3.2-build.1 in org system / space
        p-dataflow as system_services...
    Service broker error: This service instance can only be used with TLS enabled. In
        order to enable TLS for this service instance run "cf update-service
        SERVICE_INSTANCE -c '{"enable_tls": [HOSTNAME]}'"
    FAILED
    
  2. Following the procedure in the Enable TLS section of the MySQL for PCF documentation, enable TLS on the p-dataflow-db MySQL for PCF service instance, which is located in the system org and p-dataflow space.

  3. Restart the installation of Pivotal Spring Cloud Data Flow.

Installation Steps

  1. Download the product file from Pivotal Network.

  2. Navigate to the Ops Manager Installation Dashboard and click Import a Product to upload the product file.

  3. Under the Import a Product button, click + next to the version number of Pivotal Spring Cloud Data Flow. This adds the tile to your staging area.

  4. Click the newly added Spring Cloud Data Flow tile. In the Settings tab, click Assign AZs and Networks.

    Setting AZs and Network

    Select the availability zones for the tile to use. In the Network section, select the PAS (or ERT) network.

  5. In the Settings tab, click Service Broker.

    Setting Broker Relational Database Service

    Enter the relational database service name and plan name for the Spring Cloud Data Flow tile to use for storing its service broker’s service instance data.

    Setting Java Buildpack

    You can also configure the Java buildpack to use for deploying the broker and service instance applications (by default, the tile will use the highest-priority Java buildpack).

  6. In the Settings tab, click Data Flow Server.

    Setting Data Flow Server Relational Database Service

    Configure the default services and service plans used by the Spring Cloud Data Flow server that is deployed for each Data Flow service instance. These values can be overridden when creating a Data Flow service instance.

  7. Still in the Settings tab, click Skipper.

    Setting Skipper Relational Database Service

    Configure the relational database service name and plan used by the Spring Cloud Skipper application deployed for each Data Flow service instance.

  8. Still in the Settings tab, click Errands.

    Configuring Errands

    Spring Cloud Data Flow has four lifecycle errands. In PCF 2.2 and later, each errand can be set to always run (On) or to never run (Off). In PCF 2.1 and earlier, errands can also be set to run conditionally (When Changed). Pivotal recommends that all Spring Cloud Data Flow lifecycle errands be set to always run (On).

  9. Return to the Ops Manager Installation Dashboard.

  10. If you are using Ops Manager v2.3 or later, click Review Pending Changes. For more information about this Ops Manager page, see Reviewing Pending Product Changes.

  11. Click Apply Changes to install the Spring Cloud Data Flow tile.