Pivotal Spring Cloud Data Flow

About Spring Cloud Data Flow

Spring Cloud Data Flow is an open-source toolkit that can deploy streaming and batch data pipelines to Cloud Foundry (see the project home page). The data pipelines are composed of Spring Cloud Stream or Spring Cloud Task applications. A streaming pipeline DSL makes it easy to specify which apps to deploy and how to connect outputs and inputs. The composed task DSL is useful for when a series of task apps must be run as a directed graph.

Spring Cloud Data Flow provides REST APIs for composing and deploying data pipelines programmatically. It also provides a dashboard that includes a graphical pipeline editor for building new pipelines interactively, as well as views of deployable apps and running apps with metrics using Prometheus, InfluxDB, or other monitoring systems. Spring Cloud Data Flow also includes a shell application for working with the API from the command line.

About Pivotal Spring Cloud Data Flow

Pivotal Spring Cloud Data Flow automates the deployment of SCDF and its dependent services so that developers can use Apps Manager to deploy their own SCDF instances. They can then access their Data Flow server, either via the SCDF dashboard UI or using their PCF credentials with the SCDF shell.

Key Features

Pivotal Spring Cloud Data Flow includes the following key features:

  • Addition of the Spring Cloud Data Flow server to the Marketplace as a managed service
  • Automatic integration with dependent PCF services:
    • MySQL for PCF (or a user-provided relational database service) for apps, pipelines, and task history
    • RabbitMQ for PCF (or a user-provided RabbitMQ or Kafka service) for event messaging
  • Integration of Data Flow server with PCF’s UAA security model

Product Snapshot

The following table provides version and version-support information about Pivotal Spring Cloud Data Flow.

Element Details
Version v1.4.0
Release date April 24, 2019
Software component version v2.0.1.RELEASE
Compatible Ops Manager version(s) v2.3.x–v2.4.x
Compatible Pivotal Application Service version(s) v2.3.x–v2.4.x
IaaS support All supported by PCF

Tile and OSS Versions

The following table provides information about the OSS versions used in each version of Pivotal Spring Cloud Data Flow.

SCDF Tile SCDF OSS Spring Cloud Skipper OSS
v1.2.x v1.6.x.RELEASE v1.0.x.RELEASE
v1.3.x v1.7.x.RELEASE v1.1.x.RELEASE
v1.4.x v2.0.x.RELEASE v2.0.x.RELEASE

Requirements

Pivotal Spring Cloud Data Flow has the following requirements:

Optional

To use the Spring Cloud Data Flow shell interface with Pivotal Spring Cloud Data Flow service instances, install the following cf CLI plugins:

Feedback

Please provide any bugs, feature requests, or questions to the Pivotal Cloud Foundry Feedback list.