VMware Spring Cloud® Data Flow for Kubernetes
This topic provides an overview of VMware Spring Cloud® Data Flow for Kubernetes.
Spring Cloud Data Flow is an open-source toolkit that can deploy streaming and batch data pipelines on Kubernetes (see the project home page). The data pipelines are composed of Spring Cloud Stream or Spring Cloud Task applications. A streaming pipeline DSL makes it easy to specify which apps to deploy and how to connect outputs and inputs. The composed task DSL is useful for when a series of task apps must be run as a directed graph.
Spring Cloud Data Flow provides REST APIs for composing and deploying data pipelines programmatically. It also provides a dashboard that includes a graphical pipeline editor for building new pipelines interactively, as well as views of deployable apps and running apps with metrics using Prometheus, InfluxDB, or other monitoring systems. Spring Cloud Data Flow also includes a shell application for working with the API from the command line.
VMware Spring Cloud® Data Flow for Kubernetes (SCDF for Kubernetes) is based on the Spring Cloud Data Flow open source project. SCDF for Kubernetes orchestrates the deployment lifecycle of streaming and batch data pipelines deployed on Kubernetes. Data pipelines are composed of Spring Cloud Stream or Spring Cloud Task applications. You can use the streaming pipeline DSL and UI to specify which apps to deploy and how to connect outputs and inputs. When running a series of task applications as a directed graph, you can use the composed task DSL and UI.
The following table provides version and version-support information about VMware Spring Cloud® Data Flow for Kubernetes. The table also provides information about the OSS versions used.
|Release date||May 12, 2021|
|Software component versions:|
|Spring Cloud Data Flow PRO||v1.2.2|
|Spring Cloud Skipper OSS||v2.6.2|
|Composed Task Runner OSS||v2.7.2|
- Operational simplicity for provisioning Spring Cloud Data Flow OSS and other ecosystem projects on Kubernetes
- Certified container images for Spring Cloud Data Flow Pro, Spring Cloud Skipper, and Composed Task Runner
- Image-relocation capabilities for air-gapped deployments
- Support for using the relational databases, message brokers, and identity providers of your choice
Easily explore the provisioned software components in a development environment before promoting to production
Multi-IO stream support for constructing streaming pipelines using applications that have multiple inputs and outputs
Container registry updates feature for registered stream and task apps
The following section lists limitations of VMware Spring Cloud® Data Flow for Kubernetes.
This product is compatible with the following Kubernetes release versions:
This product is intended to be used with any Kubernetes distribution. However, given the number of Kubernetes vendors and configurations, not all of them have been tested with SCDF for Kubernetes.
To set up the dependencies required for installation, see Preparing to Install VMware Spring Cloud® Data Flow for Kubernetes.