VMware Spring Cloud® Data Flow for Kubernetes

This topic provides an overview of VMware Spring Cloud® Data Flow for Kubernetes.

About Spring Cloud Data Flow

Spring Cloud Data Flow is an open-source toolkit that can deploy streaming and batch data pipelines on Kubernetes (see the project home page). The data pipelines are composed of Spring Cloud Stream or Spring Cloud Task applications. A streaming pipeline DSL makes it easy to specify which apps to deploy and how to connect outputs and inputs. The composed task DSL is useful for when a series of task apps must be run as a directed graph.

Spring Cloud Data Flow provides REST APIs for composing and deploying data pipelines programmatically. It also provides a dashboard that includes a graphical pipeline editor for building new pipelines interactively, as well as views of deployable apps and running apps with metrics using Prometheus, InfluxDB, or other monitoring systems. Spring Cloud Data Flow also includes a shell application for working with the API from the command line.

About Spring Cloud Data Flow for Kubernetes

VMware Spring Cloud® Data Flow for Kubernetes (SCDF for Kubernetes) is based on the Spring Cloud Data Flow open source project. SCDF for Kubernetes orchestrates the deployment lifecycle of streaming and batch data pipelines deployed on Kubernetes. Data pipelines are composed of Spring Cloud Stream or Spring Cloud Task applications. You can use the streaming pipeline DSL and UI to specify which apps to deploy and how to connect outputs and inputs. When running a series of task applications as a directed graph, you can use the composed task DSL and UI.

Product Snapshot

The following table provides version and version-support information about VMware Spring Cloud® Data Flow for Kubernetes. The table also provides information about the OSS versions used.

Element Details
Version v1.2.2
Release date May 12, 2021
Software component versions:
  Spring Cloud Data Flow PRO v1.2.2
  Spring Cloud Skipper OSS v2.6.2
  Composed Task Runner OSS v2.7.2

Features

  • Operational simplicity for provisioning Spring Cloud Data Flow OSS and other ecosystem projects on Kubernetes
  • Certified container images for Spring Cloud Data Flow Pro, Spring Cloud Skipper, and Composed Task Runner
  • Image-relocation capabilities for air-gapped deployments
  • Support for using the relational databases, message brokers, and identity providers of your choice
  • Easily explore the provisioned software components in a development environment before promoting to production

  • Multi-IO stream support for constructing streaming pipelines using applications that have multiple inputs and outputs

  • Container registry updates feature for registered stream and task apps

Limitations

The following section lists limitations of VMware Spring Cloud® Data Flow for Kubernetes.

General Limitations

  • This product is compatible with the following Kubernetes release versions:

    • v1.20.x
    • v1.19.x
    • v1.18.x
    • v1.17.x
    • v1.16.x
  • This product is intended to be used with any Kubernetes distribution. However, given the number of Kubernetes vendors and configurations, not all of them have been tested with SCDF for Kubernetes.

Getting Started

To set up the dependencies required for installation, see Preparing to Install VMware Spring Cloud® Data Flow for Kubernetes.