Provision and Bind New Kafka Service

Stark & Wayne kStreams allows all Pivotal Cloud Foundry users to create and use Kafka within their applications. Applications can then use any Kafka library to publish or subscribe to the Kafka topic.

Available Service Plans

The Stark & Wayne kStreams tile provides two service plans:

  • topic plan
  • shared plan

Both give access to a shared Apache Kafka/ZooKeeper cluster. The former pre-creates a topic that all bound applications can discover (via the topicName credential) and begin using.

The latter assumes that the bound applications will create their own topics and have their own method for sharing which topics each application will use. This is an advanced case used by Spring Cloud DataFlow.

Provision and Bind

Create a new Kafka topic using the cf create-service (or cf cs alias):

cf create-service starkandwayne-kstreams topic myapp-kafka

You can share the Kafka topic with one or more applications within the same Cloud Foundry “space”.

cf bind-service myapp myapp-kafka

Applications have access to the connection information when they are next pushed, restarted, or restaged.

cf restart myapp

Credentials

The Kafka connection credentials pass to your application via environment variables, specifically $VCAP_SERVICES.

Either use a JSON parser or find a language-specific community Cloud Foundry library to make navigating this environment variable easy.

See the Language Examples for samples of looking up/using the credentials.

The credentials section contains the following keys:

json
{
  "hostname": "10.213.10.35:9092,10.213.10.36:9092,10.213.10.34:9092",
  "topicName": "67f85c81-ee6f-4791-b169-864f2e18d929",
  "uri": "kafka://10.213.10.35:9092,10.213.10.36:9092,10.213.10.34:9092/67f85c81-ee6f-4791-b169-864f2e18d929",
  "zkPeers": "10.213.10.31:2181,10.213.10.32:2181,10.213.10.33:2181"
}

Delete

When you no longer want your Kafka topic, delete it with the following command:

cf delete-service myapp-kafka
Create a pull request or raise an issue on the source for this page in GitHub