Pivotal Cloud Cache for PCF v1.2

Pivotal Cloud Cache Developer Guide

This document describes how a Pivotal Cloud Foundry (PCF) app developer can choose a service plan, create and delete Pivotal Cloud Cache (PCC) service instances, and bind an app.

You must install the Cloud Foundry Command Line Interface (cf CLI) to run the commands in this topic.

Viewing All Plans Available for Pivotal Cloud Cache

Run cf marketplace -s p-cloudcache to view all plans available for PCC. The plan names displayed are configured by the operator on tile installation.

$ cf marketplace -s p-cloudcache

Getting service plan information for service p-cloudcache as admin...

service plan   description      free or paid
extra-small    Caching Plan 1   free
small          Caching Plan 2   free
medium         Caching Plan 3   free
large          Caching Plan 4   free
extra-large    Caching Plan 5   free

Creating a Pivotal Cloud Cache Service Instance

Run cf create-service p-cloudcache PLAN-NAME SERVICE-INSTANCE-NAME to create a service instance. Replace PLAN-NAME with the name from the list of available plans. Replace SERVICE-INSTANCE-NAME with a name of your choice. Use this name to refer to your service instance with other commands. Service instance names can include alpha-numeric characters, hyphens, and underscores.

$ cf create-service p-cloudcache extra-large my-cloudcache

Service instances are created asynchronously. Run the cf services command to view the current status of the service creation, and of other service instances in the current org and space:

$ cf services
Getting services in org my-org / space my-space as user...

name            service        plan    bound apps   last operation
my-cloudcache   p-cloudcache   small                create in progress

When completed, the status changes from create in progress to create succeeded.

Provide Optional Parameters

You can create a customized service instance by passing optional parameters to cf create-service using the -c flag. The -c flag accepts a valid JSON object containing service-specific configuration parameters, provided either in-line or in a file.

The PCC service broker supports the following parameters:

  • num_servers: An integer that specifies the number of server instances in the cluster. The minimum value is 4. The maximum and default values are configured by the operator.
  • new_size_percentage: An integer that specifies the percentage of the heap to allocate to young generation. This value must be between 5 and 83. By default, the new size is 2 GB or 10% of heap, whichever is smaller.

The following example creates the service with five service instances in the cluster:

$ cf create-service p-cloudcache small my-cloudcache -c '{"num_servers": 5}'

Enable Session State Caching with the Java Buildpack

When the service instance name is followed by the session-replication tag, the Java buildpack downloads all the required resources for session state caching. This feature is available in Java buildpack version 3.19 and higher, up to but not including version 4. It is then available again in version 4.3.

To enable session state caching, do one of the following:

  • When creating your service instance name, append it with the session-replication tag. For example, for the p-cloudcache service:

     $ cf create-service p-cloudcache my-service-instance -t session-replication
  • When updating your service instance name (for example, if the updated name is new-service-instance), append it with the session-replication tag:

    $ cf update-service new-service-instance -t session-replication
  • End the service instance name with the text -session-replication: my-service-instance-session-replication.

Enable Session State Caching Using Spring Session

Session state caching for apps using Spring Session is a beta feature. It uses pre-release versions of Spring Session libraries.

To use spring-session with PCC, follow the steps below:

  1. Make the following changes to the app:

    • Replace existing Spring Session @EnableXXXHttpSession annotation with @EnableGemFireHttpSession(maxInactiveIntervalInSeconds = N) where N is seconds.
    • Add the spring-session-data-geode and spring-data-geode dependencies to the build.
    • Add beans to the Spring app config.

    For more information, see the spring-session-data-gemfire-example repository.

  2. Create a region named ClusteredSpringSessions in gfsh using the cluster_operator credentials: create region --name=ClusteredSpringSessions --type=PARTITION_HEAP_LRU

Dev Plans

A dev plan is a type of service plan. You can use a dev plan for minimal development and testing. The plan’s locator and server are colocated on a single VM.

WARNING: All of the dev plan data is stored on a single VM. When your service instance is updated or upgraded, the data is wiped from the VM and the VM itself is recycled. Therefore, there is data loss when you perform an update or upgrade.

If your dev plan has been configured to create a dev region by default, PCC creates a sample region called example_partition_region of type PARTITION_REDUNDANT_HEAP_LRU. You can use this region for development and testing.

If the example_partition_region has not been created, it is probably because post-deploy scripts are not enabled for Ops Manager. For more information, see Default Data Region for the Dev Plan.

Updating a Pivotal Cloud Cache Service Instance

You can apply all optional parameters to an existing service instance using the cf update-service command. You can, for example, scale up a cluster by increasing the number of servers.

Previously specified optional parameters are persisted through subsequent updates. To return the service instance to default values, you must explicitly specify the defaults as optional parameters.

For example, if you create a service instance with five servers using a plan that has a default value of four servers:

$ cf create-service p-cloudcache small my-cloudcache -c '{"num_servers": 5}'

And you set the new_size_percentage to 50%:

$ cf update-service my-cloudcache -c '{"new_size_percentage": 50}'

Then the resulting service instance has 5 servers and new_size_percentage of 50% of heap.

Cluster Rebalancing

When updating a cluster to increase the number of servers, the available heap size is increased. When this happens, PCC automatically rebalances data in the cache to distribute data across the cluster.

This automatic rebalancing does not occur when a server leaves the cluster and later rejoins, for example when a VM is re-created, or network connectivity lost and restored. In this case, you must manually rebalance the cluster using the gfsh rebalance command while authenticated as a cluster operator.

Note: You must first connect with gfsh before you can use the rebalance command.

About Changes to the Service Plan

Your PCF operator can change details of the service plan available on the Marketplace. If your operator changes the default value of one of the optional parameters, this does not affect existing service instances.

However, if your operator changes the allowed values of one of the optional parameters, existing instances that exceed the new limits are not affected, but any subsequent service updates that change the optional parameter must adhere to the new limits.

For example, if the PCF operator changes the plan by decreasing the maximum value for num_servers, any future service updates must adhere to the new num_servers value limit.

You might see the following error message when attempting to update a service instance:

$ cf update-service  my-cloudcache -c '{"num_servers": 5}'
Updating service instance my-cloudcache as admin...
Server error, status code: 502, error code: 10001, message: Service broker error: Service cannot be updated at this time, please try again later or contact your operator for more information

This error message indicates that the operator has made an update to the plan used by this service instance. You must wait for the operator to apply plan changes to all service instances before you can make further service instance updates.

Accessing a Service Instance

After you have created a service instance, you can start accessing it. Usually, you set up cache regions before using your service instance from a deployed CF app. You can do this with the gfsh command line tool. To connect, you must set up a service key, see Create Service Keys below.

Create Service Keys

Service keys provide a way to access your service instance outside the scope of a deployed CF app. Run cf create-service-key SERVICE-INSTANCE-NAME KEY-NAME to create a service key. Replace SERVICE-INSTANCE-NAME with the name you chose for your service instance. Replace KEY-NAME with a name of your choice. You can use this name to refer to your service key with other commands.

$ cf create-service-key my-cloudcache my-service-key

Run cf service-key SERVICE-INSTANCE-NAME KEY-NAME to view the newly created service key.

$ cf service-key my-cloudcache my-service-key

The cf service-key returns output in the following format:

  "locators": [
  "urls": {
    "gfsh": "gfsh-url",
    "pulse": "pulse-url"
  "users": [
      "password": "developer-password",
      "username": "developer"
      "password": "cluster_operator-password",
      "username": "cluster_operator"

This returned structure contains the following information:

  • cluster_operator and developer credentials
  • gfsh-url: A URL usable to connect the gfsh client to the service instance
  • pulse-url: A URL usable to view the Pulse dashboard in a web browser, which allows monitoring of the service-instance status. You can use the developer credentials to authenticate.

The cluster_operator and developer roles provide access to different functions on the service instance.

You can use the cluster_operator credentials to administer the service instance. For example, you can use the cluster_operator credentials to define or delete cache regions.

You can use the developer credentials to perform create, retrieve, update, and delete (CRUD) data operations. You can also use the cluster_operator credentials to perform the same functions the developer credentials. As a best practice, you should use the role with the least amount of permissions necessary.

Connect with gfsh over HTTPS

Connecting over HTTPS uses the same certificate you use to secure traffic into ERT; that is, the certificate you use where your TLS termination occurs. Before you can connect, you must create a truststore.

Create a Truststore

To create a truststore, use the same certificate you used to configure TLS termination. We suggest using the keytool command line utility to create a truststore file.

  1. Locate the certificate you use to configure TLS termination.
  2. Using your certificate, run the keytool command.

    For example:

    $ keytool -import -alias ENV -file CERTIFICATE.CER -keystore TRUSTSTORE-FILE-PATH"


    • ENV is your system environment.
    • CERTIFICATE.CER is your certificate file.
    • TRUSTSTORE-FILE-PATH is the path to the location where you want to create the truststore file, including the name you want to give the file.
  3. When you run this command, you are prompted to enter a keystore password. Create a password and remember it!

  4. When prompted for the certificate details, enter yes to trust the certificate.

The following example shows how to run keytool and what the output looks like:

$ keytool -import -alias prod-ssl -file /tmp/loadbalancer.cer -keystore /tmp/truststore/prod.myTrustStore 
Enter keystore password:
Re-enter new password:
Owner: CN=*, OU=Cloud Foundry, O=Pivotal, L=New York, ST=New York, C=US
Issuer: CN=*, OU=Cloud Foundry, O=Pivotal, L=New York, ST=New York, C=US
Serial number: bd84912917b5b665
Valid from: Sat Jul 29 09:18:43 EDT 2017 until: Mon Apr 07 09:18:43 EDT 2031
Certificate fingerprints:
   MD5:  A9:17:B1:C9:6C:0A:F7:A3:56:51:6D:67:F8:3E:94:35
   SHA1: BA:DA:23:09:17:C0:DF:37:D9:6F:47:05:05:00:44:6B:24:A1:3D:77
   SHA256: A6:F3:4E:B8:FF:8F:72:92:0A:6D:55:6E:59:54:83:30:76:49:80:92:52:3D:91:4D:61:1C:A1:29:D3:BD:56:57
   Signature algorithm name: SHA256withRSA
   Version: 3


#1: ObjectId: Criticality=true

#2: ObjectId: Criticality=false
SubjectAlternativeName [
  DNSName: *
  DNSName: *
  DNSName: *
  DNSName: *
  DNSName: *
  DNSName: *

Trust this certificate? [no]:  yes
Certificate was added to keystore

Establish the Connection with HTTPS

After you have created the truststore, you can connect using HTTPS.

  1. Download the Pivotal GemFire ZIP archive from Pivotal Network.
  2. Unzip and locate the correct binary for your architecture. Use gfsh with Unix or gfsh.bat with Windows.
  3. Add the gfsh binary to your path.
  4. Set the JAVA_ARGS environment variable with the following command:

    export JAVA_ARGS=""

    Where: TRUSTSTORE-FILE-PATH is the path to the TrustStore file you created in Create a Truststore.

    For example,

    $ export JAVA_ARGS=""

  5. Run the gfsh command-line interface, and then issue a connect command that specifies an HTTPS URL of the form:

    connect --use-http=true --url=<HTTP-gfsh-URL>

Establish the Connection with HTTPS in a Development Environment

When working in a non-production, development environment, a developer may choose to work in a less secure manner by eliminating the truststore and SSL mutual authentication.

The steps to establish the gfsh connection become:

  1. Download the Pivotal GemFire ZIP archive from Pivotal Network.
  2. Unzip and locate the correct binary for your architecture. Use gfsh with Unix or gfsh.bat with Windows.
  3. Add the gfsh binary to your path.
  4. Run the gfsh command-line interface, and then issue a connect command that specifies an HTTPS URL of the form:

    connect --use-http=true --use-ssl --skip-ssl-validation=true
     --url=<HTTPS-gfsh-URL> --user=<cluster_operator_XXX>

Using Pivotal Cloud Cache

Create Regions with gfsh

After connecting with gfsh as a cluster_operator, you can define a new cache region.

The following command creates a partitioned region with a single redundant copy:

gfsh>create region --name=my-cache-region --type=PARTITION_REDUNDANT_HEAP_LRU
     Member      | Status
---------------- | -------------------------------------------------------
cacheserver-z2-1 | Region "/my-cache-region" created on "cacheserver-z2-1"
cacheserver-z3-2 | Region "/my-cache-region" created on "cacheserver-z3-2"
cacheserver-z1-0 | Region "/my-cache-region" created on "cacheserver-z1-0"
cacheserver-z1-3 | Region "/my-cache-region" created on "cacheserver-z1-3"

See Region Design for guidelines on choosing a region type.

You can test the newly created region by writing and reading values with gfsh:

gfsh>put --region=/my-cache-region --key=test --value=thevalue
Result      : true
Key Class   : java.lang.String
Key         : test
Value Class : java.lang.String
Old Value   : NULL

gfsh>get --region=/my-cache-region --key=test
Result      : true
Key Class   : java.lang.String
Key         : test
Value Class : java.lang.String
Value       : thevalue

In practice, you should perform these get/put operations from a deployed PCF app. To do this, you must bind the service instance to these apps.

Bind an App to a Service Instance

Binding your apps to a service instance enables the apps to connect to the service instance and read or write data to the region. Run cf bind-service APP-NAME SERVICE-INSTANCE-NAME to bind an app to your service instance. Replace APP-NAME with the name of the app. Replace SERVICE-INSTANCE-NAME with the name you chose for your service instance.

$ cf bind-service my-app my-cloudcache

Binding an app to the service instance provides connection information through the VCAP_SERVICES environment variable. Your app can use this information to configure components, such as the GemFire client cache, to use the service instance.

The following is a sample VCAP_SERVICES environment variable:

  "p-cloudcache": [
      "credentials": {
    "locators": [
    "urls": {
      "gfsh": "",
      "pulse": ""
    "users": [
        "password": "some_developer_password",
        "username": "developer"
        "password": "some_password",
        "username": "cluster_operator"
      "label": "p-cloudcache",
      "name": "test-service",
      "plan": "caching-small",
      "provider": null,
      "syslog_drain_url": null,
      "tags": [],
      "volume_mounts": []

Use the Pulse Dashboard

You can access the Pulse dashboard for a service instance by accessing the pulse-url you obtained from a service key in a web browser.

Use either the cluster_operator or developer credentials to authenticate.

Access Service Instance Metrics

To access service metrics, you must have Enable Plan selected under Service Plan Access on the page where you configure your tile properties. (For details, see the Configure Service Plans page.)

PCC service instances output metrics to the Loggregator Firehose. You can use the Firehose plugin to view metrics output on the CF CLI directly or connect the output to any other Firehose nozzle; for example, the nozzle for Datadog.

PCC v1.2.x supports metrics for the whole cluster and metrics for each member. Each server and locator in the cluster outputs metrics.

Service Instance (Cluster-wide) Metrics

  • serviceinstance.MemberCount: the number of VMs in the cluster
  • serviceinstance.TotalHeapSize: the total MBs of heap available in the cluster
  • serviceinstance.UsedHeapSize: the total MBs of heap in use in the cluster

Member (per-VM) Metrics

  • member.GarbageCollectionCount: the number of JVM garbage collections that have occured on this member since startup
  • member.CpuUsage: the percentage of CPU time used by the GemFire process
  • member.GetsAvgLatency: the avg latency of GET requests to this GemFire member
  • member.PutsAvgLatency: the avg latency of PUT requests to this GemFire member
  • member.JVMPauses: the number of JVM pauses that have occured on this member since startup
  • member.FileDescriptorLimit: the number of files this member allows to be open at once
  • member.TotalFileDescriptorOpen: the number of files this member has open now
  • member.FileDescriptorRemaining: the number of files that this member could open before hitting its limit
  • member.TotalHeapSize: the number of megabytes allocated for the heap
  • member.UsedHeapSize: the number of megabytes currently in use for the heap
  • member.UnusedHeapSizePercentage: the percentage of the total heap size that is not currently being used

Access Service Broker Metrics

Service broker metrics are on by default and can be accessed through the Firehose nozzle plugin. For more information on broker metrics, see On Demand Broker Metrics.

Export gfsh Logs

You can get logs and .gfs stats files from your PCC service instances using the export logs command in gfsh.

  1. Use the Connect with gfsh procedure to connect to the service instance for which you want to see logs.
  2. Run export logs.
  3. Find the ZIP file in the directory where you started gfsh. This file contains a folder for each member of the cluster. The member folder contains the associated log files and stats files for that member.

For more information about the gfsh export command, see the gfsh export documentation.

Deleting a Service Instance

You can delete service instances using the cf CLI. Before doing so, you must remove any existing service keys and app bindings.

  1. Run cf delete-service-key SERVICE-INSTANCE-NAME KEY-NAME to delete the service key.
  2. Run cf unbind-service APP-NAME SERVICE-INSTANCE-NAME to unbind your app from the service instance.
  3. Run cf delete-service SERVICE-INSTANCE-NAME to delete the service instance.
$ cf delete-service-key my-cloudcache my-service-key
$ cf unbind-service my-app my-cloudcache
$ cf delete-service my-cloudcache

Deletions are asynchronous. Run cf services to view the current status of the service instance deletion.

Connecting a Spring Boot App to Pivotal Cloud Cache with Session State Caching

This section describes the two ways you can connect a Spring Boot app to PCC:

  • Using a Tomcat app with a WAR file. This is the default method for Tomcat apps.
  • Using the spring-session-data-gemfire library. This method requires that you use the correct version of these libraries.

Use the Tomcat App

In PCC v1.1 and later, to get a Spring Boot app running with session state caching (SSC) on PCC, you must create a WAR file using the spring-boot-starter-tomcat plugin instead of the spring-boot-maven plugin to create a JAR file.

For example, if you want your app to use SSC, you cannot use spring-boot-maven to build a JAR file and push your app to PCF, because the Java buildpack does not pull in the necessary JAR files for SSC when it detects a Spring JAR file.

To build your WAR file, add this depedency to your pom.xml:


For a full example of running a Spring Boot app that connects with SSC, run this app and use this following for your pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi=""


  <description>Demo project for GemFire Http Session State caching</description>

    <relativePath/> <!-- lookup parent from repository -->



l      <artifactId>spring-boot-starter-tomcat</artifactId>



Use a Spring Session Data GemFire App

You can connect your Spring app to PCC to do session state caching, but you must use the correct version of the spring-session-data-gemfire library to do so. Build the app for PCC v1.2 with a dependency on Spring Session Data GemFire v2.0.0.M2.

Upgrade Spring Session Data GemFire

PCC v1.2.0 only works with Spring Session Data GemFire 2.0.0.M2.

Before your operator upgades to PCC v1.2, stop your app. If you don’t, the upgrade process will break the app. Therefore, Pivotal recommends manually stopping your app before your operator upgrades PCC.

Note: Stopping your app while the operator upgrades to PCC v1.2 will cause downtime.

After your operator has upgraded to PCC v1.2, when you’re ready to restart your app, follow these steps to upgrade to Spring Session Data GemFire 2.0.0.M2:

  1. Rebuild your app using a build.gradle file that depends on the correct version of Spring Session Data GemFire. Here is an example build.gradle file:

    version = '0.0.1-SNAPSHOT'
    buildscript {
      ext {
        springBootVersion = '2.0.0.M3'
      repositories {
        maven { url "" }
        maven { url "" }
      dependencies {
    apply plugin: 'java'
    apply plugin: 'org.springframework.boot'
    apply plugin: 'idea'
        downloadSources = true
        downloadJavadoc = true
    sourceCompatibility = 1.8
    targetCompatibility = 1.8
    repositories {
      maven { url "" }
      maven { url "" }
      maven { url "" }
      maven { url "" }
      maven { url "" }
    dependencies {
  2. Clear the session state region.

  3. Start the rebuilt app.

Creating Continuous Queries Using Spring Data GemFire

To create continuous queries with the Spring Data GemFire library, you must have the following:

  • Spring Data GemFire v2.0.1 release
  • Spring Boot v2.0.0+

To create continuous queries, perform the following steps:

  1. Set subscriptionEnabled and readyForEvents to true on the client cache object through the annotation ClientCacheApplication:

    @ClientCacheApplication(name = "GemFireSpringApplication", readyForEvents = true, subscriptionEnabled = true)
  2. To enable durable queries, set durableClientId and keepAlive through the annotation:

    @ClientCacheApplication(name = "GemFireSpringApplication", durableClientId = "durable-client-id", keepAlive = true, readyForEvents = true, subscriptionEnabled = true)
  3. Use the annotation ContinuousQuery to specify the continuous query and bind it to a function to handle the query when it is triggered:

    public class ContinuousQuery {
        @ContinuousQuery(name = "PestoQuery", query = "SELECT * FROM /Region", durable = true)
        public void handlehanges(CqEvent event) {

    If you want to specify the continuous query in its own class, you must annotate @Component on the class so the Spring component scan detects the @ContinuousQuery annotation.

    If you want the continuous query to be durable, you must set the durable attribute to true.

For more information, see the Spring Data GemFire documentation.

Create a pull request or raise an issue on the source for this page in GitHub