Using Hazelcast IMDG Enterprise for VMware Tanzu

Checking Availability
Create and Bind a Service to Your App
- JSON Config Structure
Configuring WAN Replication
- Source Cluster
- Target Cluster
Configuring User Code Deployment
- Summary of Functionality
- Technical Design
Configuring Transport Layer Security
Performing a Rolling Upgrade

This topic describes how developers use Hazelcast IMDG Enterprise for VMware Tanzu.

After your VMware Tanzu operator installs the Hazelcast IMDG Enterprise for VMware Tanzu tile, it automatically registers itself to the Marketplace.

Follow these steps to create an instance of the Hazelcast service based on an available plan and bind it to your app. The plans available to you are determined by your VMware Tanzu operator. For more information about configuring plans, see Installing and Configuring Hazelcast IMDG Enterprise for VMware Tanzu.

Checking Availability

To check availability of Hazelcast IMDG Enterprise for VMware Tanzu, run the following command:

$ cf marketplace
Getting services from marketplace in org system / space system as admin...
OK

service         plans                            description
hazelcast       small, medium, large    Hazelcast Service

Create and Bind a Service to Your App

Create a file named hazelcast.json using this sample Hazelcast JSON file as a template.

Note: Hazelcast IMDG Enterprise for VMware Tanzu v1.8.0 and later require JSON config file that uses new structure. For more information about JSON config structure, see JSON Config Structure below.
Replace YOUR_LICENSE_KEY with the active Hazelcast Enterprise License or a trial license. For more information, see License.

To create a cluster, run the following command:

$ cf create-service hazelcast PLAN SERVICE_INSTANCE_NAME -c hazelcast.json

For example:

$ cf create-service hazelcast small hz-cluster -c hazelcast.json
Creating service instance hz-cluster in org system / space system as admin...
OK

Create in progress. Use 'cf services' or 'cf service hz-cluster' to check operation status.

You can create 2 nodes Hazelcast cluster only with one line command as well:

  $ cf cs hazelcast PLAN SERVICE_INSTANCE_NAME -c '{"hazelcast": {"license-key": "PUT_LICENSE_HERE"}}'

To bind the service to your app, run the following commands:

$ cf bind-service MY-APPLICATION SERVICE_INSTANCE_NAME
$ cf restage MY-APPLICATION

For example,

$ cf bind-service myapp hz-cluster
$ cf restage myapp

Start using the Hazelcast IMDG Enterprise. Necessary parameters are passed to your application in VCAP_SERVICES environment variable. See this sample Hazelcast Spring Boot application for further information.

JSON Config Structure

Hazelcast IMDG Enterprise 3.12 comes with YAML 1.2 configuration support, YAML 1.2 has a JSON support as well. It gives an opportunity to use native JSON configuration support at tile side accordingly.

The JSON structure consists of two section/JSONs, hazelcast-pcf section contains tile specific parameters, hazelcast section is derived from Hazelcast YAML file:

{
  "hazelcast-pcf": {
    "instanceCount": 2,
    "jvmOptions": "",
    "tlsArchiveUrl": "",
    "userCodeURl": "",
    "mancenterAppName": "mancenter",
    "mancenterLicense": "",
    "mancenterJvmOptions": ""
  },
  "hazelcast": {
    "license-key": "PUT_LICENSE_HERE",
    "management-center": {
      "enabled": true
    }
  }
}

A hazelcast-full-example.yaml sample config is available to show syntax for advanced settings. You can copy the relevant parts to your own YAML config file and change the values according to your needs.

Then you can convert your hazelcast.yaml configuration file with one line command that utilizes yq and jq tools:

yq r -j /path/to/hazelcast.yaml | xargs jq .

Or you can convert the yaml file via any YAML to JSON converter.

Lastly, copy the JSON output directly into hazelcast section at main service configuration JSON.

Configuring WAN Replication

For more information on configuring WAN replication in VMware Tanzu, see the official documentation.

Source Cluster

In this example, we will use the Eureka service registry for target cluster discovery. This was tested with Eureka server v1.4.12.

Edit the hazelcast.json file for the source cluster to include the following wanReplicationConfigs configuration:

{
  "hazelcast": {
    "wan-replication": {
      "my-wan-cluster": {
        "batch-publisher": {
          "publisherId": {
            "cluster-name": "cluster-name",
            "queue-capacity": 1000,
            "queue-full-behavior": "DISCARD_AFTER_MUTATION",
            "batch-max-delay-millis": 3000,
            "batch-size": 50,
            "max-target-endpoints": 5,
            "aws": {
              "enabled": false
            },
            "eureka": {
              "enabled": true,
              "environment": "test",
              "name": "hazelcast-target",
              "namespace": "hazelcast",
              "self-registration": false,
              "serviceUrl.default": "http://<my-eureka>/eureka",
              "shouldUseDns": false,
              "use-classpath-eureka-client-props": false
            }
          }
        }
      }
    }
  }
}

Change cluster-name to match your existing target cluster configuration, serviceUrl.default to match your Eureka service endpoint, and any other config values to your needs.

In this example we want to set up WAN replication for a map called my-map. This can be achieved by including the following hazelcast section in your hazelcast.json:

{
  "hazelcast": {
    "map": {
      "my-map": {
        "wan-replication-ref": {
          "my-wan-cluster": {
            "filters": [],
            "merge-policy-class-name:": "PassThroughMergePolicy",
            "republishing-enabled": false
          }
        }
      }
    }
  }
}

Change any value as needed.

Target Cluster

This example uses Eureka service registry for cluster member discovery. This allows the other cluster (the source cluster) to find the addresses for this cluster by means of Eureka, as shown in the previous example.

Edit the hazelcast.json file for the target cluster to include the following hazelcast section:

{
  "hazelcast": {
    "network": {
      "multicast": {
        "enabled": false
      },
      "eureka": {
        "enabled": true,
        "environment": "test",
        "name": "hazelcast-target",
        "namespace": "hazelcast",
        "self-registration": true,
        "serviceUrl.default": "http://<my-eureka>/eureka",
        "shouldUseDns": false,
        "use-classpath-eureka-client-props": false
      }
    }
  }
}

Change serviceUrl.default to match your Eureka service endpoint, and any other config values to your needs.

Configuring User Code Deployment

This section explains how to configure user code deployment, i.e. how to deploy user code at service creation/start-up time.

Summary of Functionality

You can specify additional jar(s) to be added to the server’s classpath. A new optional JSON configuration key (userCodeUrl) has been defined and can be used to specify one or more URLs for the additional jar(s).

Technical Design

The new JSON configuration key userCodeUrl can be instantiated either as a string or as string[]. For example, a single jar can be specified with the following syntax.

{
    "hazelcast-pcf": {
        "userCodeUrl": "<usercode-url>"
    }
}

To specify multiple jars, the following syntax can be used:

{
    "hazelcast-pcf": {
      "userCodeUrl": [
        "<usercode-url-1>",
        "<usercode-url-n>"
      ]
    }
}

Every string specified in userCodeUrl must be a valid, reachable URL.

At server start-up time, the jar(s) will be downloaded and added to Hazelcast member’s classpath.

During service update, all existing user code jars will be removed before re-downloading the jar(s) specified in userCodeUrl. The user is therefore expressly allowed to update, add, or delete user code jars with a service-update operation by changing the userCodeUrl property.

Configuring Transport Layer Security

This section explains how to configure SSL/TLS for secure communication. SSL (Secure Sockets Layer) protocol was introduced to establish an encrypted communication across your cluster with key stores and trust stores. SSL is now deprecated, so you will be using its successor TLS (Transport Layer Security).

Summary of Functionality

Hazelcast allows you to encrypt socket level communication between Hazelcast members and between Hazelcast clients and members, for end to end encryption. For more information, see the official Hazelcast TLS documentation.

Technical Design

The new JSON configuration key tlsArchiveUrl is a string that must contain a valid, reachable URL for a .tar.gz (or .tgz) archive. The archive must contain exactly two files, the Hazelcast key store (file must be named keystore) and the Hazelcast trust store (file must be named truststore).

{
    "hazelcast-pcf": {
        "tlsArchiveUrl": "<archive-url>"
    }
}

Example Configuration

The following snippet contains the relevant parts of json configuration for enabling TLS. The tlsArchiveUrl points to an archive containing password-protected key and trust stores. Passwords for the stores are specified in the configuration itself, with keyStorePassword and trustStorePassword. Always specify "protocol": "TLS" and specify the correct keyManagerAlgorithm and trustManagerAlgorithm for your stores.

{
  "hazelcast-pcf": {
      "tlsArchiveUrl": "https://mydomain/myfiles/test.tgz"
  },
  "hazelcast": {
    "network": {
      "ssl": {
        "enabled": true,
        "factory-class-name": "com.hazelcast.nio.ssl.BasicSSLContextFactory",
        "properties": {
          "keyManagerAlgorithm": "SunX509",
          "keyStorePassword": 123456,
          "protocol": "TLS",
          "trustManagerAlgorithm": "SunX509",
          "trustStorePassword": 123456
        }
      }
    }
  }
}

Using OpenSSL

Hazelcast provides integration with OpenSSL, which can provide significant performance improvements. For more information, see the official Hazelcast OpenSSL integration documentation.

The OpenSSL libraries and supporting artifacts are already included with Hazelcast IMDG Enterprise for VMware Tanzu. To enable the OpenSSL integration the configuration, you must include "factoryClassName": "com.hazelcast.nio.ssl.OpenSSLEngineFactory" in the sslConfig, as in the example below.

{
  "hazelcast": {
    "network": {
      "ssl": {
        "enabled": true,
        "factory-class-name": "com.hazelcast.nio.ssl.OpenSSLEngineFactory",
        "properties": {}
      }
    }
  }
}

Performing a Rolling Upgrade

Hazelcast clusters can be update with a Rolling Upgrade procedure.

Minor vs Patch release upgrade

You can update IMDG version x.y.z to version x.y.z’ where z’ > z at any time. This is called a patch release upgrade.

The protocol version of all members will stay the same and it is identified by x.y.

For instance, suppose you are upgrading from 3.11.3 to 3.11.4. In this case the protocol version is not going to change and it is 3.11.

You can also update IMDG version to x.y.z to version x.y’.z’ where y’ = y + 1 (and no particular relation exists between z and z’). This is called a minor release upgrade.

In this case to conclude the Rolling Upgrade procedure, after all members are upgraded to version x.y’.z’ the protocol version needs to be upgraded to version x.y’.

For instance, suppose you are upgrading from 3.11.4 to 3.12.4. In this case, at the conclusion of the rolling upgrade procedure, the protocol version needs to be upgraded from 3.11 to 3.12.

Please note that version of the tiles are independent of the version of Hazelcast IMDG, but they do follow semantic versioning. Please check the IMDG versions for the old and the new tile in the documentation.

Minor release upgrade: Configuration needed

In order to trigger the protocol version upgrade for a minor release upgrade, the REST API feature must be enabled in the configuration for the cluster members. It can be permanently enabled or just temporarily for the Rolling Upgrade procedure. In any case, if you are doing a Minor release upgrade you have to enable the CLUSTER_WRITE REST API endpoint group before starting the Rolling Upgrade procedure.

In order to enable the REST API, you must add the following property to your json configuration:

{
  "hazelcast": {
    "rest-api": {
      "enabled": true,
      "endpoint-groups": {
        "CLUSTER_WRITE": {
          "enabled": true
        }
      }
    }
  }
}

Starting the Rolling Upgrade procedure

After a successful tile update (e.g. via the Ops Manager), you can trigger a Rolling Update for a particular service instance by executing:

$ cf update-service my-hazelcast-instance -c my-hazelcast-config.json

Of course, you must have enabled the REST API in your json configuration in case of a minor version upgrade.

You can monitor the RU live with the like of

$ watch cf service my-hazelcast-instance

Minor release upgrade: Upgrading the protocol version

Go to Management Center, log in. Go to Administration Panel, then Rolling Upgrade. Enter cluster name and the new protocol version (e.g., “4.0”). Click Upgrade Protocol.

The Rolling Upgrade is now complete.

Disabling the REST API

At this point, if you wish to disable the REST API you can remove it from the configuration and then trigger another rolling update of the configuration:

$ cf update-service my-hazelcast-instance -c my-hazelcast-config.json

The new configuration will be applied to all members.

Create a pull request or raise an issue on the source for this page in GitHub