Using Hazelcast IMDG Enterprise for VMware Tanzu
This topic describes how developers use Hazelcast IMDG Enterprise for VMware Tanzu.
After your VMware Tanzu operator installs the Hazelcast IMDG Enterprise for VMware Tanzu tile, it automatically registers itself to the Marketplace.
Follow these steps to create an instance of the Hazelcast service based on an available plan and bind it to your app. The plans available to you are determined by your VMware Tanzu operator. For more information about configuring plans, see Installing and Configuring Hazelcast IMDG Enterprise for VMware Tanzu.
Checking Availability
To check availability of Hazelcast IMDG Enterprise for VMware Tanzu, run the following command:
$ cf marketplace Getting services from marketplace in org system / space system as admin... OK
service plans description hazelcast small, medium, large Hazelcast Service
Create and Bind a Service to Your App
Create a file named
hazelcast.json
using this sample Hazelcast JSON file as a template.Note: Hazelcast IMDG Enterprise for VMware Tanzu v1.8.0 and later require JSON config file that uses new structure. For more information about JSON config structure, see JSON Config Structure below.
Replace
YOUR_LICENSE_KEY
with the active Hazelcast Enterprise License or a trial license. For more information, see License.To create a cluster, run the following command:
$ cf create-service hazelcast PLAN SERVICE_INSTANCE_NAME -c hazelcast.json
For example:
$ cf create-service hazelcast small hz-cluster -c hazelcast.json Creating service instance hz-cluster in org system / space system as admin... OK
Create in progress. Use 'cf services' or 'cf service hz-cluster' to check operation status.You can create 2 nodes Hazelcast cluster only with one line command as well:
$ cf cs hazelcast PLAN SERVICE_INSTANCE_NAME -c '{"hazelcast": {"license-key": "PUT_LICENSE_HERE"}}'
To bind the service to your app, run the following commands:
$ cf bind-service MY-APPLICATION SERVICE_INSTANCE_NAME $ cf restage MY-APPLICATION
For example,
$ cf bind-service myapp hz-cluster $ cf restage myapp
Start using the Hazelcast IMDG Enterprise. Necessary parameters are passed to your application in
VCAP_SERVICES
environment variable. See this sample Hazelcast Spring Boot application for further information.
JSON Config Structure
Hazelcast IMDG Enterprise 3.12 comes with YAML 1.2
configuration support, YAML 1.2
has a JSON
support as well. It gives an opportunity to use native JSON configuration support at tile side accordingly.
The JSON structure consists of two section/JSONs, hazelcast-pcf
section contains tile specific parameters, hazelcast
section is derived from Hazelcast YAML file:
{
"hazelcast-pcf": {
"instanceCount": 2,
"jvmOptions": "",
"tlsArchiveUrl": "",
"userCodeURl": "",
"mancenterAppName": "mancenter",
"mancenterLicense": "",
"mancenterJvmOptions": ""
},
"hazelcast": {
"license-key": "PUT_LICENSE_HERE",
"management-center": {
"enabled": true
}
}
}
A hazelcast-full-example.yaml sample config is available to show syntax for advanced settings. You can copy the relevant parts to your own YAML config file and change the values according to your needs.
Then you can convert your hazelcast.yaml
configuration file with one line command that utilizes yq and jq tools:
yq r -j /path/to/hazelcast.yaml | xargs jq .
Or you can convert the yaml file via any YAML to JSON
converter.
Lastly, copy the JSON output directly into hazelcast
section at main service configuration JSON.
Configuring WAN Replication
For more information on configuring WAN replication in VMware Tanzu, see the official documentation.
Source Cluster
In this example, we will use the Eureka service registry for target cluster discovery. This was tested with Eureka server v1.4.12
.
Edit the hazelcast.json
file for the source cluster to include the following wanReplicationConfigs
configuration:
{
"hazelcast": {
"wan-replication": {
"my-wan-cluster": {
"batch-publisher": {
"publisherId": {
"cluster-name": "cluster-name",
"queue-capacity": 1000,
"queue-full-behavior": "DISCARD_AFTER_MUTATION",
"batch-max-delay-millis": 3000,
"batch-size": 50,
"max-target-endpoints": 5,
"aws": {
"enabled": false
},
"eureka": {
"enabled": true,
"environment": "test",
"name": "hazelcast-target",
"namespace": "hazelcast",
"self-registration": false,
"serviceUrl.default": "http://<my-eureka>/eureka",
"shouldUseDns": false,
"use-classpath-eureka-client-props": false
}
}
}
}
}
}
}
Change cluster-name
to match your existing target cluster configuration, serviceUrl.default
to match your Eureka service endpoint, and any other config values to your needs.
In this example we want to set up WAN replication for a map called my-map
. This can be achieved by including the following hazelcast
section in your hazelcast.json
:
{
"hazelcast": {
"map": {
"my-map": {
"wan-replication-ref": {
"my-wan-cluster": {
"filters": [],
"merge-policy-class-name:": "PassThroughMergePolicy",
"republishing-enabled": false
}
}
}
}
}
}
Change any value as needed.
Target Cluster
This example uses Eureka service registry for cluster member discovery. This allows the other cluster (the source cluster) to find the addresses for this cluster by means of Eureka, as shown in the previous example.
Edit the hazelcast.json
file for the target cluster to include the following hazelcast
section:
{
"hazelcast": {
"network": {
"multicast": {
"enabled": false
},
"eureka": {
"enabled": true,
"environment": "test",
"name": "hazelcast-target",
"namespace": "hazelcast",
"self-registration": true,
"serviceUrl.default": "http://<my-eureka>/eureka",
"shouldUseDns": false,
"use-classpath-eureka-client-props": false
}
}
}
}
Change serviceUrl.default
to match your Eureka service endpoint, and any other config values to your needs.
Configuring User Code Deployment
This section explains how to configure user code deployment, i.e. how to deploy user code at service creation/start-up time.
Summary of Functionality
You can specify additional jar(s) to be added to the server’s classpath. A new optional JSON configuration key (userCodeUrl
) has been defined and can be used to specify one or more URLs for the additional jar(s).
Technical Design
The new JSON configuration key userCodeUrl
can be instantiated either as a string
or as string[]
. For example, a single jar can be specified with the following syntax.
{
"hazelcast-pcf": {
"userCodeUrl": "<usercode-url>"
}
}
To specify multiple jars, the following syntax can be used:
{
"hazelcast-pcf": {
"userCodeUrl": [
"<usercode-url-1>",
"<usercode-url-n>"
]
}
}
Every string specified in userCodeUrl
must be a valid, reachable URL.
At server start-up time, the jar(s) will be downloaded and added to Hazelcast member’s classpath.
During service update, all existing user code jars will be removed before re-downloading the jar(s) specified in userCodeUrl
. The user is therefore expressly allowed to update, add, or delete user code jars with a service-update
operation by changing the userCodeUrl
property.
Configuring Transport Layer Security
This section explains how to configure SSL/TLS for secure communication. SSL (Secure Sockets Layer) protocol was introduced to establish an encrypted communication across your cluster with key stores and trust stores. SSL is now deprecated, so you will be using its successor TLS (Transport Layer Security).
Summary of Functionality
Hazelcast allows you to encrypt socket level communication between Hazelcast members and between Hazelcast clients and members, for end to end encryption. For more information, see the official Hazelcast TLS documentation.
Technical Design
The new JSON configuration key tlsArchiveUrl
is a string
that must contain a valid, reachable URL for a .tar.gz
(or .tgz
) archive. The archive must contain exactly two files, the Hazelcast key store (file must be named keystore
) and the Hazelcast trust store (file must be named truststore
).
{
"hazelcast-pcf": {
"tlsArchiveUrl": "<archive-url>"
}
}
Example Configuration
The following snippet contains the relevant parts of json configuration for enabling TLS. The tlsArchiveUrl
points to an archive containing password-protected key and trust stores. Passwords for the stores are specified in the configuration itself, with keyStorePassword
and trustStorePassword
. Always specify "protocol": "TLS"
and specify the correct keyManagerAlgorithm
and trustManagerAlgorithm
for your stores.
{
"hazelcast-pcf": {
"tlsArchiveUrl": "https://mydomain/myfiles/test.tgz"
},
"hazelcast": {
"network": {
"ssl": {
"enabled": true,
"factory-class-name": "com.hazelcast.nio.ssl.BasicSSLContextFactory",
"properties": {
"keyManagerAlgorithm": "SunX509",
"keyStorePassword": 123456,
"protocol": "TLS",
"trustManagerAlgorithm": "SunX509",
"trustStorePassword": 123456
}
}
}
}
}
Using OpenSSL
Hazelcast provides integration with OpenSSL, which can provide significant performance improvements. For more information, see the official Hazelcast OpenSSL integration documentation.
The OpenSSL libraries and supporting artifacts are already included with Hazelcast IMDG Enterprise for VMware Tanzu. To enable the OpenSSL integration the configuration, you must include "factoryClassName": "com.hazelcast.nio.ssl.OpenSSLEngineFactory"
in the sslConfig
, as in the example below.
{
"hazelcast": {
"network": {
"ssl": {
"enabled": true,
"factory-class-name": "com.hazelcast.nio.ssl.OpenSSLEngineFactory",
"properties": {}
}
}
}
}
Performing a Rolling Upgrade
Hazelcast clusters can be update with a Rolling Upgrade procedure.
Minor vs Patch release upgrade
You can update IMDG version x.y.z to version x.y.z’ where z’ > z at any time. This is called a patch release upgrade.
The protocol version of all members will stay the same and it is identified by x.y.
For instance, suppose you are upgrading from 3.11.3 to 3.11.4. In this case the protocol version is not going to change and it is 3.11.
You can also update IMDG version to x.y.z to version x.y’.z’ where y’ = y + 1 (and no particular relation exists between z and z’). This is called a minor release upgrade.
In this case to conclude the Rolling Upgrade procedure, after all members are upgraded to version x.y’.z’ the protocol version needs to be upgraded to version x.y’.
For instance, suppose you are upgrading from 3.11.4 to 3.12.4. In this case, at the conclusion of the rolling upgrade procedure, the protocol version needs to be upgraded from 3.11 to 3.12.
Please note that version of the tiles are independent of the version of Hazelcast IMDG, but they do follow semantic versioning. Please check the IMDG versions for the old and the new tile in the documentation.
Minor release upgrade: Configuration needed
In order to trigger the protocol version upgrade for a minor release upgrade,
the REST API feature must be enabled in the configuration for the cluster members.
It can be permanently enabled or just temporarily for the Rolling Upgrade procedure.
In any case, if you are doing a Minor release upgrade you have to enable the CLUSTER_WRITE
REST API endpoint group
before starting the Rolling Upgrade procedure.
In order to enable the REST API, you must add the following property to your json
configuration:
{
"hazelcast": {
"rest-api": {
"enabled": true,
"endpoint-groups": {
"CLUSTER_WRITE": {
"enabled": true
}
}
}
}
}
Starting the Rolling Upgrade procedure
After a successful tile update (e.g. via the Ops Manager), you can trigger a Rolling Update for a particular service instance by executing:
$ cf update-service my-hazelcast-instance -c my-hazelcast-config.json
Of course, you must have enabled the REST API in your json
configuration
in case of a minor version upgrade.
You can monitor the RU live with the like of
$ watch cf service my-hazelcast-instance
Minor release upgrade: Upgrading the protocol version
Go to Management Center, log in. Go to Administration Panel, then Rolling Upgrade. Enter cluster name and the new protocol version (e.g., “4.0”). Click Upgrade Protocol.
The Rolling Upgrade is now complete.
Disabling the REST API
At this point, if you wish to disable the REST API you can remove it from the configuration and then trigger another rolling update of the configuration:
$ cf update-service my-hazelcast-instance -c my-hazelcast-config.json
The new configuration will be applied to all members.