Installing GemFire for Pivotal Cloud Foundry

Prerequisites

Before you begin your GemFire for Pivotal Cloud Foundry (PCF) deployment, your system needs to meet the following minimum requirements:

  • Pivotal Cloud Foundry Ops Manager for vSphere, or
    Pivotal Cloud Foundry Ops Manager for vCloud Air or vCloud Director, or
    Pivotal Cloud Foundry Ops Manager for AWS

  • IPSec (optional): If you wish to use IPSec in your Pivotal GemFire installation, please ensure you have the IPSec BOSH release deployed before installing the Elastic Runtime and GemFire tile. For instructions on deploying the IPSec BOSH release, see the instructions here.

  • Pivotal Cloud Foundry Elastic Runtime

  • Network access and credentials for the Pivotal Cloud Foundry Ops Manager Web Console

  • Capacity in the vSphere cluster for service instances you want to deploy. By default, the GemFire service configures three different GemFire service plans. Each plan deploys the minimum of 2 locators, but deploy 3, 5, and 7 GemFire cache servers, respectively. Several supporting VMs are also required for service deployment. The resource allocations for the default GemFire service plan instances and components are as follows:

Table 1. Default Resource Requirements for a GemFire Service Instances
Service Virtual Machines CPU * RAM (MB) * Ephemeral Disk (MB) * Persistent Disk (MB) *
GemFire locator (Plan 1) 2 1 1024 2048 1024
GemFire locator (Plan 2) 2 1 1024 2048 1024
GemFire locator (Plan 3) 2 1 1024 2048 1024
GemFire Server (Plan 1) 3 2 4096 6144 8192
GemFire Server (Plan 2) 3 2 4096 6144 8192
GemFire Server (Plan 3) 3 2 4096 6144 8192
GemFire Broker 1 2 4096 4096 4096
Broker Registrar 1 1 2048 2048 0
Broker Deregistrar 1 1 2048 2048 0
Cluster Smoke Test 1 1 512 2048 0
Cluster Agent Smoke Test 1 1 512 2048 0
Service Offering Smoke Test 1 1 512 2048 0
Compilation 2 1 1024 4096 0
Totals 12 9 16384 24576 12800

*Required for each VM

Service Configuration Defaults

The GemFire for Pivotal Cloud Foundry service automatically implements the best practices an operator would normally employ when deploying GemFire. These configurations include:

  • Dynamic memory management. Dynamically set the JVM maximum and minimum memory utilization for each instance of the GemFire software process based on the VM’s total memory.

  • Garbage collection. Enable and configure memory garbage collection for each instance of the GemFire software process.

  • Logging configuration. Create and store logs at the config log level with appropriate limitations on sizing and automatic log rotation.

  • Statistics monitoring. Create and store statistics files at a sample rate of 1,000, which corresponds to a sampling rate of once per second.

  • Data overflow configuration. Overflow data to disk when crossing 70% memory utilization.

  • Memory overutilization protection. Prevent further writes to memory when crossing 80% memory utilization.

  • Client authentication. Require clients to authenticate when directly accessing cache servers.

The default configuration settings can be adjusted, if needed, on a per service instance basis.

Installation Steps

  1. Download the GemFire for Pivotal Cloud Foundry tile from Pivotal Network.

  2. Use a Web browser to log in to the Pivotal Ops Manager application. The Pivotal Cloud Foundry Ops Manager Installation Dashboard displays.

  3. Click Import a Product.

    The Add Products screen displays.

  4. Click Choose File and navigate to the file you downloaded.

    The file uploads to your Pivotal Cloud Foundry deployment.

  5. Click Add.

    Pivotal Ops Manager adds a new tile for GemFire for Pivotal Cloud Foundry to the Installation Dashboard.

    GemFire Service Tile

Self-Signed and Internal SSL Certificates

In production environments, we recommend that you use an SSL certificate signed by a reputable certificate authority (CA). Please ensure your SSL certificate has all the Subject Alternative Names (SANs):

  • *.system.example.com
  • *.apps.example.com
  • *.uaa.system.example.com
  • *.login.system.example.com

If your certificate is signed by a known authority and has the correct SANs, please disregard this section.

Internal Certificates

In the case of Internal Certificates (i.e. certificates signed by an internal Root CA), you need to configure the Ops Manager Director tile to trust the internal Root CA. To achieve this, please follow the steps below:

  1. Add the internal Root CA is to the Security Config section under the Ops Manager Director Tile.

  2. Copy the internal certificate and Root CA (certificate chain) to the Elastic Runtime tile by navigating to Pivotal Elastic Runtime > Networking > Load Balancer with TLS enabled (or HAProxy)

Self-Signed Certificates

When using self signed certificates generated by the Elastic Runtime Tile, you need to ensure that this certificate is provided to the Ops Manager Director Tile.

Generating Self-Sign Certificates using the Elastic Runtime Tile

The Elastic Runtime Tile allows you to generate self-signed certificates by navigating to Pivotal Elastic Runtime > Networking > Generate RSA Certificate. Please specify the following wildcard subdomains:

  • *.system.example.com
  • *.apps.example.com
  • *.uaa.system.example.com
  • *.login.system.example.com

Once you have generated your certificate, copy the certificate to the Ops Manager Director Tile and place it in the Security Config section.

Generating Self-SignedCertificates using OpenSSL

  1. To use the OpenSSL CLI, run the following commands:

    1. Generate a private key: openssl genrsa -des3 -out server.key 1024
    2. Generate a Certificate Signing Request (CSR) with the subjectAltName entries, and edit the subj flag as needed: openssl req -new \ -key server.key \ -out server.csr \ -subj '/C=US/ST=State/L=Locality/O=Organization/OU=Organization Unit/CN=www.example.com/emailAddress=example@example.com/subjectAltName=DNS.1=*.system.example.com,DNS.2=*.apps.example.com,DNS.3=*.uaa.system.example.com,DNS.4=*.login.system.example.com'
    3. Remove Passphrase from Key: cp server.key server.key.org openssl rsa -in server.key.org -out server.key
    4. Generate a Self-Signed Certificate: openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
  2. Upload the SSL certificate to your load balancer. Instructions for AWS:

    1. Authenticate with the AWS CLI: $ aws configure AWS Access Key ID [None]: <ACCESS KEY GOES HERE> AWS Secret Access Key [None]: <SECRET ACCESS KEY GOES HERE> Default region name [None]: us-east-1 Default output format [None]: ENTER
    2. Upload your self-signed certificate in IAM: aws iam upload-server-certificate \ --server-certificate-name <NAME OF CERTIFICATE> \ --certificate-body file://server.crt \ --private-key file://server.key
    3. Upload your self-signed certificate to your Elastic Load Balancer (ELB): aws elb set-load-balancer-listener-ssl-certificate \ --load-balancer-name <ELB NAME> \ --load-balancer-port <PORT> \ --ssl-certificate-id arn:aws:iam::<IAM NUMBER>:server-certificate/<NAME OF CERTIFICATE>
  3. Add the SSL certificate as a trusted certificate to your Ops Manager-deployed VMs.

    1. In Ops Manager, on the Director Tile, under the Security section, paste the certificate, and save. Trusted Certificates
    2. On the Elastic Runtime Tile, under the Networking section, paste both the certificate and the private key, be sure to check Disable SSL certificate verification for this environment, and save. Elastic Runtime Security Config

## WAN Replication and Self-Signed Certificates

A WAN topology in GemFire is a client-server relationship. When using self-signed certificates, each client needs to trust the server’s SSL certificate.

When using WAN replication with self-signed certificates, we strongly recommend using certificates that were generated with the OpenSSL CLI. This is because Ops Manager self-signed certificates are actually signed by Ops Manager’s internal CA certificate. If you do use a certificate generated by Ops Manager, you’ll need to trust the Ops Manager’s internal CA certificate instead.

For example, when setting up WAN replication across two PCF installations who each use their own self signed certificates, named West and East:

  1. In West Ops Manager, on the Director Tile, under Experimental Features > Trusted Certificates, add the certificate for East (appending if necessary), and save. Trusted Certificates
  2. In East Ops Manager, on the Director Tile, under Experimental Features > Trusted Certificates, add the certificate for West (appending if necessary), and save.
  3. In both installations, you need to redeploy the GemFire for Pivotal Cloud Foundry tile for these changes to take effect. The simplest way to force a redeploy is to make a small change to any of the configuration properties, such as the Service Plan Configuration.

Then proceed to configure WAN replication across your multiple sites.

Creating GemFire Service Plans

Configure GemFire service instances to set the maximum capacity for each cluster provided in your Cloud Foundry environment.

Note: If you have already configured the service, reducing the number of resources and reconfiguring the service causes all allocated instances to be destroyed. See the Release Notes for more information about known problems and limitations.

Follow these steps to configure the GemFire service:

  1. Select the GemFire for Pivotal Cloud Foundry tile to display the configuration page.

  2. (Optional) Select the Assign Networks tab to specify the vSphere Network where Ops Manager deploys the GemFire Service Broker and related virtual machines. See Configuring Network Isolation Options in Ops Manager in the Pivotal Cloud Foundry documentation for more information.

  3. Select Assign Availability Zones to specify the Availability Zone into which Ops Manager should deploy the GemFire Service Broker and virtual machines. Note that the current version of the GemFire service must be deployed to a single Availability Zone. See Configuring Ops Manager Director for VMware vSphere for more information.

    Note: The current version of the service only supports deployment to one Availability Zone, even if multiple Availability Zones are available. Deploying the service to more than one AZ results in an invalid installation.

  4. Select the Configure Service Plan 1-3 tabs to configure the each of the three GemFire plans that will be available in the Elastic Runtime CLI and Web Console. GemFire for Pivotal Cloud Foundry provides three different service plans that you can configure to provide different GemFire cluster sizes. For each of the three service plan configurations:

    1. (Optional) As a best practice, use the Service Plan Name, Service Plan Description, and Service Plan Feature Bullet 1-3 fields to describe the size of each cluster. This information will appear in the Marketplace when developers are choosing from the available plans. For example:

    Service Plan Description 2. Specify the number of locators per cluster to create in the GemFire cluster for this plan, as well as the number of cache servers per cluster. For example: Service Plan Members Note: Each service plan must specify a minimum of two locators and three cache servers per GemFire cluster. Record the number of locators and servers used in each plan to help you configure the total plan resources in the next step. If you would like to defer a plan configuration or allocation until a later time, you can do that in the Resource Config tab (see below). 3. Click Save to save the current service plan configuration. 4. Repeat the above steps to configure each of the three GemFire service plans.

  5. (Optional) Select the Resource Config tab to change the allocation of resources for of GemFire members in each service plan:

    GemFire Service Resource Config

    1. Enter the total number of GemFire locator and server Instances that you want to create for each of the available service plans.

      Note: The number of locator or server instances that you specify for a plan must be a multiple of the per cluster number locators and servers that you configured in the previous step. The only exception to this rule is if you want to defer allocating resources for the plan to a later time, in which case you can set the number of locators and servers to zero.

      For example, if you configured GemFire Service Plan 1 to provide a small GemFire cluster using the minimum of 2 locators and 3 servers, but you want to wanted to make 3 instances of this plan available in the marketplace, then you would set GemFire locator (Plan 1) to 6 instances, and GemFire server (Plan 1) to 9 instances.

      If you instead wanted to postpone configuring the plan and “save” your pending changes to the plan, then set the number of locators and servers to zero. You can then revisit the plan configuration and resource allocation at a later time. Be sure to set valid values for the cluster size and resources under Resource Config before you apply your final changes to the plan.

    2. (Optional) For GemFire service plan cache servers only, enter the amount of CPU, RAM (MB), EPHEMERAL DISK (MB), and PERSISTENT DISK (MB) resources to allocate for each server VM, based on your capacity requirements. For production deployments, Pivotal recommends increasing the number of CPUs to at least 2.

      Approximately 1 GB of configured RAM is reserved for the VM operating system; all RAM above this amount is allocated to the JVM that runs the GemFire server.

      Note: The resources allocations for GemFire locators and other service components are automatically configured to their recommended settings, and they cannot be changed.

  6. Click Save.

  7. Click the Installation Dashboard link.

    The Installation Dashboard screen displays.

  8. Click Apply Changes.

    Pivotal Cloud Foundry Ops Manager deploys a single virtual machine to run the GemFire Service Broker, preallocates all additional VMs required for the three GemFire Service plan clusters. You can access information about the deployments from the Pivotal Cloud Foundry Ops Manager console.

    After the GemFire cluster service instances are deployed, the GemFire Service Broker automatically registers the service and its service plans in the Elastic Runtime Marketplace. Pivotal Cloud Foundry users can now create and bind to instances of the configured service plans. See Using the Pivotal GemFire Service on Pivotal Cloud Foundry.

Deferring Service Plan Configuration

As mentioned in the procedure above, it is possible to leave a service plan unallocated, and defer its configuration and resource allocation to a later time. To do this, set the number of locators and servers for the plan to zero in the Resource Config tab.

When you are ready to finalize the configuration, follow the same steps in Creating GemFire Service Plans to set any required properties, and allocate resources for the plan in the Resource Config tab. Finally, click Apply changes in the Ops Manager to create the service instances that you allocated.

Application Security Groups

To enable access to the GemFire for PCF tile service, you need to ensure your security group allows access to the Gemfire locator, and server VMs configured in your deployment. The IP addresses for these can be obtained from OpsManager by clicking on GemFire Tile and navigating to Status tab. All the IPS are mentioned under the column IPS. You should ensure the following TCP ports are enabled for the IP addresses: 7071, 40404, 55221. More details can be found at Application Security Groups.

{"protocol":"tcp","destination":"<YOUR-GEMFIRE-LOCATORS-AND-SERVERS-IP-RANGE>","ports":"7071, 40404, 55221"}

Warning

We recommend updating your PCF installation’s default application security group as the default application security group in Pivotal Cloud Foundry allows all egress traffic.

Default application security group:

[
  {
          "destination": "0.0.0.0-169.254.169.253",
          "protocol": "all"
  },
  {
          "destination": "169.254.169.255-255.255.255.255",
          "protocol": "all"
  }
]

Additional application security group rules may be necessary to to ensure other PCF services continue to function correctly.

Create a pull request or raise an issue on the source for this page in GitHub