Using a9s RabbitMQ for PCF

This topic describes how developers use a9s RabbitMQ.

Use a9s RabbitMQ with an App

To use a9s RabbitMQ with an application, create a service instance and bind the service instance to your app. For more information about managing service instances, see Managing Service Instances with the cf CLI.

View the a9s RabbitMQ Service

After the tile is installed, you can see the a9s-rabbitmq and its service plans appear in your PCF marketplace. Run cf marketplace to see the service listing: bash $ cf marketplace Getting services from marketplace in org test / space test as admin... OK service plans description a9s-rabbitmq36 rabbitmq-single-small, rabbitmq-cluster-small, This is the anynines rabbitmq 36 service. rabbitmq-single-big, rabbitmq-cluster-big

Create a Service Instance

To provision a RabbitMQ database, run cf create-service. For example:

$ cf create-service a9s-rabbitmq36 rabbitmq-single-small my-rabbitmq-service

Depending on your infrastructure and service broker utilization, it might take several minutes to create the service instance.

Run cf services to view the creation status. This command displays a list of all your service instances. To view the status of a specific service instance, run cf service NAME-OF-YOUR-SERVICE.

Bind an Application to a Service Instance

After your database is created, run cf bind-service to bind the service to your application:

$ cf bind-service a9s-rabbitmq-app my-rabbitmq-service

Restage or Restart Your Application

To enable your appliation to access the service instance, run cf restage or cf restart to restage or restart your app.

Obtain Service Instance Access Credentials

After a service instance is bound to an application, the credentials of your RabbitMQ database are stored in the environment variables of the application. Run cf env APP-NAME to display the environment variables.

You can find the credentials in the VCAP_SERVICES key.

$ cf env a9s-rabbitmq-app
Getting env variables for app a9s-rabbitmq-app in org test / space test as admin...
OK

System-Provided:
{
  "VCAP_SERVICES": {
    "a9s-rabbitmq36": [
    {
      "credentials": {
        "dns_servers": [
          "172.28.10.20",
          "172.28.11.30",
          "172.28.12.23"
        ],
        "host": "d87e464.service.dc1.a9s-rabbitmq-consul",
        "host_ip": "172.28.25.12",
        "host_ips": [
          "172.28.25.12",
          "172.28.26.11",
          "172.28.27.12"
        ],
        "hosts": [
          "d87e464-mq-0.node.dc1.a9s-rabbitmq-consul",
          "d87e464-mq-1.node.dc1.a9s-rabbitmq-consul",
          "d87e464-mq-2.node.dc1.a9s-rabbitmq-consul"
        ],
        "password": "a9s-pwd",
        "port": 5672,
        "uri": "amqps://a9s-brk-usr:a9s-pwd@d87e464.service.dc1.a9s-rabbitmq-consul:5672",
        "username": "a9s-brk-usr"
      },
      "label": "a9s-rabbitmq36",
      "name": "2452-23360-25011-17366",
      "plan": "rabbitmq-replica-small-ssl",
      "provider": null,
      "syslog_drain_url": null,
      "tags": [
        "messaging",
        "queue"
      ],
      "volume_mounts": []
    }
    ]
  }
}
...

You can use the host, username, and password values to connect to your database with a RabbitMQ client.

Delete an a9s RabbitMQ Service Instance

WARNING: Before deleting a service instance, you must back up data stored in your database. This operation cannot be undone and all the data is lost when the service is deleted.

Before you can delete a service instance, you must unbind it from all apps.

List Available Services

Run cf services to list your available services.

$ cf services

Getting services in org test / space test as admin...
OK

name                  service          plan                    bound apps         last operation
my-rabbitmq-service   a9s-rabbitmq36   rabbitmq-single-small   a9s-rabbitmq-app   create succeeded

This example shows that my-rabbitmq-service is bound to the a9s-rabbitmq-app application.

Unbind a Service Instance

Run cf unbind-service to unbind the service instance from your app.

$ cf unbind-service a9s-rabbitmq-app my-rabbitmq-service

Delete a Service Instance

After unbinding the service, it is no longer bound to an app. Run cf delete-service to delete the service:

$ cf delete-service my-rabbitmq-service

It may take several minutes to delete the service. Deleting a service deprovisions the corresponding infrastructure resources. Run the cf services command to view the deletion status.

Upgrade the Service Instance to Another Service Plan

Once created, you can upgrade your service instance to another, larger service plan. A larger service plan provides more CPU, RAM and storage. For more information, see the Update a Service Instance of the Managing Service Instances with the cf CLI topic.

$ cf update-service my-rabbitmq-service -p a-bigger-plan

Depending on the one you are currently using, the plans you can upgrade to are as follows:

Base Plan Target Plan
rabbitmq-single-nano rabbitmq-single-small
rabbitmq-single-medium
rabbitmq-single-big
rabbitmq-replica-small
rabbitmq-replica-medium
rabbitmq-replica-big
rabbitmq-single-small rabbitmq-single-medium
rabbitmq-single-big
rabbitmq-replica-small
rabbitmq-replica-medium
rabbitmq-replica-big
rabbitmq-single-medium rabbitmq-single-big
rabbitmq-replica-medium
rabbitmq-replica-big
rabbitmq-single-big rabbitmq-replica-big
rabbitmq-replica-small rabbitmq-replica-medium
rabbitmq-replica-big
rabbitmq-replica-medium rabbitmq-replica-big
rabbitmq-single-nano-ssl rabbitmq-single-small-ssl
rabbitmq-single-medium-ssl
rabbitmq-single-big-ssl
rabbitmq-replica-small-ssl
rabbitmq-replica-medium-ssl
rabbitmq-replica-big-ssl
rabbitmq-single-small-ssl rabbitmq-single-medium-ssl
rabbitmq-single-big-ssl
rabbitmq-replica-small-ssl
rabbitmq-replica-medium-ssl
rabbitmq-replica-big-ssl
rabbitmq-single-medium-ssl rabbitmq-single-big-ssl
rabbitmq-replica-medium-ssl
rabbitmq-replica-big-ssl
rabbitmq-single-big-ssl rabbitmq-replica-big-ssl
rabbitmq-replica-small-ssl rabbitmq-replica-medium-ssl
rabbitmq-replica-big-ssl
rabbitmq-replica-medium-ssl rabbitmq-replica-big-ssl

Add a Graphite Endpoint

If you want to monitor your service with Graphite, you can set an endpoint to where to information will be sent with the cf update-service command. This command expects the -c flag and a JSON string containing the graphite and metrics_prefix keys.

Depending on your Graphite provider, the metrics_prefix might require that each metric must start with an API key in their name. You can also change the interval within the data is send to the endpoint. To do this, modify interval; the default is 10s.

$ cf update-service my-rabbitmq-service -c '{ "graphite": ["yourspace.your-graphite-endpoint.com:12345"], "metrics_prefix": "your-api-key.my-cluster-rabbitmq", "interval": "5"}'

Add a Syslog Endpoint

The cf update-service command used with the -c flag can let you stream your syslog to a third-party service. In this case, the command expects a JSON string containing the syslog key. You can also change the interval for the syslog with the same key than for the Graphite endpoint interval.

$ cf update-service my-rabbitmq-service -c '{ "syslog": ["logs4.your-syslog-endpoint.com:54321"], "interval": "5" }'

Cloud Foundry Application Security Groups

This section describes how to check whether a security group was created.

Each a9s Data Service will automatically create and update Cloud Foundry security groups in order to protect service instances to be accessed by apps not running in the same Cloud Foundry applications space. To understand Security Groups, see Understanding Application Security Groups.

Get Service Instance GUID

Run cf service INSTANCE_NAME --guid to get the GUID of the service instance.

$ cf service my-rabbitmq --guid
ca16f111-5073-40b7-973a-156c75dd3028

Check Available Security Groups

To see all available security groups, run cf security-groups.

$cf security-groups
Getting security groups as demo@anynines.com
OK

     Name                                         Organization     Space
#0   public_networks
#1   dns
#2   tcp_open
#3   guard_432fb752-876d-443b-a311-a075f4df2237   demonstrations   demo
#4   guard_ca16f111-5073-40b7-973a-156c75dd3028   demonstrations   demo

In the above example, you can see a security group with the name guard_ca16f111-5073-40b7-973a-156c75dd3028 was successfully created.

Note: In some circumstances, the connection between the app and the service instance is not possible. In this case, check to see if a security group was created.

Back Up and Restore Service Instances

a9s RabbitMQ provides an easy way to create backups and restore if needed.

Get Dashboard Address, Log In, and Authorize

  1. Grap the dashboard URL with cf service SERVICE-NAME. “`bash $cf service my-rabbitmq

Service instance: my-rabbitmq Service: a9s-rabbitmq Bound apps: Tags: Plan: rabbitmq-single-small Description: This is a service creating and managing dedicated RabbitMQ service instances and clusters, powered by the anynines Service Framework Documentation url: Dashboard: https://a9s-rabbitmq-dashboard.aws.ie.a9s.eu/service-instances/ca16f111-5073-40b7-973a-156c75dd3028

Last Operation Status: update succeeded Message: Started: 2017-10-26T08:28:38Z Updated: 2017-10-26T08:28:38Z ”`

  1. Browse to the dashboard URL and authenticate on the redirected page with your Cloud Foundry credentials: authentication-page

  2. Click Authorize to approve the authorization request: authorization-page

Perform a Backup

On the dashboard, you can trigger a backup by clicking Trigger backup.

service-dashboard

After a short period of time, the backup is queued.

service-dashboard

Note: Depending on the size of the data, the backup may take some time.

Restore a Backup

  1. Open the dashboard again and select the backup you would like to restore.

  2. Click Restore. After a short period of time, the restore is triggered.

service-dashboard

Note: Depending on the size of the data, the restore may take some time.

service-dashboard

Use RabbitMQ Plugins

a9s RabbitMQ allows you to enable the following RabbitMQ plugins:

  • rabbitmq_consistent_hash_exchange
  • rabbitmq_federation
  • rabbitmq_federation_management
  • rabbitmq_mqtt
  • rabbitmq_sharding
  • rabbitmq_shovel
  • rabbitmq_shovel_management
  • rabbitmq_stomp
  • rabbitmq_tracing

Installation

You can install RabbitMQ plugins by running cf create-service and cf update-service using additional configuration parameters.

$ cf create-service a9s-rabbitmq36 rabbitmq-single-small my-rabbitmq-service -c '{ "plugins": ["rabbitmq_shovel", "rabbitmq_shovel_management"] }'
$ cf update-service my-rabbitmq-service -c '{ "plugins": ["rabbitmq_shovel", "rabbitmq_shovel_management"] }'

Create RabbitMQ Users

a9s RabbitMQ allows you to create new users with different roles (permissions) by creating service keys.

The possible roles are: * management * policymaker * monitoring * administrator

You can read more about RabbitMQ roles and permissions in the Permissions section of the Management Plugin topic of the RabbitMQ documentation.

Create a New Role

You can create a new user with specific permissions by giving the role the following custom parameter. This parameter receives an array of roles:

$ cf create-service-key my-rabbitmq-service my-key -c '{"roles": ["administrator", "management"]}'

Use RabbitMQ Management Dashboard

a9s RabbitMQ has management dashboard support enabled. The dashboard is running on the service instance VM, so it is not possible to open the dashboard in your browser directly. Cloud Foundry provides a smart way to create SSH forward tunnels through a pushed application. For more information, see the Accessing Apps with SSH section of the Cloud Foundry documentation.

First, you must have an application bound to the service. For more information, see Bind an Application to a Service Instance.

Note: `cf ssh` support must be enabled in the platform. Consult your administrator to ensure it is enabled.

Get Dashboard URL and Credentials

Use the hostname of the service and the user credentials obtained in Obtain Service Instance Access Credentials for this procedure.

$ cf env rabbitmq-app
Getting env variables for app rabbitmq-app in org phartz / space develop as admin...
OK

System-Provided:
{
  "VCAP_SERVICES": {
    "a9s-rabbitmq36": [
    {
      "credentials": {
        "dns_servers": [
          "172.28.10.20",
          "172.28.11.30",
          "172.28.12.23"
        ],
        "host": "d87e464.service.dc1.a9s-rabbitmq-consul",
        "host_ip": "172.28.25.12",
        "host_ips": [
          "172.28.25.12",
          "172.28.26.11",
          "172.28.27.12"
        ],
        "hosts": [
          "d87e464-mq-0.node.dc1.a9s-rabbitmq-consul",
          "d87e464-mq-1.node.dc1.a9s-rabbitmq-consul",
          "d87e464-mq-2.node.dc1.a9s-rabbitmq-consul"
        ],
        "password": "a9s-pwd",
        "port": 5672,
        "uri": "amqps://a9s-brk-usr:a9s-pwd@d87e464.service.dc1.a9s-rabbitmq-consul:5672",
        "username": "a9s-brk-usr"
      },
      "label": "a9s-rabbitmq36",
      "name": "2452-23360-25011-17366",
      "plan": "rabbitmq-replica-small-ssl",
      "provider": null,
      "syslog_drain_url": null,
      "tags": [
        "messaging",
        "queue"
      ],
      "volume_mounts": []
    }
    ]
  }
}

The hostname d67901c.service.dc1.a9svs, username a9s-brk-usr and password a9s-password are required in the next step.

Create Tunnel to the Management Dashboard

With cf ssh, you can create an SSH forward tunnel to the management dashboard.

$ cf ssh rabbitmq-app -L 15672:d67901c.service.dc1.a9svs:15672
vcap@956aaf4e-6da9-4f69-4b1d-8e631a403312:~$

Note: Close the session by running exit.

Log In to the Management Dashboard

When the SSH tunnel is open, you can access the instance using the address localhost:15672.

Log in to the management dashboard with your user credentials. management-dashboard-login

Upon successfully logging in, you can access the dashboard. management-dashboard

Make a Service Instance Locally Available

You can access any of the a9s Data Services locally and connect with a local client to the service for any purpose, such as debugging. Cloud Foundry provides a smart way to create SSH forward tunnels through a pushed application. For more information about this feature, see the Accessing Apps with SSH section of the Cloud Foundry documentation.

First, you must have an app bound to the service. To do this, see Bind an Application to a Service Instance.

Note: `cf ssh` support must be enabled in the platform. Consult your administrator to ensure it is enabled.

Get Dashboard URL and Credentials

Use the hostname of the service and the user credentials obtained in Obtain Service Instance Access Credentials for this procedure.

$ cf env a9s-rabbitmq-app
Getting env variables for app a9s-rabbitmq-app in org test / space test as admin...
OK

System-Provided:
{
  "VCAP_SERVICES": {
   "a9s-rabbitmq": [
    {
      "credentials": {
       "host": [
        "d67901c.service.dc1.a9svs"
       ],
       "password": "a9s-brk-usr",
       "username": "a9s-password"
     },
     "label": "a9s-rabbitmq",
     "name": "my-rabbitmq-service",
     "plan": "rabbitmq-cluster-small"
    }
   ]
  }
}
...

The hostname d67901c.service.dc1.a9svs, username a9s-brk-usr and password a9s-password are required in the next step.

Create Tunnel to the Management Dashboard

With cf ssh, you can create am SSH forward tunnel to the management dashboard. Use port 9200 to connect to the a9s RabbitMQ Instance.

$ cf ssh a9s-rabbitmq-app -L 9200:d67901c.service.dc1.a9svs:9200
vcap@956aaf4e-6da9-4f69-4b1d-8e631a403312:~$

When the SSH tunnel is open, you can access the instance using the address localhost:9200.

Note: Close the session by running exit.

Create a Fork of a Service Instance

Forking a service instance involves creating a backup of the service instance and restoring it to a different service instance.

Having two service instances is a prerequisite for the process: “`shell $ cf s Getting services in org jebreuer_anynines_com / space test as jebreuer@anynines.com

name service plan bound apps last operation rabbit1 a9s-rabbitmq37 rabbitmq-cluster-small bindingo create succeeded rabbit2 a9s-rabbitmq37 rabbitmq-cluster-small create succeeded ”`

Additional prerequisites regarding command line tools: - BASH (some shell) - CAT - OpenSSL - Python - Node (tested with v6.11.0)

Do the following:

  1. Open the service dashboard of the service instance you want to fork, such as rabbit1 in this example. You can find the dashboard URL by running the following command: “`shell $ cf service rabbit1 Showing info of service rabbit1 in org jebreuer_anynines_com / space test as jebreuer@anynines.com

name: rabbit1 service: a9s-rabbitmq37 bound apps: bindingo tags: plan: rabbitmq-cluster-small description: This is a service creating and managing dedicated RabbitMQ service instances, powered by the anynines Service Framework documentation: dashboard: https://a9s-rabbitmq-dashboard.de.a9s.eu/service-instances/950cb675-3ed9-4613-8bb6-b2d618391d2f

[…] ”`

  1. Ensure you set a encryption password for the backups using the service instance dashboard (Change Backup Settings).
  2. Create a backup using the dashboard and download the backup to your local machine. The filename is similar to racsd92baee-1522222422893.

  3. Decrypt the backup and write its contents to a file: shell $ cat racsd92baee-1522222422893 | openssl enc -aes256 -d -pass 'pass:mytopsecretpassword' > backup_settings.json

  4. Create a tunnel to the admin interface of the RabbitMQ instance that will be the fork of the original instance. You need the tunnel to get the matching version of the rabbitmqadmin script as well as to restore the backed-up data. shell $ cf ssh someapp -L 127.0.0.1:15672:racsd92baee.service.dc1.a9ssvc:15672 racsd92baee.service.dc1.a9ssvc is the host of the rabbit2 service instance.

  5. Go to http://127.0.0.1:15672/cli/ to download the rabbitmqadmin tool. rabbitmqadmin is a Python script.

  6. Download a copy of the backup script restore_queues.js. Make sure to chmod u+x the script.

  7. Restore the backed-up queues using the restore script. For example: shell ./restore_queues.js $(which python) ~/Downloads/rabbitmqadmin 127.0.0.1 15672 a9s-brk-usr-xxxxxxxx xxxxxxxyyyyyyyyyzzzzzzzzz ./backup_settings.json

Create a pull request or raise an issue on the source for this page in GitHub