Preparing a Service Offering

Warning: Container Services Manager for VMware Tanzu is currently in beta and is intended for evaluation and test purposes only. Do not use this product in a production environment.

Page last updated:

This topic describes how to prepare an offered service for Container Services Manager for VMware Tanzu (KSM).

Overview

KSM enables platform operators to offer Helm charts as on-demand services to developers on VMware Tanzu Application Service for VMs (TAS for VMs).

Before you add a service offering to cf marketplace, you must configure the KSM package. You can also configure the package to enable additional features.

To prepare a service offering:

  1. Create a Sample Service Offering Directory
  2. (Optional) Define Plans Configuration
  3. (Optional) Offer Multiple Charts in a Single Offering
  4. (Optional) Create Binding Template for App Consumption
  5. Load Container Images into a Private Container Registry

Prerequisites

Before you add your service offering to KSM, you must:

  • Determine which YAML files you want to include in the service offering: For more infomation, see About Service Offerings.
  • Install the KSM Command Line Interface (CLI): For more information, see Install the KSM CLI.
  • Register a Kubernetes cluster using the KSM CLI: For more information, see Managing Kubernetes Clusters for KSM.
  • Install External Dependencies: If your Helm charts have any external dependencies, they must be installed before you offer your service. For example, if your Helm chart requires an Ingress controller to enable service through ingress, the Ingress controller must be manually installed onto the cluster. For information about Ingress, see Ingress in the Kubernetes documentation.

Create a Sample Service Offering Directory

You can use the KSM CLI to create a directory structure for your service offering. When you create the service offering directory with the KSM CLI, the directory includes the following sample files:

  • SERVICE-NAME/bind.yaml
  • SERVICE-NAME/ksm.yaml
  • SERVICE-NAME/plans.yaml
  • SERVICE-NAME/plans/small.yaml
  • SERVICE-NAME/plans/medium.yaml

To create a sample service offering directory:

  1. Create a directory for your service offering by running:

    ksm offer init SERVICE-NAME
    

    Where SERVICE-NAME is the name of the service offering.

    Note: If you do not include a plans.yaml, a ksm.yaml file, or override files with your service offering, one plan is offered. This plan uses the default values for the Helm chart.

(Optional) Define Plans Configuration

Note: The properties in plans.yaml and PLAN-FILE.yaml files override configurations in override files. For information about override files, see Override Chart Values.

In cf marketplace, each service has a set of plans that developers can provision. A plan is a template for service instances. For example, different plans might represent a large or a small instance of a database.

For Kubernetes services, each KSM plan represents a set of values overriding the default values in the Helm chart. These values are configured in plans.yaml. For examples of plans.yaml files, see Container Services Manager CLIs-sample on GitHub.

If you want to override the default settings in values.yaml and offer custom plans:

  1. Create and edit plans/PLAN-FILE.yaml with the Helm values that you want to override. PLAN-FILE.yaml must consist of only lowercase letters, digits, ., or -.

    Ensure that the properties you add to PLAN-FILE.yaml match the properties in values.yaml and that you only change the values.

    For example:

    ---
    resources:
      requests:
        memory: 128Mi
        cpu: 100m
    
  2. (Optional) Create and edit resource-quotas/RESOURCE-QUOTA-FILE.yaml with the ResourceQuota definition you want to use. This ResourceQuota is created in each Kubernetes namespace where a service instance is provisioned. For more information about ResourceQuota objects, see the Kubernetes documentation.

    For example:

    ---
    apiVersion: v1
    kind: ResourceQuota
    metadata:
      name: mem-cpu-demo
    spec:
      hard:
        requests.cpu: "1"
        requests.memory: 1Gi
        limits.cpu: "2"
        limits.memory: 2Gi
    
  3. Edit your plans.yaml file to add your custom plan information using the following template:

    - name: PLAN-NAME
      description: PLAN-DESCRIPTION
      file: PLAN-FILE
      clusterName: CLUSTER-NAME
      resourceQuotaPath: RESOURCE-QUOTA-FILE
    

    Where:

    • PLAN-NAME is the name of the plan that developers see when they run cf marketplace The plan name is listed under plans. Ensure that the plan name is lowercase only and contains no special characters.
    • PLAN-DESCRIPTION is the description of the plan that developers see when they run cf marketplace. The plan description is listed under description.
    • PLAN-FILE is the name of the plan file in the plans subdirectory.
    • (Optional) CLUSTER-NAME is the name of a cluster that is registered with KSM. For more information, see Register a Kubernetes Cluster. If omitted, service instances are provisioned to the default cluster. For more infomation, see Set a Default Cluster.
    • (Optional) RESOURCE-QUOTA-FILE is name of the resource quota file for the plan.

    For example:

    - name: "small"
      description: "default (small) plan for mysql"
      file: "small.yaml"
      resourceQuotaPath: "quota.yaml"
    - name: "medium"
      description: "medium sized plan for mysql"
      file: "medium.yaml"
      clusterName: "medium-cluster"
    

Note: Developers can override plan and Helm chart values using the -c flag when they provision or update a service instance with the cf CLI. They can use the -c flag to provide service-specific configuration parameters.

When developers provide these configuration parameters, the JSON configurations are mapped directly onto the chart values and take precedence over any chart default and plan values. For information about service-specific configuration parameters, see Arbitrary Parameters.

(Optional) Offer Multiple Charts in a Single Offering

KSM enables you to offer multiple Helm charts in a single service offering. You might want to offer multiple Helm charts in the following cases:

  • A cluster-scoped operator or controller where the individual charts are custom resources
  • A namespace-scoped operator chart with another chart that represents the custom resource
  • A service that uses multiple charts. For example, a service that uses a data service chart and a monitoring service chart.

To offer a service with multiple Helm charts, you must edit the ksm.yaml file. The charts section in a ksm.yaml file is an ordered list of the charts that are included in the service offering.

  1. Edit the ksm.yaml file directory using the following as a template:

    marketplace-name: SERVICE-NAME
    charts:
      - chart: CHART-NAME
        version: CHART-VERSION
        offered: OFFERED-VALUE
        scope: SCOPE-VALUE
      - chart: CHART-NAME
        version: CHART-VERSION
        offered: OFFERED-VALUE
        scope: SCOPE-VALUE
    

    Where:

    • SERVICE-NAME is the name of the service that developers see when they run cf marketplace. The service name is listed under service.
    • CHART-NAME is the name of a chart that you are including in the service offering.
    • CHART-VERSION is the version of the chart that you are including in the service offering.
    • OFFERED-VALUE is set to either true or false. For more information, see the below table.
    • SCOPE-VALUE is set to either cluster or namespace. For more information, see the below table.

The following table describes the Helm chart configurations in ksm.yaml:

Key Values Description
scope cluster If you use this value, the chart is only deployed once on a cluster. Subsequent service instances created on the same Kubernetes cluster do not cause a cluster-scoped chart to be deployed additional times.

You can use this scope for operators and controllers that manage multiple instances from a single deployment.
namespace If you use this value, the chart is deployed on the cluster every time the service is created.
offered true | false Only a single chart can have offered to true. The chart that is set to true must also have scope set to namespace.

The offered chart represents the service the user binds to. However, you can expose more resources with cf bind-service.

(Optional) Create Binding Template for App Consumption

Apps running in TAS for VMs gain access to bound service instances through an environment variable credentials hash called VCAP_SERVICES. Libraries such as Spring Cloud Cloud Foundry Connector automatically support services if their credentials in VCAP_SERVICES are formatted in a valid JSON object.

For an example of the requirements for using Spring Cloud Cloud Foundry Connector with a MySQL service, see the Spring Cloud Cloud Foundry Connector documentation.

Chart authors can edit the bind.yaml file to enable the broker to return a valid JSON object containing service credentials. This bind.yaml file contains a Jsonnet template. When a developer binds the service instance to their app, the service credentials are delivered to VCAP_SERVICES for app consumption.

For information about Jsonnet templates, see Jsonnet.

To edit and test the bind.yaml file:

  1. Edit the bind.yaml file.

    For example, the following bind.yaml enables the broker to return MySQL credentials in a valid JSON obejct:

    template: |
      {
        hostname: $.services[0].status.loadBalancer.ingress[0].ip,
        name: $.services[0].name,
        jdbcUrl: "jdbc:mysql://" + self.hostname + "/my_db?user=" + self.username + "&password=" + self.password + "&useSSL=false",
        uri: "mysql://" + self.username + ":" + self.password + "@" + self.hostname + ":" + self.port + "/my_db?reconnect=true",
        password: $.secrets[0].data['mysql-root-password'],
        port: 3306,
        username: "root"
      }
    

    Note: The values of $.services, $.secrets and $.ingresses are from the namespace where the chart is deployed.

  2. Test if the bind.yaml template is creating the expected JSON object:

    1. Ensure that you have registered a default cluster. See Managing Clusters.

    2. Use the registered default cluster as your local configuration.

    3. Ensure that your kubectl points to the registered default cluster.

    4. Install the helm chart that you are including in the service offering. Wait for the service to be ready and for the public IP address to be available:

      helm install RELEASE-NAME HELM-CHART.tgz -n NAMESPACE --set service.type=LoadBalancer --wait
      

      Where:

      • RELEASE-NAME is the name of the release instance you’re creating.
      • HELM-CHART is the helm chart file name.
      • NAMESPACE is the name of the namespace in which you are installing your helm chart.

      For example:

      kubectl create namespace mysql-ns
      helm install mysql mysql-0.1.4.tgz -n mysql-ns --set service.type=LoadBalancer --wait
      

    5. Execute the test-bind-template by running the following command:

      ksm test-bind-template CLUSTER-NAME NAMESPACE bind.yaml
      

      Where:

      • CLUSTER-NAME is the name of the KSM-registered cluster where your chart is installed.
      • NAMESPACE is the name of the namespace in which you installed your helm chart.

        For example:
         $ ksm test-bind-template my-cluster mysql-ns ./bind.yaml
        Services, secrets, and ingress:
        {
        "secrets": [
        {
            "data": {
                "mysql-password": "aBcDefG1HIJ",
                "mysql-root-password": "kLmNoP2qrSt3"
            },
            "name": "1-23456789-mysql"
        },
        ],
        "services": [
        {
            "metadata": {
                "name": "1-23456789-mysql",
                "namespace": "ksm-f5d531d5-7b07-4bc0-8114-8aba5e2bd706",
                "uid": "9f2d94c1-4900-11ea-a03a-42010a800053",
                "resourceVersion": "4943530",
                "creationTimestamp": "2020-02-06T16:49:20Z",
                "labels": {
                    "app": "1-23456789-mysql",
                    "heritage": "Helm",
                    "release": "1-23456789"
                }
            },
            "name": "1-23456789-mysql",
            "spec": {
                "ports": [
                    {
                        "name": "mysql",
                        "protocol": "TCP",
                        "port": 3306,
                        "targetPort": "mysql",
                        "nodePort": 32257
                    }
                ],
                "selector": {
                    "app": "1-23456789-mysql"
                },
                "clusterIP": "10.24.2.216",
                "type": "LoadBalancer",
                "sessionAffinity": "None",
                "externalTrafficPolicy": "Cluster"
            },
            "status": {
                "loadBalancer": {
                    "ingress": [
                        {
                            "ip": "33.44.55.66"
                        }
                    ]
                }
            }
        }
        ]
        }
        Rendered template {
        "hostname": "33.44.55.66",
        "jdbcUrl": "jdbc:mysql://33.44.55.66/my_db?user=root&password=kLmNoP2qrSt3&useSSL=false",
        "name": "1-23456789-mysql",
        "password": "kLmNoP2qrSt3",
        "port": 3306,
        "uri": "mysql://root:kLmNoP2qrSt3@33.44.55.66:3306/my_db?reconnect=true",
        "username": "root"
        }
    6. Examine the output.

      The first part shows the available secrets and services information. The Rendered template section shows how the bind.yaml template rendered the data.

(Optional) Template Values

Helm uses the literal contents of the values.yaml file as the concrete set of substitutions when creating a release. However, KSM uses values.yaml as a template for multiple releases.

This difference can create cases where values.yaml needs to go through the Helm templating engine before any template files in the /templates directory are rendered with those values.

values.yaml might need to go through the Helm templating engine in the following cases:

  • When a chart uses Ingress controller. For information, see Ingress in the Kubernetes documentation.
  • When there is a unique name, in addition to the release name, required in a chart.

KSM runs the values through the Helm templating engine. For example, in a chart with ingress hosts, you can edit values.yaml to use the release name as the unique subdomain discriminator in the ingress definition:

...
ingress:
  enabled: true
  hosts:
  - name: "{{ .Release.Name }}.example.com"
...

The plan-specific values files also go through the Helm templating engine.

Load Container Images into a Private Container Registry

VMware recommends using a private container registry in production. KSM can then modify your Helm charts to point to images in the private container registry.

If either of the following applies, you must configure a private container registry:

  • Your environment is air-gapped and cannot connect to public container registries such as Docker Hub or Quay. For information about commonly used public container registries, see Docker Hub, Harbor, and Quay.
  • The images referenced in the Helm chart are not public.

If you do not load your container images in to a private container registry, a developer cannot create a service instance offered on KSM.

To load your container images:

  1. Load your container images by following the instructions for your specific container registry. For example, if you use Harbor, follow the instructions in Pulling and pushing images using Docker client in GitHub.

  2. Confirm that your values.yaml file uses image, images, or global.imageRegistry keys. If you use the keys above, KSM always uses the private registry configured during installation to fetch your images.

    For example:

    • If you use a chart with a single image:

      ---
      image: "my-image"
      imageTag: "5.7.14"
      
    • If you use a chart with multiple images:

      ---
      images:
        component1:
          image: "my-first-image"
          imageTag: "5.7.14"
        component2:
          image: "my-second-image"
          imageTag: "1.2.3"
      
    • If you use a chart with a global.imageRegistry key:

      ---
      global:
        imageRegistry: docker.example.com/image-registry
      

    If the Helm chart uses a key other than an image, images or global.imageRegistry key, then KSM does not change the image reference used to pull from the private container registry.