Installing and Configuring KSM

Warning: Container Services Manager for VMware Tanzu is currently in beta and is intended for evaluation and test purposes only. Do not use this product in a production environment.

Page last updated:

This topic describes how to install and configure Container Services Manager for VMware Tanzu (KSM) using Helm.

Overview

Helm is a Kubernetes open source package manager. You can use a Helm chart to install and configure KSM.

To install and configure KSM using Helm:

  1. Get KSM Resources From VMware Tanzu Network

  2. Install the KSM CLI

  3. Move KSM Images

  4. Configure the KSM Helm Chart

  5. Install the Helm Chart

  6. Configure Security

  7. (Optional) Install Prometheus

Prerequisites

Before you install KSM using Helm:

  • KSM CLI: For information about installing the KSM CLI, see Install the KSM CLI .

  • Helm 3 CLI: For information about installing the Helm CLI, see the Helm documentation.

  • Docker CLI: For information about installing Docker, see Docker.

  • TAS for VMs Deployment: A running TAS for VMs deployment.

  • Kubernetes Cluster: A running Kubernetes cluster. KSM supports Tanzu Kubernetes Grid Integrated Edition clusters. For information about TKGI, see Tanzu Kubernetes Grid Integrated Edition.

  • S3 Compatible Storage: KSM requires a S3-compatible bucket to store offered charts and the chart cache. For more information, see Configuring External Storage.

  • Private Container Image Registry: You need this to manage container images in air-gapped environments. VMware recommends using a registry in production deployments. You can use a registry such as Harbor.

  • Verify that a default StorageClass exists on the cluster where you want to install KSM: If you are using Tanzu Kubernetes Grid Integrated Edition, see Specify a Default StorageClass. For more information about Storage Classes, see Kubernetes documentation.

  • Install the Ingress controller on the cluster where you want to install KSM: You must also reserve a subdomain for KSM in your DNS. For information about Ingress, see the Kubernetes documentation.

Get KSM Resources From VMware Tanzu Network

To get the resources needed from VMware Tanzu Network to install KSM:

  1. Log in and navigate to Container Services Manager for VMware Tanzu (KSM) in VMware Tanzu Network.

  2. Get the following resources:

    • The KSM Command Line Interface (CLI): Click CLIs and download the CLI for your operating system.
    • Docker pull commands: Click Image References and record the docker pull commands for the broker, daemon, chartmuseum, and postgresql. You can alternatively download the image tgz files from the images folder.
    • Helm chart TGZ file: Click ksm-VERSION-NUMBER.tgz and download the Helm chart for KSM.

Install the KSM CLI

To install the KSM Command Line Interface (CLI):

  1. Rename the downloaded KSM CLI file as ksm.

  2. Make the KSM binary act as an executable file by running:

    chmod +x ksm
    
  3. Move the binary file into your PATH by running:

    mv ksm /usr/local/bin/ksm
    
  4. Ensure KSM CLI is working properly:

    ksm version
    

Move KSM Images

To move your KSM images to a private container image registry:

  1. Move the images to your local system by running the docker pull commands you recorded in Get KSM Resources From VMware Tanzu Network above.

    If you downloaded the tgz files instead of using the docker pull commands, load those images to your local system by running docker load -i FILE-PATH.

  2. Tag the images for your registry by running these commands:

    docker tag registry.pivotal.io/tanzu-service-manager/broker:VERSION-NUMBER \
     REGISTRY/tanzu-service-manager/broker:VERSION-NUMBER
    
    docker tag registry.pivotal.io/tanzu-service-manager/daemon:VERSION-NUMBER \
     REGISTRY/tanzu-service-manager/daemon:VERSION-NUMBER
    
    docker tag registry.pivotal.io/tanzu-service-manager/chartmuseum:CHARTMUSEUM-VERSION-NUMBER \
     REGISTRY/tanzu-service-manager/chartmuseum:CHARTMUSEUM-VERSION-NUMBER
    
    docker tag registry.pivotal.io/container-services-manager/postgresql:POSTGRESQL-VERSION-NUMBER \
     REGISTRY/container-services-manager/postgresql:POSTGRESQL-VERSION-NUMBER
    

    Where:

  3. Create the tanzu-service-manager project in your registry.

  4. Push the images to your registry by running these commands:

    docker push REGISTRY/tanzu-service-manager/broker:VERSION-NUMBER
    docker push REGISTRY/tanzu-service-manager/daemon:VERSION-NUMBER
    docker push REGISTRY/tanzu-service-manager/chartmuseum:CHARTMUSEUM-VERSION-NUMBER
    docker push REGISTRY/tanzu-service-manager/postgresql:POSTGRESQL-VERSION-NUMBER
    

Configure the KSM Helm Chart

To configure the KSM Helm chart:

  1. Unzip the downloaded KSM Helm chart by running:

    tar zxvf ksm-VERSION-NUMBER.tgz
    

    The above command creates a new directory named ksm.

  2. In the ksm/values.yaml file, add the credentials for your S3-compatible bucket using the template below:

    chartmuseum:
      env:
        open:
          STORAGE_AMAZON_BUCKET: BUCKET-NAME
          STORAGE_AMAZON_ENDPOINT: ENDPOINT
        secret:
          AWS_ACCESS_KEY_ID: ACCESS-KEY
          AWS_SECRET_ACCESS_KEY: SECRET
    

    Where:

    • BUCKET-NAME is your S3 bucket name.
    • ENDPOINT is your S3 endpoint. For example, in Google Cloud Platform (GCP) it is storage.googleapis.com.
    • ACCESS-KEY is your S3 access key ID.
    • SECRET is your S3 secret access key.

    Note: The above credentials are for AWS. Depending on your IaaS, the credentials might not be a comprehensive list of the keys you need. For example, you might need to set STORAGE_AMAZON_REGION if you are not using using the default region. For more information about configurations, see ChartMuseum Helm Chart in GitHub.

  3. Enable Ingress service access:

    1. Get the annotation information required by your Ingress controller.

      • If you are using your own TLS certificates: Create secrets with TLS certificate data in the same namespace where KSM will be installed:

        kubectl create secret tls daemon-cert --key DAEMON-KEY-FILE --cert DAEMON-CERT-FILE -n  KSM-NAMESPACE
        kubectl create secret tls broker-cert --key BROKER-KEY-FILE --cert BROKER-CERT-FILE -n  KSM-NAMESPACE
        

        Where:

        • KSM-NAMESPACE is the namespace where KSM will be installed.
        • DAEMON-KEY-FILE and DAEMON-CERT-FILE are the paths to your TLS private key and certificate for the daemon.
        • BROKER-KEY-FILE and BROKER-CERT-FILE are the paths to your TLS private key and certificate for the broker.
      • If you are using an automated certificate management provider such as cert-manager: Follow the procedures to install and configure the prerequisites for the certificate management provider you are using.

        For example, the prerequisite for cert-manager is to set up an Issuer on the cluster. To set up an Issuer, see Configuration in the cert-manager documentation.

    2. Add the following to your ksm/values.yaml file:

      ingress:
       enabled: true
       hosts:
       - INGRESS-DOMAIN
       annotations:
         ANNOTATION-KEY: ANNOTATION-VALUE
       tls:
       - secretName: daemon-cert
         hosts:
           - daemon.INGRESS-DOMAIN
       - secretName: broker-cert
         hosts:
           - broker.INGRESS-DOMAIN
      

      Where:

      • INGRESS-DOMAIN is the name of your provisioned domain.
      • ANNOTATION-KEY and ANNOTATION-VALUE is the annotation required by your Ingress controller.

        These values depend on the Ingress controller and certificate management option you use. For example, see the annotations for nginx Ingress Controller and cert-manager:

        annotations:
            kubernetes.io/ingress.class: nginx
            cert-manager.io/issuer: "letsencrypt-prod"
        
  4. Define a secure password to authenticate your services by adding:

    broker:
     password: BROKER-PASSWORD
    daemon:
     password: DAEMON-PASSWORD
    chartmuseum:
     env:
       open:
         BASIC_AUTH_PASS: CHARTMUSEUM-PASSWORD
    postgresql:
     postgresqlPassword: POSTGRESQL-PASSWORD
     encryptionKey: POSTGRESQL-ENCRYPTION-KEY
    

    Where:

    • BROKER-PASSWORD is a secure password for the KSM broker.
    • DAEMON-PASSWORD is a secure password for the KSM daemon.
    • CHARTMUSEUM-PASSWORD is a secure password for ChartMuseum.
    • POSTGRESQL-PASSWORD is a secure password for PostgreSQL.
    • POSTGRESQL-ENCRYPTION-KEY is an encryption key that is at least 12 characters.
  5. Define values for your registry by adding:

    broker:
     image:
       repository: REGISTRY/container-services-manager/broker
    daemon:
     image:
       repository: REGISTRY/container-services-manager/daemon
    chartmuseum:
     image:
       repository: REGISTRY/container-services-manager/chartmuseum
    postgresql:
     image:
       registry: REGISTRY
    

    Where REGISTRY is your private container image registry.

  6. Add the credentials for the registry where the KSM images are by adding:

    imageCredentialsForKSMImages:
     registry: REGISTRY
     username: REGISTRY-USERNAME
     password: REGISTRY-PASSWORD
    

    Where:

    • REGISTRY is the registry you configured for installation images.
    • REGISTRY-USERNAME is the username for the registry.
    • REGISTRY-PASSWORD is the password for the registry.

    KSM is using this registry for:

    • KSM installation docker images: a new secret named registrySecretName of type dockerconfigjson is created with these credentials.
  7. Add the credentials for the registry where the service instance images will come from:

    imageCredentialsForServiceInstances:
     registry: REGISTRY-INSTANCES
     username: REGISTRY-INSTANCES-USERNAME
     password: REGISTRY-INSTANCES-PASSWORD
    

    Where:

    • REGISTRY-INSTANCES is the registry you configured to offer images.
    • REGISTRY-INSTANCES-USERNAME is the username for the registry.
    • REGISTRY-INSTANCES-PASSWORD is the password for the registry.

    KSM is using this registry for:

    • The backing registry for the services that KSM deploys: KSM modifies the Helm charts that you offer to point to images in the registry.

    Note: Although this configuration is optional, VMware recommends using a private container registry in production.

  8. Add your Cloud Foundry environment by adding:

    cf:
     apiAddress: http://api.SYSTEM-DOMAIN
     username: CF-USERNAME
     password: CF-PASSWORD
     brokerName: ksm
     brokerUrl: http://broker.INGRESS-DOMAIN
    

    Where:

    • SYSTEM-DOMAIN is the system domain for TAS for VMs.
    • CF-USERNAME is the username for a TAS for VMs account with cloud_controller.admin permissions.
    • CF-PASSWORD is the password for the TAS for VMs account.
    • INGRESS-DOMAIN is the name of your provisioned domain.
  9. Specify which Postgres database to use:

    • Internal Postgres Configuration: VMware does not recommend internally configuring Postgres for production environments.

      postgresql:
        enabled: true
        image:
          registry: REGISTRY
          repository: container-services-manager/postgresql
          tag: latest
        serviceAccount:
          enabled: true
        postgresqlUsername: USERNAME
        postgresqlPassword: PASSWORD
        postgresqlDatabase: DATABASE
        encryptionKey: ENCRYPTION-KEY
      
    • External Postgres configuration:

      postgresql:
        enabled: false
        postgresqlUsername: USERNAME
        postgresqlPassword: PASSWORD
        postgresqlDatabase: DATABASE
        encryptionKey: ENCRYPTION-KEY
        host: HOST
        dbSkipSSL: DB-SKIP-SSL
        caCert: |
          CA-CERT
      

      Where:

      • REGISTRY is the registry location for the TAS for VMs image.
      • USERNAME is the PostgresSQL username.
      • PASSWORD is the PostgresSQL password.
      • DATABASE is the schema within PostgresSQL where the tables for KSM are located.
      • ENCRYPTION-KEY is used to symmetrically encrypt sensitive data before you save it in PostgresSQL.
      • HOST is the domain for the database.
      • DP-SKIP-SSL can be set to true when an internal database is used. SSL or TLS should be used through service mesh.
      • CA-CERT is the SSL or TLS certificate authority (CA) certificate for the PostgresSQL database connection. This certificate is required when dbSkipSSL is false.

Install the Helm Chart

To install the Helm chart:

  1. From the root level of the chart, install the Helm chart by running these commands:

    kubectl create ns KSM-NAMESPACE
    helm install RELEASE-NAME . -n KSM-NAMESPACE --wait
    

    Where:

    • RELEASE-NAME is a name you choose for the release.
    • KSM-NAMESPACE is a name you choose for the KSM dedicated namespace.
  2. If are using a load balancer:

    1. Retrieve the broker IP address by running:

      export BROKER_IP=$(kubectl get service ksm-ksm-broker -o=jsonpath='{@.status.loadBalancer.ingress[0].ip}')
      
    2. Upgrade your Helm release by running:

      helm upgrade RELEASE-NAME . --reuse-values \
        --set cf.brokerUrl="http://${BROKER_IP}" \
        --set cf.brokerName=ksm \
        --set cf.apiAddress=http://api.SYSTEM-DOMAIN \
        --set cf.username=CF-USERNAME \
        --set cf.password=CF-PASSWORD \
        -n KSM-NAMESPACE
      

      Where:

      • RELEASE-NAME is the name of the release.
      • SYSTEM-DOMAIN is the system domain for TAS for VMs.
      • CF-USERNAME is the username for a TAS for VMs account with cloud_controller.admin permissions.
      • CF-PASSWORD is the password for the TAS for VMs account.
      • KSM-NAMESPACE is a name you choose for the KSM dedicated namespace.

      The above commands add your Cloud Foundry environment to KSM.

Configure Security

VMware recommends configuring security on your Kubernetes cluster for KSM.

To configure security:

  1. Secure KSM secrets by using a secret provider. See Encrypting Secret Data at Rest in the Kubernetes documentation.
  2. Enable network policies on the cluster to secure traffic between services. See Network Policies in the Kubernetes documentation. Some settings can vary between clouds. For example, in GKE, network policies are not enabled by default. For more information, see your cloud-specific documentation.
  3. Secure traffic between KSM components. For example, you can secure traffic using a service mesh such as Linkerd. For instructions on how to use Linkerd, see Configure Service Mesh below.

Configure Service Mesh

To configure a Linkerd service mesh:

  1. Install Linkerd on your Kubernetes cluster using one of the following methods:

    • Install the service mesh using the Linkerd CLI by following the procedures in Getting Started in the Linkerd documentation.
    • Install the service mesh using a Helm chart by following the procedures in Installing Linkerd with Helm in the Linkerd documentation.
  2. Add the following annotations in the KSM Helm chart values.yaml file:

    broker:
     annotations:
       linkerd.io/inject: enabled
    daemon:
     annotations:
       linkerd.io/inject: enabled
    chartmuseum:
     replica:
       annotations:
         linkerd.io/inject: enabled
     daemon:
       annotations:
         linkerd.io/inject: enabled
     chartmuseum:
       replica:
         annotations:
           linkerd.io/inject: enabled
     postgresql:
       master:
         podAnnotations:
           linkerd.io/inject: enabled
       slave:
         podAnnotations:
           linkerd.io/inject: enabled
     minio:
       podAnnotations:
         linkerd.io/inject: enabled
    
  3. Ensure you have the correct permissions to run linkerd tap on the cluster by following the procedures in the Securing Your Cluster in the Linkerd documentation.

  4. Verify that KSM is added to the mesh by running:

    linkerd stat deployments -n KSM-NAMESPACE
    

    Where KSM-NAMESPACE is the namespace where KSM is running.

  5. Verify that TLS is working on the deployment by running:

    linkerd -n KSM-NAMESPACE tap deploy/DAEMON-DEPLOYMENT
    

    Where DAEMON-DEPLOYMENT is the deployment for the daemon. For example, RELEASE-NAME-ksm-daemon.

(Optional) Install Prometheus

You can view metrics for KSM if you have Prometheus running in the cluster. You must install Prometheus in each cluster you want to view metrics for.

To install Prometheus to a cluster:

  1. Install the Prometheus Helm chart by running these commands:

    kubectl create ns prometheus
    
    helm install prometheus stable/prometheus -n prometheus
    
  2. Create a Kubernetes port forward to your local host by running these commands:

    export POD_NAME=$(kubectl get pods --namespace prometheus -l "app=prometheus,component=server" -o jsonpath="{.items[0].metadata.name}")
    
    kubectl --namespace prometheus port-forward $POD_NAME 9090
    
  3. Access the Prometheus UI in your web browser at http://localhost:9090.

  4. To view metrics for KSM, type {app_kubernetes_io_name="ksm"} in the expression box and click Execute.

Next Steps

After installing and configuring KSM, follow the procedures below: